[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:02:59.665519 27934 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.71.190:39055
I20260709 13:02:59.667249 27934 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:02:59.668377 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:59.686522 27941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:59.686846 27943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:59.688616 27934 server_base.cc:1061] running on GCE node
W20260709 13:02:59.686925 27940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:00.946097 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:00.946379 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:00.946666 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602180946618 us; error 0 us; skew 500 ppm
I20260709 13:03:00.947569 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:00.954586 27934 webserver.cc:533] Webserver started at http://127.27.71.190:40999/ using document root <none> and password file <none>
I20260709 13:03:00.955569 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:00.955789 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:00.956393 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:00.963450 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/master-0-root/instance:
uuid: "fbfc8dd108034ea6b31953119fd5ab54"
format_stamp: "Formatted at 2026-07-09 13:03:00 on dist-test-slave-mlt7"
I20260709 13:03:00.973428 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:03:00.979998 27949 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:00.981799 27934 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.001s
I20260709 13:03:00.982602 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/master-0-root
uuid: "fbfc8dd108034ea6b31953119fd5ab54"
format_stamp: "Formatted at 2026-07-09 13:03:00 on dist-test-slave-mlt7"
I20260709 13:03:00.983071 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:01.053057 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:01.054857 27934 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:03:01.055392 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:01.144383 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.190:39055
I20260709 13:03:01.144510 28000 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.190:39055 every 8 connection(s)
I20260709 13:03:01.154577 28001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:01.184476 28001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54: Bootstrap starting.
I20260709 13:03:01.193200 28001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:01.195501 28001 log.cc:826] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54: Log is configured to *not* fsync() on all Append() calls
I20260709 13:03:01.201198 28001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54: No bootstrap required, opened a new log
I20260709 13:03:01.226923 28001 raft_consensus.cc:359] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbfc8dd108034ea6b31953119fd5ab54" member_type: VOTER }
I20260709 13:03:01.227569 28001 raft_consensus.cc:385] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:01.227846 28001 raft_consensus.cc:740] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbfc8dd108034ea6b31953119fd5ab54, State: Initialized, Role: FOLLOWER
I20260709 13:03:01.228992 28001 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbfc8dd108034ea6b31953119fd5ab54" member_type: VOTER }
I20260709 13:03:01.229550 28001 raft_consensus.cc:399] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:03:01.229842 28001 raft_consensus.cc:493] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:03:01.230172 28001 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:01.235249 28001 raft_consensus.cc:515] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbfc8dd108034ea6b31953119fd5ab54" member_type: VOTER }
I20260709 13:03:01.236110 28001 leader_election.cc:304] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbfc8dd108034ea6b31953119fd5ab54; no voters: 
I20260709 13:03:01.238001 28001 leader_election.cc:290] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:03:01.238373 28004 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:01.240883 28004 raft_consensus.cc:697] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [term 1 LEADER]: Becoming Leader. State: Replica: fbfc8dd108034ea6b31953119fd5ab54, State: Running, Role: LEADER
I20260709 13:03:01.242012 28004 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbfc8dd108034ea6b31953119fd5ab54" member_type: VOTER }
I20260709 13:03:01.243142 28001 sys_catalog.cc:565] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:03:01.253928 28005 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbfc8dd108034ea6b31953119fd5ab54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbfc8dd108034ea6b31953119fd5ab54" member_type: VOTER } }
I20260709 13:03:01.254853 28005 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:01.254779 28006 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbfc8dd108034ea6b31953119fd5ab54. Latest consensus state: current_term: 1 leader_uuid: "fbfc8dd108034ea6b31953119fd5ab54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbfc8dd108034ea6b31953119fd5ab54" member_type: VOTER } }
I20260709 13:03:01.255584 28006 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:01.259102 28013 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:03:01.275547 28013 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:03:01.278978 27934 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:03:01.295809 28013 catalog_manager.cc:1383] Generated new cluster ID: 366bb28db94f4d06bb8d3e63b97f565f
I20260709 13:03:01.296168 28013 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:03:01.323899 28013 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:03:01.326258 28013 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:03:01.348068 28013 catalog_manager.cc:6070] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54: Generated new TSK 0
I20260709 13:03:01.349315 28013 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:03:01.410259 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:01.419536 28023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:01.421568 28024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:01.424288 27934 server_base.cc:1061] running on GCE node
W20260709 13:03:01.424873 28026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:01.426117 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:01.426455 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:01.426678 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602181426654 us; error 0 us; skew 500 ppm
I20260709 13:03:01.427623 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:01.432641 27934 webserver.cc:533] Webserver started at http://127.27.71.129:39445/ using document root <none> and password file <none>
I20260709 13:03:01.433209 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:01.433447 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:01.433753 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:01.434916 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-0-root/instance:
uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
format_stamp: "Formatted at 2026-07-09 13:03:01 on dist-test-slave-mlt7"
I20260709 13:03:01.440910 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.007s
I20260709 13:03:01.445600 28031 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:01.446511 27934 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:03:01.446810 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-0-root
uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
format_stamp: "Formatted at 2026-07-09 13:03:01 on dist-test-slave-mlt7"
I20260709 13:03:01.447119 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:01.469763 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:01.471303 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:01.474347 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:01.480557 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:01.480873 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:01.481243 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:01.481446 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:01.614681 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.129:34675
I20260709 13:03:01.614756 28093 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.129:34675 every 8 connection(s)
I20260709 13:03:01.623180 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:01.636842 28098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:01.640462 28099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:01.651625 28101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:01.651959 27934 server_base.cc:1061] running on GCE node
I20260709 13:03:01.653062 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:01.653934 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:01.654165 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602181654145 us; error 0 us; skew 500 ppm
I20260709 13:03:01.654971 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:01.656245 28094 heartbeater.cc:344] Connected to a master server at 127.27.71.190:39055
I20260709 13:03:01.656800 28094 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:01.658279 28094 heartbeater.cc:507] Master 127.27.71.190:39055 requested a full tablet report, sending...
I20260709 13:03:01.662104 27934 webserver.cc:533] Webserver started at http://127.27.71.130:43379/ using document root <none> and password file <none>
I20260709 13:03:01.662365 27966 ts_manager.cc:194] Registered new tserver with Master: 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675)
I20260709 13:03:01.662767 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:01.663029 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:01.663338 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:01.665035 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-1-root/instance:
uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
format_stamp: "Formatted at 2026-07-09 13:03:01 on dist-test-slave-mlt7"
I20260709 13:03:01.666014 27966 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36858
I20260709 13:03:01.671531 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:03:01.676433 28106 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:01.677431 27934 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:03:01.677779 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-1-root
uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
format_stamp: "Formatted at 2026-07-09 13:03:01 on dist-test-slave-mlt7"
I20260709 13:03:01.678120 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:01.707686 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:01.709067 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:01.710670 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:01.713735 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:01.713930 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:01.714231 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:01.714395 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:01.823284 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.130:37771
I20260709 13:03:01.823463 28168 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.130:37771 every 8 connection(s)
I20260709 13:03:01.831497 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:01.840039 28172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:01.846874 28169 heartbeater.cc:344] Connected to a master server at 127.27.71.190:39055
I20260709 13:03:01.847342 28169 heartbeater.cc:461] Registering TS with master...
W20260709 13:03:01.848238 28173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:01.848239 28169 heartbeater.cc:507] Master 127.27.71.190:39055 requested a full tablet report, sending...
I20260709 13:03:01.851612 27966 ts_manager.cc:194] Registered new tserver with Master: 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:01.851912 27934 server_base.cc:1061] running on GCE node
W20260709 13:03:01.851739 28175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:01.853657 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:03:01.853703 27966 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36874
W20260709 13:03:01.854154 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:01.854382 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602181854364 us; error 0 us; skew 500 ppm
I20260709 13:03:01.855191 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:01.858143 27934 webserver.cc:533] Webserver started at http://127.27.71.131:36823/ using document root <none> and password file <none>
I20260709 13:03:01.858793 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:01.859041 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:01.859381 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:01.860839 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-2-root/instance:
uuid: "b0087b2c108442708a47eb31a1a17cda"
format_stamp: "Formatted at 2026-07-09 13:03:01 on dist-test-slave-mlt7"
I20260709 13:03:01.866868 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:03:01.871886 28180 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:01.873008 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:03:01.873311 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-2-root
uuid: "b0087b2c108442708a47eb31a1a17cda"
format_stamp: "Formatted at 2026-07-09 13:03:01 on dist-test-slave-mlt7"
I20260709 13:03:01.873628 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602179482400-27934-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:01.886930 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:01.888561 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:01.890431 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:01.893242 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:01.893555 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:01.893850 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:01.894033 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:01.996592 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.131:35523
I20260709 13:03:01.996739 28242 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.131:35523 every 8 connection(s)
I20260709 13:03:02.019413 28243 heartbeater.cc:344] Connected to a master server at 127.27.71.190:39055
I20260709 13:03:02.019901 28243 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:02.021021 28243 heartbeater.cc:507] Master 127.27.71.190:39055 requested a full tablet report, sending...
I20260709 13:03:02.023378 27966 ts_manager.cc:194] Registered new tserver with Master: b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:02.023479 27934 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.021444636s
I20260709 13:03:02.024962 27966 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36888
I20260709 13:03:02.090606 27966 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36900:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:03:02.094776 27966 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:03:02.214061 28208 tablet_service.cc:1511] Processing CreateTablet for tablet a71bcaa524ed40b8aed6076294fbe947 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:03:02.214699 28206 tablet_service.cc:1511] Processing CreateTablet for tablet 08508a2dc83347879ae38bb7afd65ac8 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:03:02.217322 28207 tablet_service.cc:1511] Processing CreateTablet for tablet ef5db39115cb4b8cba7eee65c5479020 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:03:02.219974 28205 tablet_service.cc:1511] Processing CreateTablet for tablet eb12b90a406c4f0d83468f73bb6f304c (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:03:02.215808 28208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71bcaa524ed40b8aed6076294fbe947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.223237 28205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb12b90a406c4f0d83468f73bb6f304c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.228462 28207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef5db39115cb4b8cba7eee65c5479020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.228535 28133 tablet_service.cc:1511] Processing CreateTablet for tablet ef5db39115cb4b8cba7eee65c5479020 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:03:02.231312 28134 tablet_service.cc:1511] Processing CreateTablet for tablet a71bcaa524ed40b8aed6076294fbe947 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:03:02.233803 28131 tablet_service.cc:1511] Processing CreateTablet for tablet eb12b90a406c4f0d83468f73bb6f304c (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:03:02.235735 28206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08508a2dc83347879ae38bb7afd65ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.229871 28132 tablet_service.cc:1511] Processing CreateTablet for tablet 08508a2dc83347879ae38bb7afd65ac8 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:03:02.237181 28130 tablet_service.cc:1511] Processing CreateTablet for tablet c2bdcea6a1734755a656586304a1eef1 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:03:02.243129 28204 tablet_service.cc:1511] Processing CreateTablet for tablet c2bdcea6a1734755a656586304a1eef1 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:03:02.245023 28204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2bdcea6a1734755a656586304a1eef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.245666 28133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef5db39115cb4b8cba7eee65c5479020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.248178 28131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb12b90a406c4f0d83468f73bb6f304c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.250078 28129 tablet_service.cc:1511] Processing CreateTablet for tablet fda99a8c4108494bac452ede28bb16c8 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:03:02.258934 28134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71bcaa524ed40b8aed6076294fbe947. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:02.263063 28014 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:02.267736 28132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08508a2dc83347879ae38bb7afd65ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.268793 28129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fda99a8c4108494bac452ede28bb16c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.287483 28130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2bdcea6a1734755a656586304a1eef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.305964 28059 tablet_service.cc:1511] Processing CreateTablet for tablet a71bcaa524ed40b8aed6076294fbe947 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:03:02.301555 28057 tablet_service.cc:1511] Processing CreateTablet for tablet 08508a2dc83347879ae38bb7afd65ac8 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:03:02.306978 28055 tablet_service.cc:1511] Processing CreateTablet for tablet c2bdcea6a1734755a656586304a1eef1 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:03:02.309517 28058 tablet_service.cc:1511] Processing CreateTablet for tablet ef5db39115cb4b8cba7eee65c5479020 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:03:02.311321 28057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08508a2dc83347879ae38bb7afd65ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.313645 28054 tablet_service.cc:1511] Processing CreateTablet for tablet fda99a8c4108494bac452ede28bb16c8 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:03:02.297922 28056 tablet_service.cc:1511] Processing CreateTablet for tablet eb12b90a406c4f0d83468f73bb6f304c (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:03:02.311769 28053 tablet_service.cc:1511] Processing CreateTablet for tablet 2e624d08cc1249649be7f2c957496145 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:03:02.304033 28203 tablet_service.cc:1511] Processing CreateTablet for tablet fda99a8c4108494bac452ede28bb16c8 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:03:02.319775 28058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef5db39115cb4b8cba7eee65c5479020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.324469 28054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fda99a8c4108494bac452ede28bb16c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.333994 28056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb12b90a406c4f0d83468f73bb6f304c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.322376 28203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fda99a8c4108494bac452ede28bb16c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.343122 28055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2bdcea6a1734755a656586304a1eef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.364434 28053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e624d08cc1249649be7f2c957496145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.372424 28059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a71bcaa524ed40b8aed6076294fbe947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.379818 28128 tablet_service.cc:1511] Processing CreateTablet for tablet 2e624d08cc1249649be7f2c957496145 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:03:02.382755 28128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e624d08cc1249649be7f2c957496145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.398489 28202 tablet_service.cc:1511] Processing CreateTablet for tablet 2e624d08cc1249649be7f2c957496145 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:03:02.403759 28202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e624d08cc1249649be7f2c957496145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.420073 28052 tablet_service.cc:1511] Processing CreateTablet for tablet 9208d2ef87e0448ebcfe9acd861208f7 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:03:02.423111 28052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9208d2ef87e0448ebcfe9acd861208f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.452888 28127 tablet_service.cc:1511] Processing CreateTablet for tablet 9208d2ef87e0448ebcfe9acd861208f7 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:03:02.458964 28127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9208d2ef87e0448ebcfe9acd861208f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.504371 28051 tablet_service.cc:1511] Processing CreateTablet for tablet 14fc44d49558470399e63e7d5c26d94f (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:03:02.510708 28051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14fc44d49558470399e63e7d5c26d94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.529758 28205 tablet_service.cc:1511] Processing CreateTablet for tablet 14fc44d49558470399e63e7d5c26d94f (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:03:02.583654 28205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14fc44d49558470399e63e7d5c26d94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.524998 28261 tablet_bootstrap.cc:492] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda: Bootstrap starting.
I20260709 13:03:02.525529 28262 tablet_bootstrap.cc:492] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Bootstrap starting.
I20260709 13:03:02.518288 28207 tablet_service.cc:1511] Processing CreateTablet for tablet 9208d2ef87e0448ebcfe9acd861208f7 (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:03:02.600517 28207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9208d2ef87e0448ebcfe9acd861208f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.593801 28263 tablet_bootstrap.cc:492] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd: Bootstrap starting.
I20260709 13:03:02.607357 28261 tablet_bootstrap.cc:654] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.593750 28128 tablet_service.cc:1511] Processing CreateTablet for tablet 14fc44d49558470399e63e7d5c26d94f (DEFAULT_TABLE table=test-workload [id=d4e8b66c725445b587d0e48185b2d556]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:03:02.615028 28128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14fc44d49558470399e63e7d5c26d94f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.619093 28261 tablet_bootstrap.cc:492] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda: No bootstrap required, opened a new log
I20260709 13:03:02.619612 28261 ts_tablet_manager.cc:1403] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda: Time spent bootstrapping tablet: real 0.129s	user 0.007s	sys 0.012s
I20260709 13:03:02.623834 28261 raft_consensus.cc:359] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:02.624969 28261 raft_consensus.cc:385] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.625298 28261 raft_consensus.cc:740] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.626124 28261 consensus_queue.cc:260] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:02.637097 28243 heartbeater.cc:499] Master 127.27.71.190:39055 was elected leader, sending a full tablet report...
I20260709 13:03:02.638787 28261 ts_tablet_manager.cc:1434] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.009s
I20260709 13:03:02.640139 28262 tablet_bootstrap.cc:654] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.640300 28261 tablet_bootstrap.cc:492] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda: Bootstrap starting.
I20260709 13:03:02.648684 28263 tablet_bootstrap.cc:654] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.661119 28262 tablet_bootstrap.cc:492] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657: No bootstrap required, opened a new log
I20260709 13:03:02.661732 28262 ts_tablet_manager.cc:1403] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent bootstrapping tablet: real 0.165s	user 0.014s	sys 0.009s
I20260709 13:03:02.670100 28094 heartbeater.cc:499] Master 127.27.71.190:39055 was elected leader, sending a full tablet report...
I20260709 13:03:02.672219 28265 raft_consensus.cc:493] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.673674 28265 raft_consensus.cc:515] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:02.677105 28265 leader_election.cc:290] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:02.687980 28263 tablet_bootstrap.cc:492] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd: No bootstrap required, opened a new log
I20260709 13:03:02.688633 28263 ts_tablet_manager.cc:1403] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent bootstrapping tablet: real 0.096s	user 0.013s	sys 0.009s
I20260709 13:03:02.678205 28261 tablet_bootstrap.cc:654] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.685495 28262 raft_consensus.cc:359] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.694953 28262 raft_consensus.cc:385] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.695348 28262 raft_consensus.cc:740] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.696386 28262 consensus_queue.cc:260] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.699074 28263 raft_consensus.cc:359] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.704036 28263 raft_consensus.cc:385] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.704495 28263 raft_consensus.cc:740] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.710330 28263 consensus_queue.cc:260] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.712411 28169 heartbeater.cc:499] Master 127.27.71.190:39055 was elected leader, sending a full tablet report...
I20260709 13:03:02.724367 28262 ts_tablet_manager.cc:1434] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.010s
I20260709 13:03:02.725024 28144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" is_pre_election: true
I20260709 13:03:02.726130 28262 tablet_bootstrap.cc:492] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Bootstrap starting.
I20260709 13:03:02.726599 28261 tablet_bootstrap.cc:492] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda: No bootstrap required, opened a new log
I20260709 13:03:02.727057 28261 ts_tablet_manager.cc:1403] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda: Time spent bootstrapping tablet: real 0.087s	user 0.011s	sys 0.006s
I20260709 13:03:02.728005 28263 ts_tablet_manager.cc:1434] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent starting tablet: real 0.039s	user 0.012s	sys 0.003s
W20260709 13:03:02.729159 28182 leader_election.cc:343] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:02.731113 28263 tablet_bootstrap.cc:492] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd: Bootstrap starting.
I20260709 13:03:02.733342 28262 tablet_bootstrap.cc:654] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.730724 28261 raft_consensus.cc:359] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.735622 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" is_pre_election: true
I20260709 13:03:02.736109 28261 raft_consensus.cc:385] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.736454 28261 raft_consensus.cc:740] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Initialized, Role: FOLLOWER
W20260709 13:03:02.737782 28181 leader_election.cc:343] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:02.737434 28261 consensus_queue.cc:260] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.738240 28181 leader_election.cc:304] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0087b2c108442708a47eb31a1a17cda; no voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:02.741312 28261 ts_tablet_manager.cc:1434] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:03:02.742386 28261 tablet_bootstrap.cc:492] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda: Bootstrap starting.
I20260709 13:03:02.745743 28265 raft_consensus.cc:2749] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:02.748245 28263 tablet_bootstrap.cc:654] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:02.757174 28244 tablet.cc:2406] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:03:02.763556 28261 tablet_bootstrap.cc:654] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.764729 28262 tablet_bootstrap.cc:492] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657: No bootstrap required, opened a new log
I20260709 13:03:02.765264 28262 ts_tablet_manager.cc:1403] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.017s
I20260709 13:03:02.768338 28262 raft_consensus.cc:359] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.769091 28262 raft_consensus.cc:385] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.769387 28262 raft_consensus.cc:740] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.770109 28262 consensus_queue.cc:260] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.772538 28262 ts_tablet_manager.cc:1434] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:03:02.773437 28262 tablet_bootstrap.cc:492] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657: Bootstrap starting.
I20260709 13:03:02.774173 28263 tablet_bootstrap.cc:492] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd: No bootstrap required, opened a new log
I20260709 13:03:02.774668 28263 ts_tablet_manager.cc:1403] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.008s
I20260709 13:03:02.777732 28263 raft_consensus.cc:359] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.778475 28263 raft_consensus.cc:385] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.778785 28263 raft_consensus.cc:740] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.779475 28263 consensus_queue.cc:260] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.780020 28262 tablet_bootstrap.cc:654] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.781597 28263 ts_tablet_manager.cc:1434] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:03:02.785953 28263 tablet_bootstrap.cc:492] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd: Bootstrap starting.
I20260709 13:03:02.791981 28262 tablet_bootstrap.cc:492] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657: No bootstrap required, opened a new log
I20260709 13:03:02.792511 28262 ts_tablet_manager.cc:1403] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.007s
I20260709 13:03:02.795539 28262 raft_consensus.cc:359] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.796324 28262 raft_consensus.cc:385] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.796648 28262 raft_consensus.cc:740] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.797436 28262 consensus_queue.cc:260] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.798434 28261 tablet_bootstrap.cc:492] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda: No bootstrap required, opened a new log
I20260709 13:03:02.798913 28261 ts_tablet_manager.cc:1403] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.009s
I20260709 13:03:02.799964 28262 ts_tablet_manager.cc:1434] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:02.800884 28262 tablet_bootstrap.cc:492] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Bootstrap starting.
I20260709 13:03:02.801829 28261 raft_consensus.cc:359] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.802583 28261 raft_consensus.cc:385] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.802897 28261 raft_consensus.cc:740] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.804211 28261 consensus_queue.cc:260] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.803738 28263 tablet_bootstrap.cc:654] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.810240 28261 ts_tablet_manager.cc:1434] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:03:02.811239 28261 tablet_bootstrap.cc:492] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda: Bootstrap starting.
I20260709 13:03:02.818895 28263 tablet_bootstrap.cc:492] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd: No bootstrap required, opened a new log
I20260709 13:03:02.819474 28263 ts_tablet_manager.cc:1403] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.007s
I20260709 13:03:02.820286 28261 tablet_bootstrap.cc:654] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.821410 28262 tablet_bootstrap.cc:654] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.822728 28263 raft_consensus.cc:359] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.823619 28263 raft_consensus.cc:385] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.824081 28263 raft_consensus.cc:740] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.825330 28263 consensus_queue.cc:260] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.832095 28263 ts_tablet_manager.cc:1434] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:03:02.832245 28261 tablet_bootstrap.cc:492] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda: No bootstrap required, opened a new log
I20260709 13:03:02.832789 28261 ts_tablet_manager.cc:1403] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.003s
I20260709 13:03:02.833288 28263 tablet_bootstrap.cc:492] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd: Bootstrap starting.
I20260709 13:03:02.835646 28261 raft_consensus.cc:359] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.836437 28261 raft_consensus.cc:385] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.836789 28261 raft_consensus.cc:740] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.837630 28261 consensus_queue.cc:260] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.840545 28261 ts_tablet_manager.cc:1434] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:02.841536 28263 tablet_bootstrap.cc:654] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.841585 28261 tablet_bootstrap.cc:492] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda: Bootstrap starting.
I20260709 13:03:02.847276 28262 tablet_bootstrap.cc:492] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: No bootstrap required, opened a new log
I20260709 13:03:02.847827 28262 ts_tablet_manager.cc:1403] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.008s
I20260709 13:03:02.851514 28261 tablet_bootstrap.cc:654] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.851282 28262 raft_consensus.cc:359] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.857053 28262 raft_consensus.cc:385] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.857542 28262 raft_consensus.cc:740] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.858884 28262 consensus_queue.cc:260] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.862432 28262 ts_tablet_manager.cc:1434] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:03:02.864055 28262 tablet_bootstrap.cc:492] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Bootstrap starting.
I20260709 13:03:02.865092 28263 tablet_bootstrap.cc:492] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd: No bootstrap required, opened a new log
I20260709 13:03:02.865640 28263 ts_tablet_manager.cc:1403] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.018s
I20260709 13:03:02.869051 28263 raft_consensus.cc:359] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:02.869817 28263 raft_consensus.cc:385] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.870101 28263 raft_consensus.cc:740] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.871236 28262 tablet_bootstrap.cc:654] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.870807 28263 consensus_queue.cc:260] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:02.873469 28263 ts_tablet_manager.cc:1434] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:02.874490 28263 tablet_bootstrap.cc:492] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd: Bootstrap starting.
I20260709 13:03:02.885007 28261 tablet_bootstrap.cc:492] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda: No bootstrap required, opened a new log
I20260709 13:03:02.885586 28261 ts_tablet_manager.cc:1403] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.003s
I20260709 13:03:02.888221 28263 tablet_bootstrap.cc:654] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.889122 28261 raft_consensus.cc:359] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.889958 28261 raft_consensus.cc:385] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.890316 28261 raft_consensus.cc:740] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.891458 28261 consensus_queue.cc:260] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.894411 28262 tablet_bootstrap.cc:492] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: No bootstrap required, opened a new log
I20260709 13:03:02.895128 28262 ts_tablet_manager.cc:1403] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent bootstrapping tablet: real 0.031s	user 0.022s	sys 0.008s
I20260709 13:03:02.900115 28262 raft_consensus.cc:359] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.900892 28261 ts_tablet_manager.cc:1434] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:03:02.901434 28262 raft_consensus.cc:385] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.901958 28262 raft_consensus.cc:740] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.903471 28262 consensus_queue.cc:260] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.909235 28263 tablet_bootstrap.cc:492] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd: No bootstrap required, opened a new log
I20260709 13:03:02.909642 28262 ts_tablet_manager.cc:1434] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.003s
I20260709 13:03:02.909782 28263 ts_tablet_manager.cc:1403] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.015s
I20260709 13:03:02.910854 28262 tablet_bootstrap.cc:492] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Bootstrap starting.
I20260709 13:03:02.915391 28261 tablet_bootstrap.cc:492] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda: Bootstrap starting.
I20260709 13:03:02.913528 28263 raft_consensus.cc:359] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.911412 28270 raft_consensus.cc:493] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.920463 28270 raft_consensus.cc:515] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.922578 28262 tablet_bootstrap.cc:654] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.923753 28270 leader_election.cc:290] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:02.924432 28263 raft_consensus.cc:385] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.925048 28263 raft_consensus.cc:740] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.926347 28263 consensus_queue.cc:260] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.932092 28263 ts_tablet_manager.cc:1434] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.002s
I20260709 13:03:02.933172 28263 tablet_bootstrap.cc:492] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd: Bootstrap starting.
I20260709 13:03:02.933624 28261 tablet_bootstrap.cc:654] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.943676 28263 tablet_bootstrap.cc:654] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.948787 28262 tablet_bootstrap.cc:492] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657: No bootstrap required, opened a new log
I20260709 13:03:02.950769 28262 ts_tablet_manager.cc:1403] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.008s
I20260709 13:03:02.955567 28272 raft_consensus.cc:493] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.956346 28272 raft_consensus.cc:515] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.961616 28272 leader_election.cc:290] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:02.973464 28262 raft_consensus.cc:359] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.975214 28262 raft_consensus.cc:385] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.981940 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" is_pre_election: true
I20260709 13:03:02.982798 28069 raft_consensus.cc:2468] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 0.
I20260709 13:03:02.984916 28263 tablet_bootstrap.cc:492] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd: No bootstrap required, opened a new log
I20260709 13:03:02.985509 28263 ts_tablet_manager.cc:1403] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.021s
I20260709 13:03:02.986791 28261 tablet_bootstrap.cc:492] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda: No bootstrap required, opened a new log
I20260709 13:03:02.987258 28261 ts_tablet_manager.cc:1403] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda: Time spent bootstrapping tablet: real 0.072s	user 0.021s	sys 0.001s
I20260709 13:03:02.988617 28107 leader_election.cc:304] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: 
I20260709 13:03:02.979395 28262 raft_consensus.cc:740] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.989671 28270 raft_consensus.cc:2804] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:02.990047 28270 raft_consensus.cc:493] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.990377 28270 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:02.990235 28262 consensus_queue.cc:260] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.988977 28263 raft_consensus.cc:359] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:02.992103 28263 raft_consensus.cc:385] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.992532 28263 raft_consensus.cc:740] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.996994 28144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08508a2dc83347879ae38bb7afd65ac8" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" is_pre_election: true
I20260709 13:03:02.998497 28144 raft_consensus.cc:2468] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 0.
I20260709 13:03:03.000535 28261 raft_consensus.cc:359] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.001294 28261 raft_consensus.cc:385] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.001623 28261 raft_consensus.cc:740] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.997051 28263 consensus_queue.cc:260] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.004917 28218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda" is_pre_election: true
I20260709 13:03:03.004443 28261 consensus_queue.cc:260] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.006402 28033 leader_election.cc:304] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: 
I20260709 13:03:03.006522 28218 raft_consensus.cc:2468] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 0.
I20260709 13:03:03.010125 28272 raft_consensus.cc:2804] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:03.009796 28270 raft_consensus.cc:515] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.010573 28272 raft_consensus.cc:493] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.012650 28270 leader_election.cc:290] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.015108 28272 raft_consensus.cc:3060] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.014562 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:03.016451 28069 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.018028 28218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:03.022548 28218 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.026471 28263 ts_tablet_manager.cc:1434] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent starting tablet: real 0.041s	user 0.001s	sys 0.008s
I20260709 13:03:03.027638 28263 tablet_bootstrap.cc:492] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd: Bootstrap starting.
I20260709 13:03:03.041354 28262 ts_tablet_manager.cc:1434] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent starting tablet: real 0.090s	user 0.007s	sys 0.009s
I20260709 13:03:03.043218 28261 ts_tablet_manager.cc:1434] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda: Time spent starting tablet: real 0.056s	user 0.005s	sys 0.006s
I20260709 13:03:03.049237 28272 raft_consensus.cc:515] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.054071 28272 leader_election.cc:290] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 election: Requested vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.052758 28069 raft_consensus.cc:2468] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 1.
I20260709 13:03:03.057430 28107 leader_election.cc:304] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: 
I20260709 13:03:03.055763 28262 tablet_bootstrap.cc:492] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Bootstrap starting.
I20260709 13:03:03.060500 28261 tablet_bootstrap.cc:492] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda: Bootstrap starting.
I20260709 13:03:03.060974 28293 raft_consensus.cc:2804] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:03.060482 28218 raft_consensus.cc:2468] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 1.
I20260709 13:03:03.060106 28272 raft_consensus.cc:493] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.061589 28293 raft_consensus.cc:697] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Becoming Leader. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:03.063235 28272 raft_consensus.cc:515] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
W20260709 13:03:03.059257 28033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:37771, user_credentials={real_user=slave}} blocked reactor thread for 54323.7us
I20260709 13:03:03.064306 28293 consensus_queue.cc:237] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.072544 28144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08508a2dc83347879ae38bb7afd65ac8" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:03.073570 28144 raft_consensus.cc:3060] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.075055 28272 leader_election.cc:290] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.075251 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e624d08cc1249649be7f2c957496145" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" is_pre_election: true
I20260709 13:03:03.077340 28143 raft_consensus.cc:2468] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 0.
I20260709 13:03:03.095817 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08508a2dc83347879ae38bb7afd65ac8" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:03.096102 28263 tablet_bootstrap.cc:654] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.096947 28217 raft_consensus.cc:3060] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.109392 28216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e624d08cc1249649be7f2c957496145" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda" is_pre_election: true
W20260709 13:03:03.112118 28032 leader_election.cc:343] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:03.113122 28033 leader_election.cc:304] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: b0087b2c108442708a47eb31a1a17cda
I20260709 13:03:03.114044 28272 raft_consensus.cc:2804] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:03.114494 28272 raft_consensus.cc:493] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.114907 28272 raft_consensus.cc:3060] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.115552 28218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08508a2dc83347879ae38bb7afd65ac8" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda" is_pre_election: true
I20260709 13:03:03.121927 28144 raft_consensus.cc:2468] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 1.
I20260709 13:03:03.122314 27965 catalog_manager.cc:5697] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.124984 28261 tablet_bootstrap.cc:654] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.126439 28033 leader_election.cc:304] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: 
I20260709 13:03:03.122318 28272 raft_consensus.cc:515] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.127620 28294 raft_consensus.cc:2804] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:03.129128 28272 leader_election.cc:290] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 election: Requested vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.132257 28144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e624d08cc1249649be7f2c957496145" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:03.132999 28144 raft_consensus.cc:3060] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.132282 28262 tablet_bootstrap.cc:654] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.140090 28218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e624d08cc1249649be7f2c957496145" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
W20260709 13:03:03.141667 28032 leader_election.cc:343] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:03.142165 28263 tablet_bootstrap.cc:492] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd: No bootstrap required, opened a new log
I20260709 13:03:03.142764 28263 ts_tablet_manager.cc:1403] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent bootstrapping tablet: real 0.115s	user 0.017s	sys 0.009s
I20260709 13:03:03.127657 28217 raft_consensus.cc:2468] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 1.
I20260709 13:03:03.145718 28144 raft_consensus.cc:2468] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 1.
I20260709 13:03:03.147441 28261 tablet_bootstrap.cc:492] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda: No bootstrap required, opened a new log
I20260709 13:03:03.148804 28261 ts_tablet_manager.cc:1403] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda: Time spent bootstrapping tablet: real 0.090s	user 0.014s	sys 0.016s
I20260709 13:03:03.148940 28033 leader_election.cc:304] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: b0087b2c108442708a47eb31a1a17cda
I20260709 13:03:03.147706 28263 raft_consensus.cc:359] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.150358 28272 raft_consensus.cc:2804] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:03.153049 28272 raft_consensus.cc:697] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Becoming Leader. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:03.154022 28261 raft_consensus.cc:359] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.154876 28261 raft_consensus.cc:385] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.154202 28272 consensus_queue.cc:237] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.155162 28263 raft_consensus.cc:385] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.155562 28263 raft_consensus.cc:740] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.155887 28294 raft_consensus.cc:697] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Becoming Leader. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:03.156384 28263 consensus_queue.cc:260] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.156810 28294 consensus_queue.cc:237] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.156387 28262 tablet_bootstrap.cc:492] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657: No bootstrap required, opened a new log
I20260709 13:03:03.158290 28262 ts_tablet_manager.cc:1403] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent bootstrapping tablet: real 0.113s	user 0.028s	sys 0.028s
I20260709 13:03:03.163621 28263 ts_tablet_manager.cc:1434] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.003s
I20260709 13:03:03.158876 28261 raft_consensus.cc:740] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.164717 28262 raft_consensus.cc:359] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:03.166465 28262 raft_consensus.cc:385] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.166805 28262 raft_consensus.cc:740] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.166913 28261 consensus_queue.cc:260] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.167541 28262 consensus_queue.cc:260] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:03.170146 28261 ts_tablet_manager.cc:1434] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.004s
I20260709 13:03:03.171377 28261 tablet_bootstrap.cc:492] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda: Bootstrap starting.
I20260709 13:03:03.170230 28262 ts_tablet_manager.cc:1434] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:03:03.172395 28263 tablet_bootstrap.cc:492] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd: Bootstrap starting.
I20260709 13:03:03.174044 28262 tablet_bootstrap.cc:492] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657: Bootstrap starting.
I20260709 13:03:03.178206 28293 raft_consensus.cc:493] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.178747 28293 raft_consensus.cc:515] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.181072 28293 leader_election.cc:290] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.181545 28265 raft_consensus.cc:493] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.182031 28265 raft_consensus.cc:515] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.182847 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" is_pre_election: true
I20260709 13:03:03.183107 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda" is_pre_election: true
I20260709 13:03:03.183585 28069 raft_consensus.cc:2468] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 0.
I20260709 13:03:03.184244 28265 leader_election.cc:290] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:03.183736 28217 raft_consensus.cc:2468] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 0.
I20260709 13:03:03.185647 28144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" is_pre_election: true
I20260709 13:03:03.185988 28107 leader_election.cc:304] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: 
I20260709 13:03:03.186292 28144 raft_consensus.cc:2468] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 0.
I20260709 13:03:03.185120 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" is_pre_election: true
I20260709 13:03:03.187772 28182 leader_election.cc:304] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c7e1f88ab2b4c8393022b5e5d7fb657, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:03.188334 28272 raft_consensus.cc:493] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.187546 28069 raft_consensus.cc:2468] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 0.
I20260709 13:03:03.188648 28265 raft_consensus.cc:2804] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:03.188657 28293 raft_consensus.cc:2804] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:03.189311 28265 raft_consensus.cc:493] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.189172 28272 raft_consensus.cc:515] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:03.189478 28293 raft_consensus.cc:493] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.190918 28293 raft_consensus.cc:3060] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.193764 28263 tablet_bootstrap.cc:654] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.194959 28144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" is_pre_election: true
I20260709 13:03:03.196313 28261 tablet_bootstrap.cc:654] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.196408 28144 raft_consensus.cc:2468] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 0.
I20260709 13:03:03.190145 28265 raft_consensus.cc:3060] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.198863 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda" is_pre_election: true
I20260709 13:03:03.198969 28033 leader_election.cc:304] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: 
I20260709 13:03:03.199681 28217 raft_consensus.cc:2468] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 0.
I20260709 13:03:03.200786 28297 raft_consensus.cc:2804] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:03.201190 28297 raft_consensus.cc:493] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.201538 28297 raft_consensus.cc:3060] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.195494 28272 leader_election.cc:290] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:03.219781 28293 raft_consensus.cc:515] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.213973 28272 raft_consensus.cc:493] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.220052 28297 raft_consensus.cc:515] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:03.227255 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:03.227998 28217 raft_consensus.cc:3060] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.228672 28293 leader_election.cc:290] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.221756 28272 raft_consensus.cc:515] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.230729 28272 leader_election.cc:290] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.226817 27963 catalog_manager.cc:5697] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129). New cstate: current_term: 1 leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.235000 28262 tablet_bootstrap.cc:654] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.231438 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:03.239341 28069 raft_consensus.cc:3060] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.234544 28218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda" is_pre_election: true
I20260709 13:03:03.246832 28265 raft_consensus.cc:515] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.253968 28265 leader_election.cc:290] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:03.256429 28144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:03.257974 28297 leader_election.cc:290] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 election: Requested vote from peers b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:03.258229 28144 raft_consensus.cc:2393] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b0087b2c108442708a47eb31a1a17cda in current term 1: Already voted for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in this term.
I20260709 13:03:03.250337 28218 raft_consensus.cc:2468] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 0.
I20260709 13:03:03.262008 28217 raft_consensus.cc:2468] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 1.
I20260709 13:03:03.262089 28032 leader_election.cc:304] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:03.262156 28293 raft_consensus.cc:493] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.266053 28272 raft_consensus.cc:2804] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:03.267976 28272 raft_consensus.cc:493] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.267933 28032 leader_election.cc:304] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:03.268512 28272 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.269058 28297 raft_consensus.cc:2804] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:03.269598 28297 raft_consensus.cc:697] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Becoming Leader. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:03.262588 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" is_pre_election: true
I20260709 13:03:03.270980 28143 raft_consensus.cc:2468] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 0.
I20260709 13:03:03.270577 28297 consensus_queue.cc:237] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:03.266263 28293 raft_consensus.cc:515] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.275096 28068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:03.263159 28265 raft_consensus.cc:493] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:03.277151 28014 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:03.276328 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:03.280673 28182 leader_election.cc:304] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0087b2c108442708a47eb31a1a17cda; no voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:03.281422 28069 raft_consensus.cc:2468] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 1.
I20260709 13:03:03.281368 28143 raft_consensus.cc:3060] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.278733 28265 raft_consensus.cc:515] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.300442 28107 leader_election.cc:304] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: 
I20260709 13:03:03.301399 28270 raft_consensus.cc:2804] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:03.311100 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:03.316314 28261 tablet_bootstrap.cc:492] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda: No bootstrap required, opened a new log
I20260709 13:03:03.317042 28261 ts_tablet_manager.cc:1403] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda: Time spent bootstrapping tablet: real 0.146s	user 0.024s	sys 0.027s
I20260709 13:03:03.323751 28308 raft_consensus.cc:2749] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:03.323618 28272 raft_consensus.cc:515] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.325091 28308 raft_consensus.cc:493] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.326467 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" is_pre_election: true
I20260709 13:03:03.327785 28069 raft_consensus.cc:2393] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b0087b2c108442708a47eb31a1a17cda in current term 1: Already voted for candidate 5633fd0a2e6844d7868edd694d2ee9cd in this term.
I20260709 13:03:03.328742 28265 leader_election.cc:290] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:03.330149 27963 catalog_manager.cc:5697] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129). New cstate: current_term: 1 leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.331341 28262 tablet_bootstrap.cc:492] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657: No bootstrap required, opened a new log
I20260709 13:03:03.331948 28262 ts_tablet_manager.cc:1403] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent bootstrapping tablet: real 0.158s	user 0.017s	sys 0.025s
I20260709 13:03:03.332680 28144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:03.336162 28144 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.325608 28308 raft_consensus.cc:515] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.338562 28217 raft_consensus.cc:2393] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in current term 1: Already voted for candidate b0087b2c108442708a47eb31a1a17cda in this term.
I20260709 13:03:03.341149 28263 tablet_bootstrap.cc:492] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd: No bootstrap required, opened a new log
I20260709 13:03:03.341622 28263 ts_tablet_manager.cc:1403] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent bootstrapping tablet: real 0.169s	user 0.015s	sys 0.012s
I20260709 13:03:03.336931 28270 raft_consensus.cc:697] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Becoming Leader. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:03.340998 28142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" is_pre_election: true
I20260709 13:03:03.345398 28263 raft_consensus.cc:359] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.349069 28263 raft_consensus.cc:385] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.349386 28263 raft_consensus.cc:740] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.350056 28263 consensus_queue.cc:260] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.341616 28272 leader_election.cc:290] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 election: Requested vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.352123 28218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:03.352685 28218 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.352734 28308 leader_election.cc:290] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:03.354605 28261 raft_consensus.cc:359] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.355403 28261 raft_consensus.cc:385] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.355728 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" is_pre_election: true
I20260709 13:03:03.356416 28069 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 0.
I20260709 13:03:03.355741 28261 raft_consensus.cc:740] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.350715 28143 raft_consensus.cc:2468] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 1.
I20260709 13:03:03.357805 28181 leader_election.cc:304] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:03.359779 28308 raft_consensus.cc:2804] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:03.360177 28308 raft_consensus.cc:493] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.360502 28308 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.359781 28270 consensus_queue.cc:237] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.362068 28218 raft_consensus.cc:2468] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 1.
I20260709 13:03:03.364123 28182 leader_election.cc:304] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0087b2c108442708a47eb31a1a17cda; no voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:03.373132 28262 raft_consensus.cc:359] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.374197 28262 raft_consensus.cc:385] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.374553 28262 raft_consensus.cc:740] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.375718 28262 consensus_queue.cc:260] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.377547 28311 raft_consensus.cc:2749] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:03.380113 28263 ts_tablet_manager.cc:1434] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:03:03.379853 28144 raft_consensus.cc:2468] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 1.
I20260709 13:03:03.380535 28261 consensus_queue.cc:260] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.381637 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" is_pre_election: true
I20260709 13:03:03.385442 28143 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 0.
I20260709 13:03:03.381044 28263 tablet_bootstrap.cc:492] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd: Bootstrap starting.
I20260709 13:03:03.389863 28033 leader_election.cc:304] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: 
I20260709 13:03:03.390808 28272 raft_consensus.cc:2804] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:03.391274 28272 raft_consensus.cc:697] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Becoming Leader. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:03.382718 28261 ts_tablet_manager.cc:1434] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda: Time spent starting tablet: real 0.065s	user 0.004s	sys 0.004s
I20260709 13:03:03.392086 28272 consensus_queue.cc:237] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.393395 28261 tablet_bootstrap.cc:492] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda: Bootstrap starting.
I20260709 13:03:03.393621 28262 ts_tablet_manager.cc:1434] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.001s
I20260709 13:03:03.394547 28262 tablet_bootstrap.cc:492] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Bootstrap starting.
I20260709 13:03:03.394794 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" is_pre_election: true
I20260709 13:03:03.395448 28069 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 0.
I20260709 13:03:03.396944 28107 leader_election.cc:304] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: 
I20260709 13:03:03.400233 28261 tablet_bootstrap.cc:654] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.401280 28293 leader_election.cc:290] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.403797 28293 raft_consensus.cc:2804] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:03.405606 28293 raft_consensus.cc:493] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.406932 28293 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.411477 28261 tablet_bootstrap.cc:492] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda: No bootstrap required, opened a new log
I20260709 13:03:03.411959 28261 ts_tablet_manager.cc:1403] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 13:03:03.414642 28261 raft_consensus.cc:359] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.415378 28261 raft_consensus.cc:385] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.415663 28261 raft_consensus.cc:740] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.414682 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda" is_pre_election: true
I20260709 13:03:03.417150 28261 consensus_queue.cc:260] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.419450 28261 ts_tablet_manager.cc:1434] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:03:03.419900 27964 catalog_manager.cc:5697] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.423249 28293 raft_consensus.cc:515] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.404695 28262 tablet_bootstrap.cc:654] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.415093 28263 tablet_bootstrap.cc:654] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.425455 28293 leader_election.cc:290] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.430399 28218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:03.433643 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:03.434494 28069 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.435838 28311 raft_consensus.cc:493] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.436589 28311 raft_consensus.cc:515] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.440485 28311 leader_election.cc:290] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:03.440630 28068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e624d08cc1249649be7f2c957496145" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" is_pre_election: true
I20260709 13:03:03.441619 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e624d08cc1249649be7f2c957496145" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" is_pre_election: true
I20260709 13:03:03.442463 28143 raft_consensus.cc:2393] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b0087b2c108442708a47eb31a1a17cda in current term 1: Already voted for candidate 5633fd0a2e6844d7868edd694d2ee9cd in this term.
I20260709 13:03:03.444094 28182 leader_election.cc:304] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0087b2c108442708a47eb31a1a17cda; no voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:03.445349 28311 raft_consensus.cc:3060] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.451256 28263 tablet_bootstrap.cc:492] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd: No bootstrap required, opened a new log
I20260709 13:03:03.451817 28263 ts_tablet_manager.cc:1403] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent bootstrapping tablet: real 0.071s	user 0.009s	sys 0.011s
I20260709 13:03:03.453601 28311 raft_consensus.cc:2749] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:03.454715 28263 raft_consensus.cc:359] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.455606 28263 raft_consensus.cc:385] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.456213 28263 raft_consensus.cc:740] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.459734 28262 tablet_bootstrap.cc:492] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657: No bootstrap required, opened a new log
I20260709 13:03:03.460238 28262 ts_tablet_manager.cc:1403] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent bootstrapping tablet: real 0.066s	user 0.017s	sys 0.011s
I20260709 13:03:03.460070 28263 consensus_queue.cc:260] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.463584 28262 raft_consensus.cc:359] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.464406 28262 raft_consensus.cc:385] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.464685 28262 raft_consensus.cc:740] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.465754 28263 ts_tablet_manager.cc:1434] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:03:03.465437 28262 consensus_queue.cc:260] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.467386 28262 ts_tablet_manager.cc:1434] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:03.472404 27963 catalog_manager.cc:5697] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129). New cstate: current_term: 1 leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.474514 28293 consensus_queue.cc:1048] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:03.483934 28069 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 1.
I20260709 13:03:03.382967 28308 raft_consensus.cc:515] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.486035 28107 leader_election.cc:304] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: b0087b2c108442708a47eb31a1a17cda
I20260709 13:03:03.488556 28308 leader_election.cc:290] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:03.489710 28217 raft_consensus.cc:2393] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in current term 1: Already voted for candidate b0087b2c108442708a47eb31a1a17cda in this term.
I20260709 13:03:03.489741 28312 raft_consensus.cc:2804] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:03.490139 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:03.491453 28312 raft_consensus.cc:697] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Becoming Leader. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:03.491729 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:03.492570 28069 raft_consensus.cc:2393] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b0087b2c108442708a47eb31a1a17cda in current term 1: Already voted for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in this term.
I20260709 13:03:03.492403 28312 consensus_queue.cc:237] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.509075 28182 leader_election.cc:304] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0087b2c108442708a47eb31a1a17cda; no voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:03.510243 28311 raft_consensus.cc:2749] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:03.511768 27965 catalog_manager.cc:5697] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129). New cstate: current_term: 1 leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.520028 27964 catalog_manager.cc:5697] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.528831 28293 consensus_queue.cc:1048] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:03.595062 28272 consensus_queue.cc:1048] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:03.605693 28294 raft_consensus.cc:493] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.606223 28294 raft_consensus.cc:515] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.608409 28294 leader_election.cc:290] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.608805 28270 raft_consensus.cc:493] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.608177 28305 consensus_queue.cc:1048] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:03.609287 28270 raft_consensus.cc:515] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.609493 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fc44d49558470399e63e7d5c26d94f" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda" is_pre_election: true
I20260709 13:03:03.609916 28272 consensus_queue.cc:1048] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:03.610239 28217 raft_consensus.cc:2468] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 0.
I20260709 13:03:03.611454 28270 leader_election.cc:290] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.611945 28032 leader_election.cc:304] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:03.612979 28272 raft_consensus.cc:2804] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:03.613411 28272 raft_consensus.cc:493] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.613807 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9208d2ef87e0448ebcfe9acd861208f7" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda" is_pre_election: true
I20260709 13:03:03.613862 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fc44d49558470399e63e7d5c26d94f" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" is_pre_election: true
I20260709 13:03:03.613952 28305 consensus_queue.cc:1048] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:03.614850 28143 raft_consensus.cc:2468] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 0.
I20260709 13:03:03.613848 28272 raft_consensus.cc:3060] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.615216 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9208d2ef87e0448ebcfe9acd861208f7" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" is_pre_election: true
I20260709 13:03:03.614850 28217 raft_consensus.cc:2468] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 0.
I20260709 13:03:03.617313 28107 leader_election.cc:304] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c7e1f88ab2b4c8393022b5e5d7fb657, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:03.618649 28322 raft_consensus.cc:2804] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:03.620608 28322 raft_consensus.cc:493] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.620966 28322 raft_consensus.cc:3060] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.616989 28069 raft_consensus.cc:2468] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 0.
I20260709 13:03:03.637375 28322 raft_consensus.cc:515] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.639839 28322 leader_election.cc:290] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.643695 28068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9208d2ef87e0448ebcfe9acd861208f7" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:03.644451 28068 raft_consensus.cc:3060] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.648231 28218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9208d2ef87e0448ebcfe9acd861208f7" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:03.646031 28272 raft_consensus.cc:515] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.648839 28218 raft_consensus.cc:3060] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.651881 28272 leader_election.cc:290] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 election: Requested vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:03.652933 28216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fc44d49558470399e63e7d5c26d94f" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:03.653543 28216 raft_consensus.cc:3060] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.655716 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fc44d49558470399e63e7d5c26d94f" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:03.656399 28143 raft_consensus.cc:3060] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.662017 28068 raft_consensus.cc:2468] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 1.
I20260709 13:03:03.663736 28218 raft_consensus.cc:2468] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 1.
I20260709 13:03:03.665259 28107 leader_election.cc:304] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c7e1f88ab2b4c8393022b5e5d7fb657, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:03.666190 28322 raft_consensus.cc:2804] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:03.667027 28322 raft_consensus.cc:697] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Becoming Leader. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:03.668021 28322 consensus_queue.cc:237] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.671320 28143 raft_consensus.cc:2468] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 1.
I20260709 13:03:03.664716 28216 raft_consensus.cc:2468] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 1.
I20260709 13:03:03.675727 28032 leader_election.cc:304] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:03.678133 28272 raft_consensus.cc:2804] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:03.678910 28272 raft_consensus.cc:697] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Becoming Leader. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:03.680168 28272 consensus_queue.cc:237] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:03.696311 27966 catalog_manager.cc:5697] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.703068 27965 catalog_manager.cc:5697] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129). New cstate: current_term: 1 leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:03.711545 28305 consensus_queue.cc:1048] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:03.726449 28272 consensus_queue.cc:1048] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:03.780042 28272 consensus_queue.cc:1048] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:03.795275 28294 consensus_queue.cc:1048] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:03.808377 28322 consensus_queue.cc:1048] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:03.821883 28270 consensus_queue.cc:1048] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:03.890982 28270 consensus_queue.cc:1048] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:03.904289 28270 consensus_queue.cc:1048] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:04.050910 28144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
mode: GRACEFUL
new_leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
 from {username='slave'} at 127.0.0.1:51774
I20260709 13:03:04.051643 28144 raft_consensus.cc:606] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Received request to transfer leadership to TS 5633fd0a2e6844d7868edd694d2ee9cd
I20260709 13:03:04.061403 28144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
mode: GRACEFUL
new_leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
 from {username='slave'} at 127.0.0.1:51774
I20260709 13:03:04.062129 28144 raft_consensus.cc:606] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Received request to transfer leadership to TS 5633fd0a2e6844d7868edd694d2ee9cd
I20260709 13:03:04.076627 28068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
mode: GRACEFUL
new_leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:42590
I20260709 13:03:04.077276 28068 raft_consensus.cc:606] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Received request to transfer leadership to TS 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:04.078594 28294 consensus_queue.cc:1048] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:04.086396 28068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e624d08cc1249649be7f2c957496145"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
mode: GRACEFUL
new_leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:42590
I20260709 13:03:04.086938 28068 raft_consensus.cc:606] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Received request to transfer leadership to TS 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:04.091622 28294 raft_consensus.cc:993] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd: : Instructing follower 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:04.092638 28305 raft_consensus.cc:1081] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Signalling peer 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:04.094617 28143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e624d08cc1249649be7f2c957496145"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:51758
I20260709 13:03:04.095228 28143 raft_consensus.cc:493] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:04.095602 28143 raft_consensus.cc:3060] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.097182 28068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14fc44d49558470399e63e7d5c26d94f"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
mode: GRACEFUL
new_leader_uuid: "b0087b2c108442708a47eb31a1a17cda"
 from {username='slave'} at 127.0.0.1:42590
I20260709 13:03:04.097770 28068 raft_consensus.cc:606] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Received request to transfer leadership to TS b0087b2c108442708a47eb31a1a17cda
I20260709 13:03:04.102684 28143 raft_consensus.cc:515] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.104672 28143 leader_election.cc:290] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 2 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:04.105397 28068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e624d08cc1249649be7f2c957496145" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:04.107240 28068 raft_consensus.cc:3055] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:04.107580 28068 raft_consensus.cc:740] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:04.108382 28068 consensus_queue.cc:260] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.109745 28068 raft_consensus.cc:3060] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.106088 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e624d08cc1249649be7f2c957496145" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:04.112232 28217 raft_consensus.cc:3060] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.117496 28297 consensus_queue.cc:1048] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:04.121446 28217 raft_consensus.cc:2468] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 2.
I20260709 13:03:04.122972 28107 leader_election.cc:304] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c7e1f88ab2b4c8393022b5e5d7fb657, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:04.124301 28322 raft_consensus.cc:2804] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:04.124787 28322 raft_consensus.cc:697] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Becoming Leader. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:04.125614 28322 consensus_queue.cc:237] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.135784 28297 raft_consensus.cc:993] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd: : Instructing follower b0087b2c108442708a47eb31a1a17cda to start an election
I20260709 13:03:04.136255 28297 raft_consensus.cc:1081] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Signalling peer b0087b2c108442708a47eb31a1a17cda to start an election
I20260709 13:03:04.137198 28068 raft_consensus.cc:2468] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 2.
I20260709 13:03:04.138435 28216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14fc44d49558470399e63e7d5c26d94f"
dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
 from {username='slave'} at 127.0.0.1:32948
I20260709 13:03:04.140340 28216 raft_consensus.cc:493] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:04.140784 28216 raft_consensus.cc:3060] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.145056 27966 catalog_manager.cc:5697] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 reported cstate change: term changed from 1 to 2, leader changed from 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129) to 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:04.147132 28216 raft_consensus.cc:515] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.149220 28216 leader_election.cc:290] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 2 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:04.150430 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fc44d49558470399e63e7d5c26d94f" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:04.151041 28143 raft_consensus.cc:3060] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.155057 28068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fc44d49558470399e63e7d5c26d94f" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:04.155710 28068 raft_consensus.cc:3055] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:04.156060 28068 raft_consensus.cc:740] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:04.156905 28068 consensus_queue.cc:260] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.157845 28068 raft_consensus.cc:3060] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.158686 28143 raft_consensus.cc:2468] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 2.
I20260709 13:03:04.160086 28182 leader_election.cc:304] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c7e1f88ab2b4c8393022b5e5d7fb657, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:04.160924 28311 raft_consensus.cc:2804] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:04.162307 28311 raft_consensus.cc:697] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 2 LEADER]: Becoming Leader. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Running, Role: LEADER
I20260709 13:03:04.163166 28311 consensus_queue.cc:237] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.164656 28068 raft_consensus.cc:2468] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 2.
I20260709 13:03:04.170636 27966 catalog_manager.cc:5697] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda reported cstate change: term changed from 1 to 2, leader changed from 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129) to b0087b2c108442708a47eb31a1a17cda (127.27.71.131). New cstate: current_term: 2 leader_uuid: "b0087b2c108442708a47eb31a1a17cda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:04.183172 28322 consensus_queue.cc:1048] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:04.196336 28312 consensus_queue.cc:1048] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:04.420655 28297 raft_consensus.cc:993] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd: : Instructing follower 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:04.421118 28297 raft_consensus.cc:1081] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Signalling peer 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:04.423133 28143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:51758
I20260709 13:03:04.423764 28143 raft_consensus.cc:493] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:04.426303 28270 raft_consensus.cc:993] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657: : Instructing follower 5633fd0a2e6844d7868edd694d2ee9cd to start an election
I20260709 13:03:04.428236 28143 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.428486 28312 raft_consensus.cc:1081] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Signalling peer 5633fd0a2e6844d7868edd694d2ee9cd to start an election
I20260709 13:03:04.430379 28068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
 from {username='slave'} at 127.0.0.1:42576
I20260709 13:03:04.430933 28068 raft_consensus.cc:493] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:04.431316 28068 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.436357 28143 raft_consensus.cc:515] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.436883 28068 raft_consensus.cc:515] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.439865 28068 leader_election.cc:290] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 2 election: Requested vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:04.439745 28144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:04.440461 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:04.441049 28217 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.441083 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:04.441715 28069 raft_consensus.cc:3055] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:04.442049 28069 raft_consensus.cc:740] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:04.442435 28216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:04.442967 28216 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.442883 28069 consensus_queue.cc:260] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.440613 28144 raft_consensus.cc:3055] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:04.444342 28069 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.444779 28144 raft_consensus.cc:740] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:04.445693 28144 consensus_queue.cc:260] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.447079 28144 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.439759 28143 leader_election.cc:290] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 2 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:04.451488 28216 raft_consensus.cc:2468] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 2.
I20260709 13:03:04.455408 28322 raft_consensus.cc:993] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657: : Instructing follower 5633fd0a2e6844d7868edd694d2ee9cd to start an election
I20260709 13:03:04.455277 28032 leader_election.cc:304] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:04.456162 28312 raft_consensus.cc:1081] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Signalling peer 5633fd0a2e6844d7868edd694d2ee9cd to start an election
I20260709 13:03:04.456226 28272 raft_consensus.cc:2804] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:04.457405 28272 raft_consensus.cc:697] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Becoming Leader. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:04.458374 28068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
 from {username='slave'} at 127.0.0.1:42576
I20260709 13:03:04.459502 28068 raft_consensus.cc:493] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:04.460065 28068 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.461509 28272 consensus_queue.cc:237] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.466993 28144 raft_consensus.cc:2468] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 2.
I20260709 13:03:04.466740 28068 raft_consensus.cc:515] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.469487 28217 raft_consensus.cc:2468] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 2.
I20260709 13:03:04.459430 28069 raft_consensus.cc:2468] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 2.
I20260709 13:03:04.470841 28107 leader_election.cc:304] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c7e1f88ab2b4c8393022b5e5d7fb657, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:04.471812 28312 raft_consensus.cc:2804] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:04.472376 28312 raft_consensus.cc:697] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Becoming Leader. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:04.472805 28144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:04.473440 28144 raft_consensus.cc:3055] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:04.473892 28144 raft_consensus.cc:740] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:04.475760 28068 leader_election.cc:290] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 2 election: Requested vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:04.476316 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:04.475350 28144 consensus_queue.cc:260] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.477000 28217 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.477675 28144 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:04.479247 27965 catalog_manager.cc:5697] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd reported cstate change: term changed from 1 to 2, leader changed from 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130) to 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129). New cstate: current_term: 2 leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:04.473369 28312 consensus_queue.cc:237] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.485713 28217 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 2.
I20260709 13:03:04.487516 28144 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 2.
I20260709 13:03:04.488653 28032 leader_election.cc:304] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:04.490167 28272 raft_consensus.cc:2804] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:04.491004 28272 raft_consensus.cc:697] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Becoming Leader. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:04.492089 28272 consensus_queue.cc:237] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:04.494379 27966 catalog_manager.cc:5697] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 reported cstate change: term changed from 1 to 2, leader changed from 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129) to 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:04.500546 27965 catalog_manager.cc:5697] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd reported cstate change: term changed from 1 to 2, leader changed from 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130) to 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129). New cstate: current_term: 2 leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:04.528488 28217 raft_consensus.cc:1275] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:04.530565 28312 consensus_queue.cc:1048] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:04.542742 28068 raft_consensus.cc:1275] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:04.547709 28270 consensus_queue.cc:1048] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:04.742126 28068 raft_consensus.cc:1275] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Refusing update from remote peer b0087b2c108442708a47eb31a1a17cda: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:04.743544 28311 consensus_queue.cc:1048] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:04.754594 28144 raft_consensus.cc:1275] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Refusing update from remote peer b0087b2c108442708a47eb31a1a17cda: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:04.760429 28350 consensus_queue.cc:1048] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:04.866854 28144 raft_consensus.cc:1275] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Refusing update from remote peer 5633fd0a2e6844d7868edd694d2ee9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:04.868453 28297 consensus_queue.cc:1048] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:04.877930 28217 raft_consensus.cc:1275] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Refusing update from remote peer 5633fd0a2e6844d7868edd694d2ee9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:04.879834 28272 consensus_queue.cc:1048] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:04.912189 28144 raft_consensus.cc:1275] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Refusing update from remote peer 5633fd0a2e6844d7868edd694d2ee9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:04.913398 28272 consensus_queue.cc:1048] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:04.921566 28217 raft_consensus.cc:1275] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Refusing update from remote peer 5633fd0a2e6844d7868edd694d2ee9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:04.923194 28305 consensus_queue.cc:1048] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:05.026162 28217 raft_consensus.cc:1275] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:05.027758 28312 consensus_queue.cc:1048] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:05.028925 28068 raft_consensus.cc:1275] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:05.030489 28312 consensus_queue.cc:1048] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:07.136480 27934 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:03:07.137506 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:07.159849 28143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
mode: GRACEFUL
new_leader_uuid: "b0087b2c108442708a47eb31a1a17cda"
 from {username='slave'} at 127.0.0.1:51784
I20260709 13:03:07.160635 28143 raft_consensus.cc:606] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Received request to transfer leadership to TS b0087b2c108442708a47eb31a1a17cda
I20260709 13:03:07.174259 28069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
mode: GRACEFUL
new_leader_uuid: "b0087b2c108442708a47eb31a1a17cda"
 from {username='slave'} at 127.0.0.1:42592
I20260709 13:03:07.174922 28069 raft_consensus.cc:606] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Received request to transfer leadership to TS b0087b2c108442708a47eb31a1a17cda
I20260709 13:03:07.176664 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:03:07.177253 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:07.304358 28272 raft_consensus.cc:993] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd: : Instructing follower b0087b2c108442708a47eb31a1a17cda to start an election
I20260709 13:03:07.304795 28272 raft_consensus.cc:1081] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Signalling peer b0087b2c108442708a47eb31a1a17cda to start an election
I20260709 13:03:07.306643 28217 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c"
dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
 from {username='slave'} at 127.0.0.1:32948
I20260709 13:03:07.307301 28217 raft_consensus.cc:493] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:07.307619 28217 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.312840 28217 raft_consensus.cc:515] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:07.315145 28217 leader_election.cc:290] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 3 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:07.316162 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:07.316287 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:07.316967 28069 raft_consensus.cc:3055] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:07.317000 28143 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.317302 28069 raft_consensus.cc:740] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:07.318090 28069 consensus_queue.cc:260] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:07.319319 28069 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.324561 28143 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 3.
I20260709 13:03:07.324707 28069 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 3.
I20260709 13:03:07.325884 28182 leader_election.cc:304] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c7e1f88ab2b4c8393022b5e5d7fb657, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:07.326592 28359 raft_consensus.cc:2804] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:07.327955 28359 raft_consensus.cc:697] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Becoming Leader. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Running, Role: LEADER
I20260709 13:03:07.328795 28359 consensus_queue.cc:237] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:07.335536 27966 catalog_manager.cc:5697] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda reported cstate change: term changed from 2 to 3, leader changed from 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129) to b0087b2c108442708a47eb31a1a17cda (127.27.71.131). New cstate: current_term: 3 leader_uuid: "b0087b2c108442708a47eb31a1a17cda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.433010 28361 raft_consensus.cc:993] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657: : Instructing follower b0087b2c108442708a47eb31a1a17cda to start an election
I20260709 13:03:07.433408 28361 raft_consensus.cc:1081] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Signalling peer b0087b2c108442708a47eb31a1a17cda to start an election
I20260709 13:03:07.434940 28217 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1"
dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
 from {username='slave'} at 127.0.0.1:32938
I20260709 13:03:07.435479 28217 raft_consensus.cc:493] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:07.435752 28217 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.439909 28217 raft_consensus.cc:515] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:07.441560 28217 leader_election.cc:290] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 3 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:07.442452 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:07.442724 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:07.443069 28069 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.443259 28143 raft_consensus.cc:3055] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:07.443610 28143 raft_consensus.cc:740] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:07.444444 28143 consensus_queue.cc:260] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:07.445668 28143 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:07.450086 28069 raft_consensus.cc:2468] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 3.
I20260709 13:03:07.451387 28181 leader_election.cc:304] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:07.452193 28359 raft_consensus.cc:2804] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:07.452284 28143 raft_consensus.cc:2468] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 3.
I20260709 13:03:07.452713 28359 raft_consensus.cc:697] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Becoming Leader. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Running, Role: LEADER
I20260709 13:03:07.453668 28359 consensus_queue.cc:237] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:07.460749 27966 catalog_manager.cc:5697] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda reported cstate change: term changed from 2 to 3, leader changed from 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130) to b0087b2c108442708a47eb31a1a17cda (127.27.71.131). New cstate: current_term: 3 leader_uuid: "b0087b2c108442708a47eb31a1a17cda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.725807 28143 raft_consensus.cc:1275] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Refusing update from remote peer b0087b2c108442708a47eb31a1a17cda: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:07.727196 28359 consensus_queue.cc:1048] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:07.737190 28069 raft_consensus.cc:1275] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Refusing update from remote peer b0087b2c108442708a47eb31a1a17cda: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:07.739399 28366 consensus_queue.cc:1048] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:08.024480 28143 raft_consensus.cc:1275] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Refusing update from remote peer b0087b2c108442708a47eb31a1a17cda: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:08.025888 28366 consensus_queue.cc:1048] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:08.040014 28068 raft_consensus.cc:1275] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Refusing update from remote peer b0087b2c108442708a47eb31a1a17cda: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:08.041529 28366 consensus_queue.cc:1048] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:08.178412 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:08.191581 28068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
mode: GRACEFUL
new_leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:42590
I20260709 13:03:08.192145 28068 raft_consensus.cc:606] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Received request to transfer leadership to TS 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:08.198563 28068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
mode: GRACEFUL
new_leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:42590
I20260709 13:03:08.199213 28068 raft_consensus.cc:606] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Received request to transfer leadership to TS 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:08.205024 28068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08508a2dc83347879ae38bb7afd65ac8"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
mode: GRACEFUL
new_leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:42590
I20260709 13:03:08.205515 28068 raft_consensus.cc:606] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Received request to transfer leadership to TS 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:08.222410 28217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c"
dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
mode: GRACEFUL
new_leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:32956
I20260709 13:03:08.223445 28217 raft_consensus.cc:606] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Received request to transfer leadership to TS 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:08.230016 28217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1"
dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
mode: GRACEFUL
new_leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:32956
I20260709 13:03:08.230583 28217 raft_consensus.cc:606] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Received request to transfer leadership to TS 9c7e1f88ab2b4c8393022b5e5d7fb657
I20260709 13:03:08.481237 28272 raft_consensus.cc:993] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd: : Instructing follower 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:08.481776 28385 raft_consensus.cc:1081] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Signalling peer 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:08.484583 28143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08508a2dc83347879ae38bb7afd65ac8"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:51758
I20260709 13:03:08.485397 28143 raft_consensus.cc:493] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:08.485991 28143 raft_consensus.cc:3060] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.490094 28272 raft_consensus.cc:993] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd: : Instructing follower 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:08.490667 28385 raft_consensus.cc:1081] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Signalling peer 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:08.493295 28144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:51758
I20260709 13:03:08.494771 28144 raft_consensus.cc:493] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:08.495254 28144 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.495676 28143 raft_consensus.cc:515] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.503811 28143 leader_election.cc:290] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 2 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:08.505069 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08508a2dc83347879ae38bb7afd65ac8" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:08.505553 28068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08508a2dc83347879ae38bb7afd65ac8" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:08.505746 28217 raft_consensus.cc:3060] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.506187 28068 raft_consensus.cc:3055] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:08.506541 28068 raft_consensus.cc:740] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:08.507346 28068 consensus_queue.cc:260] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.508015 28144 raft_consensus.cc:515] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.508708 28068 raft_consensus.cc:3060] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.510203 28144 leader_election.cc:290] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 3 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:08.511535 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:08.512158 28069 raft_consensus.cc:3055] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:08.512703 28069 raft_consensus.cc:740] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:08.513803 28216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:08.513945 28217 raft_consensus.cc:2468] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 2.
I20260709 13:03:08.514542 28216 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.514132 28069 consensus_queue.cc:260] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.517381 28069 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:08.518256 28107 leader_election.cc:304] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c7e1f88ab2b4c8393022b5e5d7fb657, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:08.519286 28361 raft_consensus.cc:2804] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:08.524183 28068 raft_consensus.cc:2468] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 2.
I20260709 13:03:08.524617 28216 raft_consensus.cc:2468] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 3.
I20260709 13:03:08.525702 28107 leader_election.cc:304] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c7e1f88ab2b4c8393022b5e5d7fb657, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:08.526192 28361 raft_consensus.cc:697] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Becoming Leader. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:08.527459 28386 raft_consensus.cc:2804] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:08.527374 28361 consensus_queue.cc:237] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.527880 28069 raft_consensus.cc:2468] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 3.
I20260709 13:03:08.528450 28386 raft_consensus.cc:697] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 LEADER]: Becoming Leader. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:08.529592 28386 consensus_queue.cc:237] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.539271 27966 catalog_manager.cc:5697] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 reported cstate change: term changed from 1 to 2, leader changed from 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129) to 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:08.544771 28272 raft_consensus.cc:993] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd: : Instructing follower 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:08.545336 28385 raft_consensus.cc:1081] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Signalling peer 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:08.547302 28144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:51758
I20260709 13:03:08.547816 28144 raft_consensus.cc:493] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:08.548215 28144 raft_consensus.cc:3060] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.558079 28144 raft_consensus.cc:515] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:08.562817 28144 leader_election.cc:290] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 2 election: Requested vote from peers b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523), 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675)
I20260709 13:03:08.563619 28216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:08.564057 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:08.564380 28216 raft_consensus.cc:3060] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.564656 28069 raft_consensus.cc:3055] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:08.565062 28069 raft_consensus.cc:740] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:08.564401 27966 catalog_manager.cc:5697] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 reported cstate change: term changed from 2 to 3, leader changed from 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129) to 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:08.565871 28069 consensus_queue.cc:260] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:08.566859 28069 raft_consensus.cc:3060] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:08.571623 28216 raft_consensus.cc:2468] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 2.
I20260709 13:03:08.573704 28107 leader_election.cc:304] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c7e1f88ab2b4c8393022b5e5d7fb657, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:08.574996 28069 raft_consensus.cc:2468] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 2.
I20260709 13:03:08.575309 28386 raft_consensus.cc:2804] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:08.576125 28386 raft_consensus.cc:697] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Becoming Leader. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:08.577489 28386 consensus_queue.cc:237] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:08.589799 27966 catalog_manager.cc:5697] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 reported cstate change: term changed from 1 to 2, leader changed from 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129) to 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:08.595894 28359 raft_consensus.cc:993] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda: : Instructing follower 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:08.596473 28366 raft_consensus.cc:1081] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Signalling peer 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:08.598121 28144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:51744
I20260709 13:03:08.598789 28144 raft_consensus.cc:493] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:08.599172 28144 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.607833 28144 raft_consensus.cc:515] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.609898 28144 leader_election.cc:290] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 4 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:08.610678 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:08.611028 28216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2bdcea6a1734755a656586304a1eef1" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:08.611485 28069 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.611713 28216 raft_consensus.cc:3055] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:08.612149 28216 raft_consensus.cc:740] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Running, Role: LEADER
I20260709 13:03:08.612812 28216 consensus_queue.cc:260] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.613759 28216 raft_consensus.cc:3060] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.617179 28069 raft_consensus.cc:2468] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 4.
I20260709 13:03:08.618291 28107 leader_election.cc:304] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: 
I20260709 13:03:08.618943 28216 raft_consensus.cc:2468] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 4.
I20260709 13:03:08.619199 28386 raft_consensus.cc:2804] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:08.619736 28386 raft_consensus.cc:697] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 LEADER]: Becoming Leader. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:08.621546 28386 consensus_queue.cc:237] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.628660 27964 catalog_manager.cc:5697] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 reported cstate change: term changed from 3 to 4, leader changed from b0087b2c108442708a47eb31a1a17cda (127.27.71.131) to 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130). New cstate: current_term: 4 leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:08.663328 28380 raft_consensus.cc:993] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda: : Instructing follower 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:08.664031 28366 raft_consensus.cc:1081] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Signalling peer 9c7e1f88ab2b4c8393022b5e5d7fb657 to start an election
I20260709 13:03:08.665822 28144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
 from {username='slave'} at 127.0.0.1:51744
I20260709 13:03:08.666373 28144 raft_consensus.cc:493] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:08.666854 28144 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.672739 28144 raft_consensus.cc:515] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.675673 28144 leader_election.cc:290] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 4 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:08.676775 28069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:08.677004 28216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:08.677558 28069 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.677757 28216 raft_consensus.cc:3055] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:08.678133 28216 raft_consensus.cc:740] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Running, Role: LEADER
I20260709 13:03:08.679091 28216 consensus_queue.cc:260] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.680663 28216 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:08.685374 28069 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 4.
I20260709 13:03:08.686000 28216 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c7e1f88ab2b4c8393022b5e5d7fb657 in term 4.
I20260709 13:03:08.686822 28107 leader_election.cc:304] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, 9c7e1f88ab2b4c8393022b5e5d7fb657; no voters: 
I20260709 13:03:08.687616 28386 raft_consensus.cc:2804] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:08.688124 28386 raft_consensus.cc:697] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 LEADER]: Becoming Leader. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:08.688905 28386 consensus_queue.cc:237] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:08.699352 27964 catalog_manager.cc:5697] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 reported cstate change: term changed from 3 to 4, leader changed from b0087b2c108442708a47eb31a1a17cda (127.27.71.131) to 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130). New cstate: current_term: 4 leader_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:08.912271 28069 raft_consensus.cc:1275] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:08.914292 28361 consensus_queue.cc:1048] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:08.915169 28069 raft_consensus.cc:1275] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:08.916767 28361 consensus_queue.cc:1048] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:08.948359 28216 raft_consensus.cc:1275] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:08.950709 28386 consensus_queue.cc:1048] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:08.953043 28217 raft_consensus.cc:1275] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:08.964869 28361 consensus_queue.cc:1048] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:09.091202 28217 raft_consensus.cc:1275] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:09.092610 28361 consensus_queue.cc:1048] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:09.100400 28216 raft_consensus.cc:1275] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 4 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:09.103879 28361 consensus_queue.cc:1048] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:09.104189 28068 raft_consensus.cc:1275] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:09.106987 28386 consensus_queue.cc:1048] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:09.125713 28068 raft_consensus.cc:1275] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:09.127267 28386 consensus_queue.cc:1048] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:09.218369 28217 raft_consensus.cc:1275] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 4 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:09.219760 28386 consensus_queue.cc:1048] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:09.229118 28068 raft_consensus.cc:1275] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 FOLLOWER]: Refusing update from remote peer 9c7e1f88ab2b4c8393022b5e5d7fb657: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:09.231673 28387 consensus_queue.cc:1048] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:11.246457 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:11.260856 28144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e624d08cc1249649be7f2c957496145"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
mode: GRACEFUL
new_leader_uuid: "b0087b2c108442708a47eb31a1a17cda"
 from {username='slave'} at 127.0.0.1:51784
I20260709 13:03:11.262382 28144 raft_consensus.cc:606] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Received request to transfer leadership to TS b0087b2c108442708a47eb31a1a17cda
I20260709 13:03:11.267007 28144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
mode: GRACEFUL
new_leader_uuid: "b0087b2c108442708a47eb31a1a17cda"
 from {username='slave'} at 127.0.0.1:51784
I20260709 13:03:11.268278 28144 raft_consensus.cc:606] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Received request to transfer leadership to TS b0087b2c108442708a47eb31a1a17cda
I20260709 13:03:11.272220 28143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
mode: GRACEFUL
new_leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
 from {username='slave'} at 127.0.0.1:51784
I20260709 13:03:11.272886 28143 raft_consensus.cc:606] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 LEADER]: Received request to transfer leadership to TS 5633fd0a2e6844d7868edd694d2ee9cd
I20260709 13:03:11.276942 28143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
mode: GRACEFUL
new_leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
 from {username='slave'} at 127.0.0.1:51784
I20260709 13:03:11.277469 28143 raft_consensus.cc:606] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 LEADER]: Received request to transfer leadership to TS 5633fd0a2e6844d7868edd694d2ee9cd
I20260709 13:03:11.281525 28143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8"
dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
mode: GRACEFUL
new_leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
 from {username='slave'} at 127.0.0.1:51784
I20260709 13:03:11.282265 28143 raft_consensus.cc:606] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Received request to transfer leadership to TS 5633fd0a2e6844d7868edd694d2ee9cd
I20260709 13:03:11.283612 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:11.284330 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:11.315148 28386 raft_consensus.cc:993] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657: : Instructing follower 5633fd0a2e6844d7868edd694d2ee9cd to start an election
I20260709 13:03:11.315850 28414 raft_consensus.cc:1081] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 LEADER]: Signalling peer 5633fd0a2e6844d7868edd694d2ee9cd to start an election
I20260709 13:03:11.318511 28068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
 from {username='slave'} at 127.0.0.1:42576
I20260709 13:03:11.319219 28068 raft_consensus.cc:493] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:11.319569 28068 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:11.326121 28068 raft_consensus.cc:515] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.328294 28068 leader_election.cc:290] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 5 election: Requested vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:11.330004 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:11.329934 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb12b90a406c4f0d83468f73bb6f304c" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:11.330933 28143 raft_consensus.cc:3055] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:11.330900 28217 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:11.331377 28143 raft_consensus.cc:740] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:11.332360 28143 consensus_queue.cc:260] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.333513 28143 raft_consensus.cc:3060] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:11.338402 28217 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 5.
I20260709 13:03:11.339942 28143 raft_consensus.cc:2468] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 5.
I20260709 13:03:11.340595 28032 leader_election.cc:304] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:11.343066 28415 raft_consensus.cc:2804] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:11.344724 28415 raft_consensus.cc:697] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 5 LEADER]: Becoming Leader. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:11.345959 28415 consensus_queue.cc:237] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.364111 27965 catalog_manager.cc:5697] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd reported cstate change: term changed from 4 to 5, leader changed from 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130) to 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129). New cstate: current_term: 5 leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.448539 28386 raft_consensus.cc:993] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657: : Instructing follower b0087b2c108442708a47eb31a1a17cda to start an election
I20260709 13:03:11.449043 28386 raft_consensus.cc:1081] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Signalling peer b0087b2c108442708a47eb31a1a17cda to start an election
I20260709 13:03:11.450625 28217 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947"
dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
 from {username='slave'} at 127.0.0.1:32938
I20260709 13:03:11.451027 28217 raft_consensus.cc:493] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:11.451260 28217 raft_consensus.cc:3060] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:11.455765 28217 raft_consensus.cc:515] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:11.458060 28217 leader_election.cc:290] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 3 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:11.459067 28068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:11.459236 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a71bcaa524ed40b8aed6076294fbe947" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:11.459949 28068 raft_consensus.cc:3060] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:11.459959 28143 raft_consensus.cc:3055] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:11.460853 28143 raft_consensus.cc:740] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:11.461678 28143 consensus_queue.cc:260] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:11.462929 28143 raft_consensus.cc:3060] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:11.466854 28068 raft_consensus.cc:2468] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 3.
I20260709 13:03:11.468386 28181 leader_election.cc:304] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:11.469146 28143 raft_consensus.cc:2468] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 3.
I20260709 13:03:11.469305 28366 raft_consensus.cc:2804] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:11.470716 28366 raft_consensus.cc:697] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Becoming Leader. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Running, Role: LEADER
I20260709 13:03:11.471784 28366 consensus_queue.cc:237] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } }
I20260709 13:03:11.479867 27965 catalog_manager.cc:5697] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda reported cstate change: term changed from 2 to 3, leader changed from 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130) to b0087b2c108442708a47eb31a1a17cda (127.27.71.131). New cstate: current_term: 3 leader_uuid: "b0087b2c108442708a47eb31a1a17cda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.515424 28414 raft_consensus.cc:993] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: : Instructing follower 5633fd0a2e6844d7868edd694d2ee9cd to start an election
I20260709 13:03:11.515956 28386 raft_consensus.cc:1081] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Signalling peer 5633fd0a2e6844d7868edd694d2ee9cd to start an election
I20260709 13:03:11.517467 28068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
 from {username='slave'} at 127.0.0.1:42576
I20260709 13:03:11.517848 28068 raft_consensus.cc:493] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:11.518379 28068 raft_consensus.cc:3060] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:11.523839 28068 raft_consensus.cc:515] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.525485 28068 leader_election.cc:290] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 2 election: Requested vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:11.526393 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:11.526659 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda99a8c4108494bac452ede28bb16c8" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:11.526959 28143 raft_consensus.cc:3055] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:11.527184 28217 raft_consensus.cc:3060] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:11.527356 28143 raft_consensus.cc:740] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:11.528115 28143 consensus_queue.cc:260] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.528992 28143 raft_consensus.cc:3060] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:11.531842 28217 raft_consensus.cc:2468] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 2.
I20260709 13:03:11.533066 28032 leader_election.cc:304] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:11.533435 28143 raft_consensus.cc:2468] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 2.
I20260709 13:03:11.533938 28415 raft_consensus.cc:2804] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:11.534422 28415 raft_consensus.cc:697] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Becoming Leader. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:11.535190 28415 consensus_queue.cc:237] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.543416 27965 catalog_manager.cc:5697] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd reported cstate change: term changed from 1 to 2, leader changed from 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130) to 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129). New cstate: current_term: 2 leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.621325 28414 raft_consensus.cc:993] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657: : Instructing follower b0087b2c108442708a47eb31a1a17cda to start an election
I20260709 13:03:11.622334 28386 raft_consensus.cc:1081] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Signalling peer b0087b2c108442708a47eb31a1a17cda to start an election
I20260709 13:03:11.624032 28217 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e624d08cc1249649be7f2c957496145"
dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
 from {username='slave'} at 127.0.0.1:32938
I20260709 13:03:11.624466 28217 raft_consensus.cc:493] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:11.624769 28217 raft_consensus.cc:3060] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:11.630357 28217 raft_consensus.cc:515] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.632138 28217 leader_election.cc:290] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 3 election: Requested vote from peers 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675), 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771)
I20260709 13:03:11.633255 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e624d08cc1249649be7f2c957496145" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:11.633327 28068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e624d08cc1249649be7f2c957496145" candidate_uuid: "b0087b2c108442708a47eb31a1a17cda" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
I20260709 13:03:11.633751 28143 raft_consensus.cc:3055] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:11.633931 28068 raft_consensus.cc:3060] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:11.634011 28143 raft_consensus.cc:740] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:11.635138 28143 consensus_queue.cc:260] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.636638 28143 raft_consensus.cc:3060] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:11.638820 28068 raft_consensus.cc:2468] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 3.
I20260709 13:03:11.640909 28181 leader_election.cc:304] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:11.643110 28366 raft_consensus.cc:2804] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:11.643301 28143 raft_consensus.cc:2468] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0087b2c108442708a47eb31a1a17cda in term 3.
I20260709 13:03:11.643599 28366 raft_consensus.cc:697] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Becoming Leader. State: Replica: b0087b2c108442708a47eb31a1a17cda, State: Running, Role: LEADER
I20260709 13:03:11.645262 28366 consensus_queue.cc:237] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.652840 27965 catalog_manager.cc:5697] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda reported cstate change: term changed from 2 to 3, leader changed from 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130) to b0087b2c108442708a47eb31a1a17cda (127.27.71.131). New cstate: current_term: 3 leader_uuid: "b0087b2c108442708a47eb31a1a17cda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.727154 28414 raft_consensus.cc:993] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657: : Instructing follower 5633fd0a2e6844d7868edd694d2ee9cd to start an election
I20260709 13:03:11.727715 28414 raft_consensus.cc:1081] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 LEADER]: Signalling peer 5633fd0a2e6844d7868edd694d2ee9cd to start an election
I20260709 13:03:11.729739 28068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020"
dest_uuid: "5633fd0a2e6844d7868edd694d2ee9cd"
 from {username='slave'} at 127.0.0.1:42576
I20260709 13:03:11.730291 28068 raft_consensus.cc:493] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:11.730687 28068 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:11.735055 28068 raft_consensus.cc:515] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.736768 28068 leader_election.cc:290] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 4 election: Requested vote from peers 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130:37771), b0087b2c108442708a47eb31a1a17cda (127.27.71.131:35523)
I20260709 13:03:11.737803 28217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b0087b2c108442708a47eb31a1a17cda"
I20260709 13:03:11.738346 28217 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:11.737818 28143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef5db39115cb4b8cba7eee65c5479020" candidate_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657"
I20260709 13:03:11.740864 28143 raft_consensus.cc:3055] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:11.741199 28143 raft_consensus.cc:740] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9c7e1f88ab2b4c8393022b5e5d7fb657, State: Running, Role: LEADER
I20260709 13:03:11.742110 28143 consensus_queue.cc:260] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.743459 28143 raft_consensus.cc:3060] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:11.744709 28217 raft_consensus.cc:2468] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 4.
I20260709 13:03:11.745865 28032 leader_election.cc:304] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5633fd0a2e6844d7868edd694d2ee9cd, b0087b2c108442708a47eb31a1a17cda; no voters: 
I20260709 13:03:11.746647 28415 raft_consensus.cc:2804] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:11.747092 28415 raft_consensus.cc:697] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 LEADER]: Becoming Leader. State: Replica: 5633fd0a2e6844d7868edd694d2ee9cd, State: Running, Role: LEADER
I20260709 13:03:11.747879 28415 consensus_queue.cc:237] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } }
I20260709 13:03:11.756664 28143 raft_consensus.cc:2468] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5633fd0a2e6844d7868edd694d2ee9cd in term 4.
I20260709 13:03:11.757542 27965 catalog_manager.cc:5697] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd reported cstate change: term changed from 3 to 4, leader changed from 9c7e1f88ab2b4c8393022b5e5d7fb657 (127.27.71.130) to 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129). New cstate: current_term: 4 leader_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.758755 28143 raft_consensus.cc:1275] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 5 FOLLOWER]: Refusing update from remote peer 5633fd0a2e6844d7868edd694d2ee9cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:03:11.761052 28415 consensus_queue.cc:1048] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:11.775192 28217 raft_consensus.cc:1275] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 5 FOLLOWER]: Refusing update from remote peer 5633fd0a2e6844d7868edd694d2ee9cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:03:11.776677 28415 consensus_queue.cc:1048] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:11.855954 28143 raft_consensus.cc:1275] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Refusing update from remote peer b0087b2c108442708a47eb31a1a17cda: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:11.857645 28366 consensus_queue.cc:1048] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:11.868099 28068 raft_consensus.cc:1275] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Refusing update from remote peer b0087b2c108442708a47eb31a1a17cda: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:11.869439 28366 consensus_queue.cc:1048] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:12.034966 28143 raft_consensus.cc:1275] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Refusing update from remote peer b0087b2c108442708a47eb31a1a17cda: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:12.036339 28366 consensus_queue.cc:1048] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:12.046677 28068 raft_consensus.cc:1275] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Refusing update from remote peer b0087b2c108442708a47eb31a1a17cda: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:12.048966 28422 consensus_queue.cc:1048] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [LEADER]: Connected to new peer: Peer: permanent_uuid: "5633fd0a2e6844d7868edd694d2ee9cd" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 34675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:12.139402 28217 raft_consensus.cc:1275] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Refusing update from remote peer 5633fd0a2e6844d7868edd694d2ee9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:12.140990 28415 consensus_queue.cc:1048] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:12.148910 28143 raft_consensus.cc:1275] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Refusing update from remote peer 5633fd0a2e6844d7868edd694d2ee9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:12.153285 28415 consensus_queue.cc:1048] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:12.285563 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:12.291711 27934 tablet_server.cc:179] TabletServer@127.27.71.129:0 shutting down...
W20260709 13:03:12.298182 28181 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:12.304651 28181 consensus_peers.cc:597] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda -> Peer 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675): Couldn't send request to peer 5633fd0a2e6844d7868edd694d2ee9cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:12.318516 28143 raft_consensus.cc:1275] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 FOLLOWER]: Refusing update from remote peer 5633fd0a2e6844d7868edd694d2ee9cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:12.322919 28440 consensus_queue.cc:1048] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c7e1f88ab2b4c8393022b5e5d7fb657" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:12.331067 28217 raft_consensus.cc:1275] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 4 FOLLOWER]: Refusing update from remote peer 5633fd0a2e6844d7868edd694d2ee9cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:12.333388 28440 consensus_queue.cc:1048] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0087b2c108442708a47eb31a1a17cda" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 35523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:12.391597 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:12.392903 27934 tablet_replica.cc:333] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd: stopping tablet replica
I20260709 13:03:12.394099 27934 raft_consensus.cc:2243] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.394688 27934 raft_consensus.cc:2272] T 9208d2ef87e0448ebcfe9acd861208f7 P 5633fd0a2e6844d7868edd694d2ee9cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.397534 27934 tablet_replica.cc:333] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd: stopping tablet replica
I20260709 13:03:12.398386 27934 raft_consensus.cc:2243] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.399091 27934 raft_consensus.cc:2272] T 14fc44d49558470399e63e7d5c26d94f P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.401022 27934 tablet_replica.cc:333] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd: stopping tablet replica
I20260709 13:03:12.401659 27934 raft_consensus.cc:2243] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.402410 27934 raft_consensus.cc:2272] T fda99a8c4108494bac452ede28bb16c8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.406597 27934 tablet_replica.cc:333] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd: stopping tablet replica
I20260709 13:03:12.409205 27934 raft_consensus.cc:2243] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.409989 27934 raft_consensus.cc:2272] T a71bcaa524ed40b8aed6076294fbe947 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.413880 27934 tablet_replica.cc:333] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd: stopping tablet replica
I20260709 13:03:12.414690 27934 raft_consensus.cc:2243] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.415678 27934 raft_consensus.cc:2272] T ef5db39115cb4b8cba7eee65c5479020 P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.417968 27934 tablet_replica.cc:333] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd: stopping tablet replica
I20260709 13:03:12.418447 27934 raft_consensus.cc:2243] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.418839 27934 raft_consensus.cc:2272] T 2e624d08cc1249649be7f2c957496145 P 5633fd0a2e6844d7868edd694d2ee9cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.421262 27934 tablet_replica.cc:333] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd: stopping tablet replica
I20260709 13:03:12.421729 27934 raft_consensus.cc:2243] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.422430 27934 raft_consensus.cc:2272] T eb12b90a406c4f0d83468f73bb6f304c P 5633fd0a2e6844d7868edd694d2ee9cd [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.424903 27934 tablet_replica.cc:333] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd: stopping tablet replica
I20260709 13:03:12.426039 27934 raft_consensus.cc:2243] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.426638 27934 raft_consensus.cc:2272] T c2bdcea6a1734755a656586304a1eef1 P 5633fd0a2e6844d7868edd694d2ee9cd [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:12.427565 28181 consensus_peers.cc:597] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda -> Peer 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675): Couldn't send request to peer 5633fd0a2e6844d7868edd694d2ee9cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:12.428664 27934 tablet_replica.cc:333] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd: stopping tablet replica
I20260709 13:03:12.429352 27934 raft_consensus.cc:2243] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.429847 27934 raft_consensus.cc:2272] T 08508a2dc83347879ae38bb7afd65ac8 P 5633fd0a2e6844d7868edd694d2ee9cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.548175 27934 tablet_server.cc:196] TabletServer@127.27.71.129:0 shutdown complete.
I20260709 13:03:12.569391 27934 tablet_server.cc:179] TabletServer@127.27.71.130:0 shutting down...
W20260709 13:03:12.589432 28181 consensus_peers.cc:597] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda -> Peer 5633fd0a2e6844d7868edd694d2ee9cd (127.27.71.129:34675): Couldn't send request to peer 5633fd0a2e6844d7868edd694d2ee9cd. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:34675: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:12.603813 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:12.604777 27934 tablet_replica.cc:333] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657: stopping tablet replica
I20260709 13:03:12.605350 27934 raft_consensus.cc:2243] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.606161 27934 raft_consensus.cc:2272] T 9208d2ef87e0448ebcfe9acd861208f7 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.608822 27934 tablet_replica.cc:333] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657: stopping tablet replica
I20260709 13:03:12.609702 27934 raft_consensus.cc:2243] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.610170 27934 raft_consensus.cc:2272] T a71bcaa524ed40b8aed6076294fbe947 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.613137 27934 tablet_replica.cc:333] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: stopping tablet replica
I20260709 13:03:12.613776 27934 raft_consensus.cc:2243] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.614557 27934 raft_consensus.cc:2272] T 08508a2dc83347879ae38bb7afd65ac8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.616331 27934 tablet_replica.cc:333] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657: stopping tablet replica
I20260709 13:03:12.616858 27934 raft_consensus.cc:2243] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.617576 27934 raft_consensus.cc:2272] T c2bdcea6a1734755a656586304a1eef1 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.619192 27934 tablet_replica.cc:333] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657: stopping tablet replica
I20260709 13:03:12.619678 27934 raft_consensus.cc:2243] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.620105 27934 raft_consensus.cc:2272] T eb12b90a406c4f0d83468f73bb6f304c P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.622330 27934 tablet_replica.cc:333] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657: stopping tablet replica
I20260709 13:03:12.622876 27934 raft_consensus.cc:2243] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.623292 27934 raft_consensus.cc:2272] T 14fc44d49558470399e63e7d5c26d94f P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.624984 27934 tablet_replica.cc:333] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657: stopping tablet replica
I20260709 13:03:12.625516 27934 raft_consensus.cc:2243] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.625955 27934 raft_consensus.cc:2272] T fda99a8c4108494bac452ede28bb16c8 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.628144 27934 tablet_replica.cc:333] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657: stopping tablet replica
I20260709 13:03:12.628633 27934 raft_consensus.cc:2243] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.629036 27934 raft_consensus.cc:2272] T ef5db39115cb4b8cba7eee65c5479020 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.631443 27934 tablet_replica.cc:333] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657: stopping tablet replica
I20260709 13:03:12.632036 27934 raft_consensus.cc:2243] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.632472 27934 raft_consensus.cc:2272] T 2e624d08cc1249649be7f2c957496145 P 9c7e1f88ab2b4c8393022b5e5d7fb657 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.664781 27934 tablet_server.cc:196] TabletServer@127.27.71.130:0 shutdown complete.
I20260709 13:03:12.688984 27934 tablet_server.cc:179] TabletServer@127.27.71.131:0 shutting down...
I20260709 13:03:12.723549 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:12.724325 27934 tablet_replica.cc:333] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda: stopping tablet replica
I20260709 13:03:12.724983 27934 raft_consensus.cc:2243] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.725736 27934 raft_consensus.cc:2272] T 9208d2ef87e0448ebcfe9acd861208f7 P b0087b2c108442708a47eb31a1a17cda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.727777 27934 tablet_replica.cc:333] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda: stopping tablet replica
I20260709 13:03:12.728368 27934 raft_consensus.cc:2243] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.729068 27934 raft_consensus.cc:2272] T 2e624d08cc1249649be7f2c957496145 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.731817 27934 tablet_replica.cc:333] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda: stopping tablet replica
I20260709 13:03:12.732355 27934 raft_consensus.cc:2243] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.732702 27934 raft_consensus.cc:2272] T 08508a2dc83347879ae38bb7afd65ac8 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.734177 27934 tablet_replica.cc:333] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda: stopping tablet replica
I20260709 13:03:12.734627 27934 raft_consensus.cc:2243] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.734993 27934 raft_consensus.cc:2272] T c2bdcea6a1734755a656586304a1eef1 P b0087b2c108442708a47eb31a1a17cda [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.736634 27934 tablet_replica.cc:333] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda: stopping tablet replica
I20260709 13:03:12.737087 27934 raft_consensus.cc:2243] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.737483 27934 raft_consensus.cc:2272] T eb12b90a406c4f0d83468f73bb6f304c P b0087b2c108442708a47eb31a1a17cda [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.739588 27934 tablet_replica.cc:333] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda: stopping tablet replica
I20260709 13:03:12.740087 27934 raft_consensus.cc:2243] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.740759 27934 raft_consensus.cc:2272] T 14fc44d49558470399e63e7d5c26d94f P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.742244 27934 tablet_replica.cc:333] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda: stopping tablet replica
I20260709 13:03:12.742656 27934 raft_consensus.cc:2243] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.742997 27934 raft_consensus.cc:2272] T fda99a8c4108494bac452ede28bb16c8 P b0087b2c108442708a47eb31a1a17cda [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.745095 27934 tablet_replica.cc:333] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda: stopping tablet replica
I20260709 13:03:12.745594 27934 raft_consensus.cc:2243] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.745949 27934 raft_consensus.cc:2272] T ef5db39115cb4b8cba7eee65c5479020 P b0087b2c108442708a47eb31a1a17cda [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.748001 27934 tablet_replica.cc:333] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda: stopping tablet replica
I20260709 13:03:12.748404 27934 raft_consensus.cc:2243] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.749023 27934 raft_consensus.cc:2272] T a71bcaa524ed40b8aed6076294fbe947 P b0087b2c108442708a47eb31a1a17cda [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.776160 27934 tablet_server.cc:196] TabletServer@127.27.71.131:0 shutdown complete.
I20260709 13:03:12.794188 27934 master.cc:562] Master@127.27.71.190:39055 shutting down...
I20260709 13:03:12.960009 27934 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.960594 27934 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.960932 27934 tablet_replica.cc:333] T 00000000000000000000000000000000 P fbfc8dd108034ea6b31953119fd5ab54: stopping tablet replica
I20260709 13:03:12.982314 27934 master.cc:584] Master@127.27.71.190:39055 shutdown complete.
5633fd0a2e6844d7868edd694d2ee9cd  4
9c7e1f88ab2b4c8393022b5e5d7fb657  4
b0087b2c108442708a47eb31a1a17cda  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (13370 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:03:13.033183 27934 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.71.190:36419
I20260709 13:03:13.034529 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:13.043041 28448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:13.043844 28447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:13.044728 28450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:13.045782 27934 server_base.cc:1061] running on GCE node
I20260709 13:03:13.046762 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:13.046983 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:13.047165 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602193047147 us; error 0 us; skew 500 ppm
I20260709 13:03:13.047732 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:13.052110 27934 webserver.cc:533] Webserver started at http://127.27.71.190:39169/ using document root <none> and password file <none>
I20260709 13:03:13.052693 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:13.052908 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:13.053973 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:13.055500 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/master-0-root/instance:
uuid: "4edbbce1d5244c858bfefc32c43b1dc9"
format_stamp: "Formatted at 2026-07-09 13:03:13 on dist-test-slave-mlt7"
I20260709 13:03:13.061383 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:03:13.065382 28455 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:13.066445 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:03:13.066766 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/master-0-root
uuid: "4edbbce1d5244c858bfefc32c43b1dc9"
format_stamp: "Formatted at 2026-07-09 13:03:13 on dist-test-slave-mlt7"
I20260709 13:03:13.067055 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:13.078331 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:13.079748 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:13.126323 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.190:36419
I20260709 13:03:13.126816 28506 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.190:36419 every 8 connection(s)
I20260709 13:03:13.132153 28507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.142813 28507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9: Bootstrap starting.
I20260709 13:03:13.148200 28507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:13.153323 28507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9: No bootstrap required, opened a new log
I20260709 13:03:13.155531 28507 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edbbce1d5244c858bfefc32c43b1dc9" member_type: VOTER }
I20260709 13:03:13.156037 28507 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:13.156291 28507 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4edbbce1d5244c858bfefc32c43b1dc9, State: Initialized, Role: FOLLOWER
I20260709 13:03:13.157009 28507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edbbce1d5244c858bfefc32c43b1dc9" member_type: VOTER }
I20260709 13:03:13.157498 28507 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:03:13.157889 28507 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:03:13.158203 28507 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.162578 28507 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edbbce1d5244c858bfefc32c43b1dc9" member_type: VOTER }
I20260709 13:03:13.163453 28507 leader_election.cc:304] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4edbbce1d5244c858bfefc32c43b1dc9; no voters: 
I20260709 13:03:13.164750 28507 leader_election.cc:290] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:03:13.165225 28510 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:13.166704 28510 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [term 1 LEADER]: Becoming Leader. State: Replica: 4edbbce1d5244c858bfefc32c43b1dc9, State: Running, Role: LEADER
I20260709 13:03:13.167536 28510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edbbce1d5244c858bfefc32c43b1dc9" member_type: VOTER }
I20260709 13:03:13.168669 28507 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:03:13.173262 28511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4edbbce1d5244c858bfefc32c43b1dc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edbbce1d5244c858bfefc32c43b1dc9" member_type: VOTER } }
I20260709 13:03:13.174782 28511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:13.174711 28512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4edbbce1d5244c858bfefc32c43b1dc9. Latest consensus state: current_term: 1 leader_uuid: "4edbbce1d5244c858bfefc32c43b1dc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edbbce1d5244c858bfefc32c43b1dc9" member_type: VOTER } }
I20260709 13:03:13.175725 28512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:13.178339 28516 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:03:13.188174 28516 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:03:13.192198 27934 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:03:13.201138 28516 catalog_manager.cc:1383] Generated new cluster ID: 59ab8a0183f04e119de548350d5edb86
I20260709 13:03:13.201431 28516 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:03:13.222908 28516 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:03:13.224651 28516 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:03:13.245591 28516 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9: Generated new TSK 0
I20260709 13:03:13.246263 28516 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:03:13.259526 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:13.266906 28528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:13.267782 28529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:13.269574 28531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:13.271087 27934 server_base.cc:1061] running on GCE node
I20260709 13:03:13.272061 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:13.272506 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:13.272704 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602193272678 us; error 0 us; skew 500 ppm
I20260709 13:03:13.273234 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:13.277561 27934 webserver.cc:533] Webserver started at http://127.27.71.129:40893/ using document root <none> and password file <none>
I20260709 13:03:13.278278 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:13.278544 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:13.278880 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:13.280165 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-0-root/instance:
uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
format_stamp: "Formatted at 2026-07-09 13:03:13 on dist-test-slave-mlt7"
I20260709 13:03:13.285145 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:03:13.289427 28536 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:13.290934 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:03:13.291291 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-0-root
uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
format_stamp: "Formatted at 2026-07-09 13:03:13 on dist-test-slave-mlt7"
I20260709 13:03:13.291569 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:13.321677 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:13.323503 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:13.325338 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:13.328995 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:13.329289 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:13.329577 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:13.329773 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:13.382407 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.129:43231
I20260709 13:03:13.382526 28598 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.129:43231 every 8 connection(s)
I20260709 13:03:13.387715 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:13.394785 28603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:13.395742 28604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:13.399192 28606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:13.400242 27934 server_base.cc:1061] running on GCE node
I20260709 13:03:13.401582 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:13.401805 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:13.402032 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602193401995 us; error 0 us; skew 500 ppm
I20260709 13:03:13.402604 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:13.406113 27934 webserver.cc:533] Webserver started at http://127.27.71.130:45207/ using document root <none> and password file <none>
I20260709 13:03:13.406862 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:13.407131 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:13.407433 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:13.407791 28599 heartbeater.cc:344] Connected to a master server at 127.27.71.190:36419
I20260709 13:03:13.408358 28599 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:13.409109 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-1-root/instance:
uuid: "47af5783a79641d4bcd2f254ce2b8c26"
format_stamp: "Formatted at 2026-07-09 13:03:13 on dist-test-slave-mlt7"
I20260709 13:03:13.409185 28599 heartbeater.cc:507] Master 127.27.71.190:36419 requested a full tablet report, sending...
I20260709 13:03:13.411720 28472 ts_manager.cc:194] Registered new tserver with Master: b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231)
I20260709 13:03:13.413587 28472 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52244
I20260709 13:03:13.414940 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.003s
I20260709 13:03:13.419739 28611 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:13.420604 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:03:13.421007 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-1-root
uuid: "47af5783a79641d4bcd2f254ce2b8c26"
format_stamp: "Formatted at 2026-07-09 13:03:13 on dist-test-slave-mlt7"
I20260709 13:03:13.421296 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:13.438642 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:13.440090 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:13.441829 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:13.444492 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:13.444798 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:13.445057 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:13.445235 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:13.495733 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.130:35293
I20260709 13:03:13.496074 28673 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.130:35293 every 8 connection(s)
I20260709 13:03:13.500579 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:13.509720 28677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:13.511597 28678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:13.513159 28680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:13.513849 27934 server_base.cc:1061] running on GCE node
I20260709 13:03:13.515321 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:13.515591 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:13.515604 28674 heartbeater.cc:344] Connected to a master server at 127.27.71.190:36419
I20260709 13:03:13.516562 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602193516495 us; error 0 us; skew 500 ppm
I20260709 13:03:13.516788 28674 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:13.517550 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:13.517562 28674 heartbeater.cc:507] Master 127.27.71.190:36419 requested a full tablet report, sending...
I20260709 13:03:13.520699 28472 ts_manager.cc:194] Registered new tserver with Master: 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:13.521814 27934 webserver.cc:533] Webserver started at http://127.27.71.131:42845/ using document root <none> and password file <none>
I20260709 13:03:13.522588 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:13.522907 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:13.522900 28472 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52250
I20260709 13:03:13.523439 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:13.525131 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-2-root/instance:
uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
format_stamp: "Formatted at 2026-07-09 13:03:13 on dist-test-slave-mlt7"
I20260709 13:03:13.530628 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:03:13.534472 28685 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:13.535382 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:03:13.535749 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-2-root
uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
format_stamp: "Formatted at 2026-07-09 13:03:13 on dist-test-slave-mlt7"
I20260709 13:03:13.536139 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602179482400-27934-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:13.574069 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:13.575371 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:13.577091 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:13.580369 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:13.580694 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:13.580950 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:13.581127 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:13.651250 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.131:45279
I20260709 13:03:13.651444 28747 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.131:45279 every 8 connection(s)
I20260709 13:03:13.667817 28748 heartbeater.cc:344] Connected to a master server at 127.27.71.190:36419
I20260709 13:03:13.668258 28748 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:13.669209 28748 heartbeater.cc:507] Master 127.27.71.190:36419 requested a full tablet report, sending...
I20260709 13:03:13.671847 28472 ts_manager.cc:194] Registered new tserver with Master: c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:13.672787 27934 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018530918s
I20260709 13:03:13.673477 28472 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52262
I20260709 13:03:13.703276 28472 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52266:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:03:13.706509 28472 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:03:13.800611 28713 tablet_service.cc:1511] Processing CreateTablet for tablet c0c7407fbedd4acc99084e230bcf1421 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:03:13.801506 28712 tablet_service.cc:1511] Processing CreateTablet for tablet 0ecfce63e8b94e33825cfa5d68a3c097 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:03:13.803643 28564 tablet_service.cc:1511] Processing CreateTablet for tablet c0c7407fbedd4acc99084e230bcf1421 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:03:13.804495 28711 tablet_service.cc:1511] Processing CreateTablet for tablet 0065b221305f415fbed169dba9a764c7 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:03:13.805117 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c7407fbedd4acc99084e230bcf1421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.807080 28708 tablet_service.cc:1511] Processing CreateTablet for tablet 62cd4db841a54f90bc187b657a2adb88 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:03:13.808459 28563 tablet_service.cc:1511] Processing CreateTablet for tablet 0ecfce63e8b94e33825cfa5d68a3c097 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:03:13.809844 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecfce63e8b94e33825cfa5d68a3c097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.809190 28709 tablet_service.cc:1511] Processing CreateTablet for tablet 7d76f50c03914a21812cb7a0af509fb5 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:03:13.811939 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 0065b221305f415fbed169dba9a764c7 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:03:13.813429 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0065b221305f415fbed169dba9a764c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.813524 28710 tablet_service.cc:1511] Processing CreateTablet for tablet 44db1cf806fa4650a3552062cf11ca43 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:03:13.802316 28713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c7407fbedd4acc99084e230bcf1421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.816161 28706 tablet_service.cc:1511] Processing CreateTablet for tablet 23d3c7d3a61e4b66b8e50ed6ce0c8f0f (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:03:13.817546 28706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23d3c7d3a61e4b66b8e50ed6ce0c8f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.817308 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 44db1cf806fa4650a3552062cf11ca43 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:03:13.818617 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44db1cf806fa4650a3552062cf11ca43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.819645 28557 tablet_service.cc:1511] Processing CreateTablet for tablet 23d3c7d3a61e4b66b8e50ed6ce0c8f0f (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:03:13.820972 28557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23d3c7d3a61e4b66b8e50ed6ce0c8f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.822471 28638 tablet_service.cc:1511] Processing CreateTablet for tablet 0ecfce63e8b94e33825cfa5d68a3c097 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:03:13.823637 28639 tablet_service.cc:1511] Processing CreateTablet for tablet c0c7407fbedd4acc99084e230bcf1421 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:03:13.825090 28558 tablet_service.cc:1511] Processing CreateTablet for tablet ab7d879ac8ed474ba5021fbf3fb2d35e (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:03:13.825232 28637 tablet_service.cc:1511] Processing CreateTablet for tablet 0065b221305f415fbed169dba9a764c7 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:03:13.826465 28558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7d879ac8ed474ba5021fbf3fb2d35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.823789 28638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecfce63e8b94e33825cfa5d68a3c097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.828526 28639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c7407fbedd4acc99084e230bcf1421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.827440 28636 tablet_service.cc:1511] Processing CreateTablet for tablet 44db1cf806fa4650a3552062cf11ca43 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:03:13.829170 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 7d76f50c03914a21812cb7a0af509fb5 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:03:13.830106 28636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44db1cf806fa4650a3552062cf11ca43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.830555 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d76f50c03914a21812cb7a0af509fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.833293 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 62cd4db841a54f90bc187b657a2adb88 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:03:13.833734 28634 tablet_service.cc:1511] Processing CreateTablet for tablet 62cd4db841a54f90bc187b657a2adb88 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:03:13.834784 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62cd4db841a54f90bc187b657a2adb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.835086 28634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62cd4db841a54f90bc187b657a2adb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.829190 28633 tablet_service.cc:1511] Processing CreateTablet for tablet ab7d879ac8ed474ba5021fbf3fb2d35e (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:03:13.837358 28633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7d879ac8ed474ba5021fbf3fb2d35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.837146 28635 tablet_service.cc:1511] Processing CreateTablet for tablet 7d76f50c03914a21812cb7a0af509fb5 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:03:13.838377 28637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0065b221305f415fbed169dba9a764c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.812023 28707 tablet_service.cc:1511] Processing CreateTablet for tablet ab7d879ac8ed474ba5021fbf3fb2d35e (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:03:13.838632 28711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0065b221305f415fbed169dba9a764c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.839723 28635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d76f50c03914a21812cb7a0af509fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.839900 28707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7d879ac8ed474ba5021fbf3fb2d35e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.841223 28710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44db1cf806fa4650a3552062cf11ca43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.842638 28708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62cd4db841a54f90bc187b657a2adb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.844125 28709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d76f50c03914a21812cb7a0af509fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.845737 28712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecfce63e8b94e33825cfa5d68a3c097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.853659 28632 tablet_service.cc:1511] Processing CreateTablet for tablet 23d3c7d3a61e4b66b8e50ed6ce0c8f0f (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:03:13.855144 28632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23d3c7d3a61e4b66b8e50ed6ce0c8f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.897818 28556 tablet_service.cc:1511] Processing CreateTablet for tablet d8dc76668dcf4cfd91850962987de661 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:03:13.899672 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8dc76668dcf4cfd91850962987de661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.933641 28705 tablet_service.cc:1511] Processing CreateTablet for tablet d8dc76668dcf4cfd91850962987de661 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:03:13.934996 28705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8dc76668dcf4cfd91850962987de661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.937588 28631 tablet_service.cc:1511] Processing CreateTablet for tablet 626ec34cdc60466bb40187641f4b40d2 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:03:13.939422 28631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626ec34cdc60466bb40187641f4b40d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.945948 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 626ec34cdc60466bb40187641f4b40d2 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:03:13.947417 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626ec34cdc60466bb40187641f4b40d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.959144 28763 tablet_bootstrap.cc:492] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87: Bootstrap starting.
I20260709 13:03:13.963382 28636 tablet_service.cc:1511] Processing CreateTablet for tablet d8dc76668dcf4cfd91850962987de661 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:03:13.965099 28636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8dc76668dcf4cfd91850962987de661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.977631 28764 tablet_bootstrap.cc:492] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26: Bootstrap starting.
I20260709 13:03:13.981163 28711 tablet_service.cc:1511] Processing CreateTablet for tablet 626ec34cdc60466bb40187641f4b40d2 (DEFAULT_TABLE table=test-workload [id=2d6fc28f04ed434092bdaaa1af96319e]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:03:13.983340 28711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626ec34cdc60466bb40187641f4b40d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:13.992188 28765 tablet_bootstrap.cc:492] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf: Bootstrap starting.
I20260709 13:03:14.005831 28763 tablet_bootstrap.cc:654] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.011076 28764 tablet_bootstrap.cc:654] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.007896 28765 tablet_bootstrap.cc:654] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:14.067437 28459 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.27.71.129:43231, user_credentials={real_user=slave}} blocked reactor thread for 50106.9us
W20260709 13:03:14.070389 28457 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.27.71.130:35293, user_credentials={real_user=slave}} blocked reactor thread for 52398us
I20260709 13:03:14.072037 28763 tablet_bootstrap.cc:492] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87: No bootstrap required, opened a new log
I20260709 13:03:14.077816 28763 ts_tablet_manager.cc:1403] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent bootstrapping tablet: real 0.119s	user 0.022s	sys 0.056s
I20260709 13:03:14.080590 28764 tablet_bootstrap.cc:492] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26: No bootstrap required, opened a new log
I20260709 13:03:14.081113 28764 ts_tablet_manager.cc:1403] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent bootstrapping tablet: real 0.104s	user 0.030s	sys 0.045s
I20260709 13:03:14.081568 28765 tablet_bootstrap.cc:492] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf: No bootstrap required, opened a new log
I20260709 13:03:14.081522 28763 raft_consensus.cc:359] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.082197 28765 ts_tablet_manager.cc:1403] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent bootstrapping tablet: real 0.091s	user 0.017s	sys 0.004s
I20260709 13:03:14.082537 28763 raft_consensus.cc:385] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.082923 28763 raft_consensus.cc:740] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.083806 28764 raft_consensus.cc:359] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:14.084473 28764 raft_consensus.cc:385] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.083937 28763 consensus_queue.cc:260] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.084749 28764 raft_consensus.cc:740] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.085738 28765 raft_consensus.cc:359] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.086898 28765 raft_consensus.cc:385] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.087067 28764 consensus_queue.cc:260] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:14.087414 28765 raft_consensus.cc:740] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.087940 28770 raft_consensus.cc:493] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.088564 28770 raft_consensus.cc:515] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:14.088869 28765 consensus_queue.cc:260] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.099725 28674 heartbeater.cc:499] Master 127.27.71.190:36419 was elected leader, sending a full tablet report...
I20260709 13:03:14.100169 28764 ts_tablet_manager.cc:1434] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.008s
I20260709 13:03:14.101715 28764 tablet_bootstrap.cc:492] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26: Bootstrap starting.
I20260709 13:03:14.099975 28599 heartbeater.cc:499] Master 127.27.71.190:36419 was elected leader, sending a full tablet report...
I20260709 13:03:14.104187 28770 leader_election.cc:290] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279), b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231)
I20260709 13:03:14.108875 28765 ts_tablet_manager.cc:1434] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.003s
I20260709 13:03:14.109665 28748 heartbeater.cc:499] Master 127.27.71.190:36419 was elected leader, sending a full tablet report...
I20260709 13:03:14.114656 28765 tablet_bootstrap.cc:492] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf: Bootstrap starting.
I20260709 13:03:14.124696 28763 ts_tablet_manager.cc:1434] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.010s
I20260709 13:03:14.126004 28763 tablet_bootstrap.cc:492] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87: Bootstrap starting.
I20260709 13:03:14.126260 28765 tablet_bootstrap.cc:654] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.131788 28764 tablet_bootstrap.cc:654] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.136497 28763 tablet_bootstrap.cc:654] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.144416 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" is_pre_election: true
I20260709 13:03:14.145274 28764 tablet_bootstrap.cc:492] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26: No bootstrap required, opened a new log
I20260709 13:03:14.145813 28764 ts_tablet_manager.cc:1403] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.004s
W20260709 13:03:14.146483 28614 leader_election.cc:343] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.151731 28764 raft_consensus.cc:359] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.152637 28764 raft_consensus.cc:385] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.152698 28763 tablet_bootstrap.cc:492] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87: No bootstrap required, opened a new log
I20260709 13:03:14.153126 28764 raft_consensus.cc:740] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.153363 28763 ts_tablet_manager.cc:1403] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.007s
I20260709 13:03:14.154042 28764 consensus_queue.cc:260] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.156715 28764 ts_tablet_manager.cc:1434] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.002s
I20260709 13:03:14.157236 28763 raft_consensus.cc:359] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.157716 28764 tablet_bootstrap.cc:492] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26: Bootstrap starting.
I20260709 13:03:14.157980 28763 raft_consensus.cc:385] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.158354 28763 raft_consensus.cc:740] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.159190 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" is_pre_election: true
I20260709 13:03:14.159103 28763 consensus_queue.cc:260] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
W20260709 13:03:14.160918 28615 leader_election.cc:343] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.161332 28763 ts_tablet_manager.cc:1434] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:03:14.161396 28615 leader_election.cc:304] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26; no voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf
I20260709 13:03:14.162402 28763 tablet_bootstrap.cc:492] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87: Bootstrap starting.
I20260709 13:03:14.162576 28770 raft_consensus.cc:2749] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:14.162645 28765 tablet_bootstrap.cc:492] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf: No bootstrap required, opened a new log
I20260709 13:03:14.164382 28765 ts_tablet_manager.cc:1403] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.010s
I20260709 13:03:14.166584 28764 tablet_bootstrap.cc:654] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.167491 28765 raft_consensus.cc:359] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.168407 28765 raft_consensus.cc:385] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.168802 28765 raft_consensus.cc:740] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.169788 28763 tablet_bootstrap.cc:654] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.169706 28765 consensus_queue.cc:260] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.173656 28765 ts_tablet_manager.cc:1434] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:03:14.175854 28765 tablet_bootstrap.cc:492] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf: Bootstrap starting.
I20260709 13:03:14.185511 28765 tablet_bootstrap.cc:654] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:14.190460 28520 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:14.199967 28765 tablet_bootstrap.cc:492] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf: No bootstrap required, opened a new log
I20260709 13:03:14.200996 28765 ts_tablet_manager.cc:1403] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.008s
I20260709 13:03:14.205596 28769 raft_consensus.cc:493] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.205385 28765 raft_consensus.cc:359] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.206125 28769 raft_consensus.cc:515] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.206799 28765 raft_consensus.cc:385] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.207310 28765 raft_consensus.cc:740] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.208104 28765 consensus_queue.cc:260] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.208874 28769 leader_election.cc:290] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.209826 28763 tablet_bootstrap.cc:492] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87: No bootstrap required, opened a new log
I20260709 13:03:14.210345 28763 ts_tablet_manager.cc:1403] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent bootstrapping tablet: real 0.048s	user 0.030s	sys 0.014s
I20260709 13:03:14.210619 28765 ts_tablet_manager.cc:1434] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:03:14.211613 28765 tablet_bootstrap.cc:492] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf: Bootstrap starting.
I20260709 13:03:14.213269 28763 raft_consensus.cc:359] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.213995 28763 raft_consensus.cc:385] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.214282 28763 raft_consensus.cc:740] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.215128 28763 consensus_queue.cc:260] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.217697 28763 ts_tablet_manager.cc:1434] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:03:14.218194 28765 tablet_bootstrap.cc:654] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.218750 28763 tablet_bootstrap.cc:492] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87: Bootstrap starting.
I20260709 13:03:14.223654 28764 tablet_bootstrap.cc:492] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26: No bootstrap required, opened a new log
I20260709 13:03:14.227185 28764 ts_tablet_manager.cc:1403] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26: Time spent bootstrapping tablet: real 0.070s	user 0.022s	sys 0.025s
I20260709 13:03:14.228264 28763 tablet_bootstrap.cc:654] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.249745 28765 tablet_bootstrap.cc:492] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf: No bootstrap required, opened a new log
I20260709 13:03:14.250456 28765 ts_tablet_manager.cc:1403] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.015s
I20260709 13:03:14.253432 28764 raft_consensus.cc:359] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.254281 28764 raft_consensus.cc:385] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.253888 28765 raft_consensus.cc:359] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.254344 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44db1cf806fa4650a3552062cf11ca43" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26" is_pre_election: true
I20260709 13:03:14.254617 28764 raft_consensus.cc:740] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.254802 28765 raft_consensus.cc:385] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.255187 28649 raft_consensus.cc:2468] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 0.
I20260709 13:03:14.255308 28765 raft_consensus.cc:740] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.255434 28764 consensus_queue.cc:260] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.256397 28765 consensus_queue.cc:260] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.257542 28538 leader_election.cc:304] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:14.258661 28769 raft_consensus.cc:2804] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:14.259029 28769 raft_consensus.cc:493] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.259222 28764 ts_tablet_manager.cc:1434] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:03:14.259289 28765 ts_tablet_manager.cc:1434] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:03:14.260627 28764 tablet_bootstrap.cc:492] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26: Bootstrap starting.
I20260709 13:03:14.260627 28765 tablet_bootstrap.cc:492] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf: Bootstrap starting.
I20260709 13:03:14.262609 28763 tablet_bootstrap.cc:492] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87: No bootstrap required, opened a new log
I20260709 13:03:14.263098 28763 ts_tablet_manager.cc:1403] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.006s
I20260709 13:03:14.266211 28763 raft_consensus.cc:359] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.267004 28763 raft_consensus.cc:385] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.267323 28763 raft_consensus.cc:740] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.268529 28765 tablet_bootstrap.cc:654] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.272392 28764 tablet_bootstrap.cc:654] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.274714 28772 raft_consensus.cc:493] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.275234 28772 raft_consensus.cc:515] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.275117 28763 consensus_queue.cc:260] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.276275 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44db1cf806fa4650a3552062cf11ca43" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" is_pre_election: true
I20260709 13:03:14.277148 28723 raft_consensus.cc:2468] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 0.
I20260709 13:03:14.280349 28769 raft_consensus.cc:3060] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.280005 28772 leader_election.cc:290] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:14.296571 28763 ts_tablet_manager.cc:1434] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent starting tablet: real 0.033s	user 0.014s	sys 0.009s
I20260709 13:03:14.298369 28763 tablet_bootstrap.cc:492] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87: Bootstrap starting.
I20260709 13:03:14.314117 28769 raft_consensus.cc:515] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.314945 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d76f50c03914a21812cb7a0af509fb5" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26" is_pre_election: true
W20260709 13:03:14.318425 28687 leader_election.cc:343] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.319703 28765 tablet_bootstrap.cc:492] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf: No bootstrap required, opened a new log
I20260709 13:03:14.320335 28765 ts_tablet_manager.cc:1403] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent bootstrapping tablet: real 0.060s	user 0.011s	sys 0.008s
I20260709 13:03:14.320722 28764 tablet_bootstrap.cc:492] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26: No bootstrap required, opened a new log
I20260709 13:03:14.321317 28764 ts_tablet_manager.cc:1403] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.010s
I20260709 13:03:14.324489 28764 raft_consensus.cc:359] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.325872 28763 tablet_bootstrap.cc:654] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.325685 28764 raft_consensus.cc:385] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.326210 28764 raft_consensus.cc:740] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.326087 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d76f50c03914a21812cb7a0af509fb5" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" is_pre_election: true
I20260709 13:03:14.326920 28574 raft_consensus.cc:2468] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 0.
I20260709 13:03:14.327242 28764 consensus_queue.cc:260] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.328883 28689 leader_election.cc:304] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 47af5783a79641d4bcd2f254ce2b8c26
I20260709 13:03:14.329372 28765 raft_consensus.cc:359] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.330365 28765 raft_consensus.cc:385] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.330453 28764 ts_tablet_manager.cc:1434] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:03:14.330724 28765 raft_consensus.cc:740] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.331219 28772 raft_consensus.cc:2804] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:14.331590 28769 leader_election.cc:290] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 election: Requested vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.332147 28764 tablet_bootstrap.cc:492] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26: Bootstrap starting.
I20260709 13:03:14.331681 28765 consensus_queue.cc:260] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.331585 28772 raft_consensus.cc:493] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.332882 28772 raft_consensus.cc:3060] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.332767 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44db1cf806fa4650a3552062cf11ca43" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:14.336354 28649 raft_consensus.cc:3060] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.342605 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44db1cf806fa4650a3552062cf11ca43" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:14.343575 28723 raft_consensus.cc:3060] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.352177 28764 tablet_bootstrap.cc:654] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.357594 28649 raft_consensus.cc:2468] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 1.
I20260709 13:03:14.361392 28723 raft_consensus.cc:2468] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 1.
I20260709 13:03:14.368253 28763 tablet_bootstrap.cc:492] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87: No bootstrap required, opened a new log
I20260709 13:03:14.376931 28538 leader_election.cc:304] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:14.368372 28770 raft_consensus.cc:493] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.370132 28772 raft_consensus.cc:515] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.378439 28769 raft_consensus.cc:2804] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:14.378612 28765 ts_tablet_manager.cc:1434] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent starting tablet: real 0.058s	user 0.036s	sys 0.013s
I20260709 13:03:14.378959 28769 raft_consensus.cc:697] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:14.379685 28765 tablet_bootstrap.cc:492] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf: Bootstrap starting.
I20260709 13:03:14.379748 28772 leader_election.cc:290] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:14.379853 28769 consensus_queue.cc:237] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.380594 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d76f50c03914a21812cb7a0af509fb5" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:14.374815 28764 tablet_bootstrap.cc:492] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26: No bootstrap required, opened a new log
I20260709 13:03:14.381228 28574 raft_consensus.cc:3060] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.381652 28764 ts_tablet_manager.cc:1403] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.007s
I20260709 13:03:14.377202 28763 ts_tablet_manager.cc:1403] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent bootstrapping tablet: real 0.079s	user 0.020s	sys 0.028s
I20260709 13:03:14.385133 28764 raft_consensus.cc:359] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.385564 28763 raft_consensus.cc:359] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.386200 28764 raft_consensus.cc:385] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.386329 28763 raft_consensus.cc:385] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.377679 28770 raft_consensus.cc:515] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.386633 28764 raft_consensus.cc:740] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.386826 28765 tablet_bootstrap.cc:654] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.387285 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d76f50c03914a21812cb7a0af509fb5" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:14.387782 28764 consensus_queue.cc:260] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.388734 28770 leader_election.cc:290] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.389881 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065b221305f415fbed169dba9a764c7" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" is_pre_election: true
I20260709 13:03:14.390002 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065b221305f415fbed169dba9a764c7" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" is_pre_election: true
I20260709 13:03:14.391006 28723 raft_consensus.cc:2468] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 0.
W20260709 13:03:14.391955 28687 leader_election.cc:343] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.394845 28764 ts_tablet_manager.cc:1434] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 13:03:14.396602 28764 tablet_bootstrap.cc:492] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26: Bootstrap starting.
I20260709 13:03:14.396345 28614 leader_election.cc:304] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:14.398211 28770 raft_consensus.cc:2804] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:14.398582 28770 raft_consensus.cc:493] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.398926 28770 raft_consensus.cc:3060] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.405797 28764 tablet_bootstrap.cc:654] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.386700 28763 raft_consensus.cc:740] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.410247 28763 consensus_queue.cc:260] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.411427 28573 raft_consensus.cc:2468] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 0.
I20260709 13:03:14.415081 28771 raft_consensus.cc:493] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.415400 28763 ts_tablet_manager.cc:1434] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.003s
I20260709 13:03:14.416998 28763 tablet_bootstrap.cc:492] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87: Bootstrap starting.
I20260709 13:03:14.373031 28793 raft_consensus.cc:493] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.418154 28793 raft_consensus.cc:515] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.415814 28771 raft_consensus.cc:515] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.426195 28572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" is_pre_election: true
W20260709 13:03:14.430718 28689 leader_election.cc:343] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.431243 28572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" is_pre_election: true
I20260709 13:03:14.430662 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26" is_pre_election: true
I20260709 13:03:14.432197 28572 raft_consensus.cc:2468] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 0.
W20260709 13:03:14.433730 28687 leader_election.cc:343] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.434363 28687 leader_election.cc:304] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c0e27b43534a4feeb11707f73bd7ccbf; no voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87
I20260709 13:03:14.435597 28772 raft_consensus.cc:2749] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:14.435882 28793 leader_election.cc:290] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:14.437520 28468 catalog_manager.cc:5697] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.441504 28615 leader_election.cc:304] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:14.443861 28771 leader_election.cc:290] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.447687 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" is_pre_election: true
I20260709 13:03:14.448069 28771 raft_consensus.cc:2804] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:14.449447 28771 raft_consensus.cc:493] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.451364 28771 raft_consensus.cc:3060] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.447686 28763 tablet_bootstrap.cc:654] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:14.450414 28614 leader_election.cc:343] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.462375 28770 raft_consensus.cc:515] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.462364 28574 raft_consensus.cc:2468] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 1.
I20260709 13:03:14.482801 28689 leader_election.cc:304] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 47af5783a79641d4bcd2f254ce2b8c26
I20260709 13:03:14.486035 28793 raft_consensus.cc:2804] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:14.487020 28793 raft_consensus.cc:697] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 LEADER]: Becoming Leader. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Running, Role: LEADER
I20260709 13:03:14.489440 28793 consensus_queue.cc:237] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.491268 28764 tablet_bootstrap.cc:492] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26: No bootstrap required, opened a new log
I20260709 13:03:14.492017 28764 ts_tablet_manager.cc:1403] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent bootstrapping tablet: real 0.096s	user 0.043s	sys 0.045s
I20260709 13:03:14.492131 28765 tablet_bootstrap.cc:492] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf: No bootstrap required, opened a new log
I20260709 13:03:14.492897 28765 ts_tablet_manager.cc:1403] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf: Time spent bootstrapping tablet: real 0.113s	user 0.063s	sys 0.038s
I20260709 13:03:14.496129 28764 raft_consensus.cc:359] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.497044 28764 raft_consensus.cc:385] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.497403 28764 raft_consensus.cc:740] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.497557 28765 raft_consensus.cc:359] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.498413 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065b221305f415fbed169dba9a764c7" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:14.498558 28765 raft_consensus.cc:385] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.498322 28764 consensus_queue.cc:260] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.499298 28573 raft_consensus.cc:3060] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.499212 28765 raft_consensus.cc:740] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.500594 28765 consensus_queue.cc:260] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.503703 28765 ts_tablet_manager.cc:1434] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:03:14.505088 28765 tablet_bootstrap.cc:492] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf: Bootstrap starting.
I20260709 13:03:14.505430 28764 ts_tablet_manager.cc:1434] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:03:14.506543 28764 tablet_bootstrap.cc:492] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26: Bootstrap starting.
I20260709 13:03:14.512807 28573 raft_consensus.cc:2468] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 1.
I20260709 13:03:14.514362 28615 leader_election.cc:304] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:14.515208 28765 tablet_bootstrap.cc:654] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.515372 28797 raft_consensus.cc:2804] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:14.522512 28769 raft_consensus.cc:493] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.523377 28769 raft_consensus.cc:515] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.529968 28769 leader_election.cc:290] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.530459 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065b221305f415fbed169dba9a764c7" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:14.531097 28723 raft_consensus.cc:3060] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.531834 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26" is_pre_election: true
I20260709 13:03:14.533705 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" is_pre_election: true
I20260709 13:03:14.534503 28722 raft_consensus.cc:2468] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 0.
W20260709 13:03:14.534724 28538 leader_election.cc:343] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.536276 28539 leader_election.cc:304] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 47af5783a79641d4bcd2f254ce2b8c26
I20260709 13:03:14.537374 28764 tablet_bootstrap.cc:654] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.540279 28769 raft_consensus.cc:2804] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:14.540688 28769 raft_consensus.cc:493] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.541074 28769 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.547406 28765 tablet_bootstrap.cc:492] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf: No bootstrap required, opened a new log
I20260709 13:03:14.547721 28763 tablet_bootstrap.cc:492] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87: No bootstrap required, opened a new log
I20260709 13:03:14.548228 28765 ts_tablet_manager.cc:1403] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent bootstrapping tablet: real 0.043s	user 0.026s	sys 0.015s
I20260709 13:03:14.548234 28763 ts_tablet_manager.cc:1403] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent bootstrapping tablet: real 0.132s	user 0.023s	sys 0.021s
I20260709 13:03:14.552443 28765 raft_consensus.cc:359] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:14.552889 28763 raft_consensus.cc:359] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.553354 28765 raft_consensus.cc:385] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.553653 28763 raft_consensus.cc:385] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.554051 28765 raft_consensus.cc:740] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.554075 28763 raft_consensus.cc:740] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.555068 28765 consensus_queue.cc:260] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:14.555171 28763 consensus_queue.cc:260] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.533710 28771 raft_consensus.cc:515] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.558741 28765 ts_tablet_manager.cc:1434] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:03:14.558725 28763 ts_tablet_manager.cc:1434] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:03:14.559899 28765 tablet_bootstrap.cc:492] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf: Bootstrap starting.
I20260709 13:03:14.559897 28763 tablet_bootstrap.cc:492] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87: Bootstrap starting.
I20260709 13:03:14.560202 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:14.561003 28573 raft_consensus.cc:3060] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.567469 28763 tablet_bootstrap.cc:654] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.567469 28765 tablet_bootstrap.cc:654] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.573639 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:14.574734 28722 raft_consensus.cc:3060] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.577126 28573 raft_consensus.cc:2468] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 1.
I20260709 13:03:14.581673 28615 leader_election.cc:304] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:14.583025 28801 raft_consensus.cc:2804] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:14.592342 28797 raft_consensus.cc:697] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Becoming Leader. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:14.570932 28770 leader_election.cc:290] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.598394 28723 raft_consensus.cc:2468] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 1.
I20260709 13:03:14.600773 28770 raft_consensus.cc:493] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.601462 28770 raft_consensus.cc:515] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.605232 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" is_pre_election: true
W20260709 13:03:14.607365 28615 leader_election.cc:343] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.593632 28797 consensus_queue.cc:237] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.616878 28801 raft_consensus.cc:697] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Becoming Leader. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:14.618150 28801 consensus_queue.cc:237] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.571190 28771 leader_election.cc:290] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.637990 28769 raft_consensus.cc:515] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.638752 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" is_pre_election: true
I20260709 13:03:14.640008 28723 raft_consensus.cc:2468] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 0.
I20260709 13:03:14.639247 28770 leader_election.cc:290] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.640832 28769 leader_election.cc:290] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 election: Requested vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.636448 28722 raft_consensus.cc:2468] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 1.
I20260709 13:03:14.651633 28764 tablet_bootstrap.cc:492] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26: No bootstrap required, opened a new log
I20260709 13:03:14.653268 28763 tablet_bootstrap.cc:492] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87: No bootstrap required, opened a new log
I20260709 13:03:14.638641 28469 catalog_manager.cc:5697] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf reported cstate change: term changed from 0 to 1, leader changed from <none> to c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131). New cstate: current_term: 1 leader_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:14.645684 28614 leader_election.cc:304] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, c0e27b43534a4feeb11707f73bd7ccbf; no voters: b0ee9adadeea4d7c8446f04a6f0a3c87
I20260709 13:03:14.657735 28764 ts_tablet_manager.cc:1403] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent bootstrapping tablet: real 0.151s	user 0.015s	sys 0.045s
I20260709 13:03:14.659039 28765 tablet_bootstrap.cc:492] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf: No bootstrap required, opened a new log
I20260709 13:03:14.659595 28765 ts_tablet_manager.cc:1403] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf: Time spent bootstrapping tablet: real 0.100s	user 0.029s	sys 0.052s
I20260709 13:03:14.660459 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
W20260709 13:03:14.661947 28538 leader_election.cc:343] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.661526 28764 raft_consensus.cc:359] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.662551 28765 raft_consensus.cc:359] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.663102 28764 raft_consensus.cc:385] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.663318 28765 raft_consensus.cc:385] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.663431 28764 raft_consensus.cc:740] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.664134 28763 ts_tablet_manager.cc:1403] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent bootstrapping tablet: real 0.104s	user 0.013s	sys 0.047s
I20260709 13:03:14.664425 28765 raft_consensus.cc:740] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.664489 28764 consensus_queue.cc:260] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.665844 28765 consensus_queue.cc:260] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.666356 28797 raft_consensus.cc:2804] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:14.666751 28797 raft_consensus.cc:493] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.667083 28797 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.667686 28763 raft_consensus.cc:359] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:14.668112 28765 ts_tablet_manager.cc:1434] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.001s
I20260709 13:03:14.668459 28763 raft_consensus.cc:385] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.668812 28763 raft_consensus.cc:740] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.655618 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:14.669751 28722 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.672062 28764 ts_tablet_manager.cc:1434] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.006s
I20260709 13:03:14.673095 28764 tablet_bootstrap.cc:492] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26: Bootstrap starting.
I20260709 13:03:14.673664 28468 catalog_manager.cc:5697] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.669299 28765 tablet_bootstrap.cc:492] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf: Bootstrap starting.
I20260709 13:03:14.677446 28763 consensus_queue.cc:260] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:14.681288 28797 raft_consensus.cc:515] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.682094 28763 ts_tablet_manager.cc:1434] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.005s
I20260709 13:03:14.683106 28763 tablet_bootstrap.cc:492] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87: Bootstrap starting.
I20260709 13:03:14.688330 28765 tablet_bootstrap.cc:654] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.689688 28763 tablet_bootstrap.cc:654] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.692373 28722 raft_consensus.cc:2468] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 1.
I20260709 13:03:14.695667 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:14.696405 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:14.697237 28722 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:14.697824 28615 leader_election.cc:343] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.699620 28539 leader_election.cc:304] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 47af5783a79641d4bcd2f254ce2b8c26
I20260709 13:03:14.703024 28769 raft_consensus.cc:2804] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:14.703531 28769 raft_consensus.cc:697] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:14.704377 28769 consensus_queue.cc:237] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.707989 28722 raft_consensus.cc:2468] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 1.
I20260709 13:03:14.709554 28614 leader_election.cc:304] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, c0e27b43534a4feeb11707f73bd7ccbf; no voters: b0ee9adadeea4d7c8446f04a6f0a3c87
I20260709 13:03:14.710572 28797 leader_election.cc:290] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.710588 28770 raft_consensus.cc:2804] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:14.711403 28770 raft_consensus.cc:697] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Becoming Leader. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:14.692373 28764 tablet_bootstrap.cc:654] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.711072 28472 catalog_manager.cc:5697] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.712255 28770 consensus_queue.cc:237] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.712755 28786 raft_consensus.cc:493] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.713259 28786 raft_consensus.cc:515] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.715528 28786 leader_election.cc:290] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.717582 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26" is_pre_election: true
I20260709 13:03:14.718269 28649 raft_consensus.cc:2468] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 0.
I20260709 13:03:14.719323 28765 tablet_bootstrap.cc:492] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf: No bootstrap required, opened a new log
I20260709 13:03:14.719785 28765 ts_tablet_manager.cc:1403] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.010s
I20260709 13:03:14.719622 28538 leader_election.cc:304] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:14.720551 28786 raft_consensus.cc:2804] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:14.720945 28786 raft_consensus.cc:493] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.722023 28786 raft_consensus.cc:3060] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.722749 28765 raft_consensus.cc:359] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.723465 28765 raft_consensus.cc:385] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.723788 28765 raft_consensus.cc:740] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.724620 28765 consensus_queue.cc:260] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.729324 28472 catalog_manager.cc:5697] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.717582 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" is_pre_election: true
I20260709 13:03:14.732832 28722 raft_consensus.cc:2468] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 0.
I20260709 13:03:14.734139 28770 raft_consensus.cc:493] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.734622 28770 raft_consensus.cc:515] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.737003 28764 tablet_bootstrap.cc:492] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26: No bootstrap required, opened a new log
I20260709 13:03:14.737484 28764 ts_tablet_manager.cc:1403] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent bootstrapping tablet: real 0.065s	user 0.006s	sys 0.004s
I20260709 13:03:14.738438 28765 ts_tablet_manager.cc:1434] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.003s
I20260709 13:03:14.738370 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" is_pre_election: true
I20260709 13:03:14.740767 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" is_pre_election: true
I20260709 13:03:14.741561 28722 raft_consensus.cc:2393] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 47af5783a79641d4bcd2f254ce2b8c26 in current term 1: Already voted for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in this term.
I20260709 13:03:14.742503 28763 tablet_bootstrap.cc:492] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87: No bootstrap required, opened a new log
I20260709 13:03:14.742697 28614 leader_election.cc:304] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26; no voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf
I20260709 13:03:14.743273 28786 raft_consensus.cc:515] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.743566 28801 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.744481 28764 raft_consensus.cc:359] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.745175 28764 raft_consensus.cc:385] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.745468 28764 raft_consensus.cc:740] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.745761 28765 tablet_bootstrap.cc:492] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf: Bootstrap starting.
I20260709 13:03:14.742964 28763 ts_tablet_manager.cc:1403] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.021s
I20260709 13:03:14.748107 28764 consensus_queue.cc:260] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.748064 28770 leader_election.cc:290] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.749891 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:14.750415 28649 raft_consensus.cc:3060] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.752039 28764 ts_tablet_manager.cc:1434] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:03:14.753067 28764 tablet_bootstrap.cc:492] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26: Bootstrap starting.
I20260709 13:03:14.758160 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:14.756134 28468 catalog_manager.cc:5697] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.760684 28764 tablet_bootstrap.cc:654] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.760108 28722 raft_consensus.cc:3060] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.767652 28765 tablet_bootstrap.cc:654] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.768134 28649 raft_consensus.cc:2468] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 1.
I20260709 13:03:14.754595 28786 leader_election.cc:290] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 election: Requested vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:14.762706 28801 raft_consensus.cc:2749] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:14.772817 28538 leader_election.cc:304] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:14.773901 28786 raft_consensus.cc:2804] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:14.774737 28786 raft_consensus.cc:697] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:14.778752 28786 consensus_queue.cc:237] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.784282 28763 raft_consensus.cc:359] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.785084 28764 tablet_bootstrap.cc:492] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26: No bootstrap required, opened a new log
I20260709 13:03:14.782021 28722 raft_consensus.cc:2468] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 1.
I20260709 13:03:14.785099 28763 raft_consensus.cc:385] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.785717 28764 ts_tablet_manager.cc:1403] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.005s
I20260709 13:03:14.785717 28763 raft_consensus.cc:740] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.786895 28763 consensus_queue.cc:260] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.788641 28764 raft_consensus.cc:359] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.789029 28763 ts_tablet_manager.cc:1434] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.001s
I20260709 13:03:14.789366 28764 raft_consensus.cc:385] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.789705 28764 raft_consensus.cc:740] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.790230 28763 tablet_bootstrap.cc:492] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87: Bootstrap starting.
I20260709 13:03:14.790411 28764 consensus_queue.cc:260] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.792554 28764 ts_tablet_manager.cc:1434] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:03:14.793589 28764 tablet_bootstrap.cc:492] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26: Bootstrap starting.
I20260709 13:03:14.789961 28472 catalog_manager.cc:5697] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.799559 28764 tablet_bootstrap.cc:654] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.799974 28765 tablet_bootstrap.cc:492] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf: No bootstrap required, opened a new log
I20260709 13:03:14.800473 28765 ts_tablet_manager.cc:1403] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.006s
I20260709 13:03:14.804356 28765 raft_consensus.cc:359] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.805666 28765 raft_consensus.cc:385] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.806025 28765 raft_consensus.cc:740] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.806988 28765 consensus_queue.cc:260] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.809662 28765 ts_tablet_manager.cc:1434] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:03:14.811137 28763 tablet_bootstrap.cc:654] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.813143 28764 tablet_bootstrap.cc:492] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26: No bootstrap required, opened a new log
I20260709 13:03:14.813823 28764 ts_tablet_manager.cc:1403] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.008s
I20260709 13:03:14.818642 28764 raft_consensus.cc:359] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.819896 28764 raft_consensus.cc:385] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.820531 28764 raft_consensus.cc:740] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.821764 28764 consensus_queue.cc:260] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.825379 28764 ts_tablet_manager.cc:1434] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:03:14.828719 28763 tablet_bootstrap.cc:492] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87: No bootstrap required, opened a new log
I20260709 13:03:14.829310 28763 ts_tablet_manager.cc:1403] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.004s
I20260709 13:03:14.831894 28763 raft_consensus.cc:359] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.832468 28763 raft_consensus.cc:385] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.832695 28763 raft_consensus.cc:740] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.833257 28763 consensus_queue.cc:260] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.834947 28763 ts_tablet_manager.cc:1434] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.002s
I20260709 13:03:14.836325 28763 tablet_bootstrap.cc:492] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87: Bootstrap starting.
I20260709 13:03:14.841302 28763 tablet_bootstrap.cc:654] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.846382 28763 tablet_bootstrap.cc:492] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87: No bootstrap required, opened a new log
I20260709 13:03:14.846812 28763 ts_tablet_manager.cc:1403] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:03:14.849146 28763 raft_consensus.cc:359] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.849684 28763 raft_consensus.cc:385] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.849973 28763 raft_consensus.cc:740] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.850623 28763 consensus_queue.cc:260] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.852324 28763 ts_tablet_manager.cc:1434] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:03:14.868820 28786 raft_consensus.cc:493] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.869336 28786 raft_consensus.cc:515] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:14.871824 28786 leader_election.cc:290] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:14.872598 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" is_pre_election: true
I20260709 13:03:14.873097 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26" is_pre_election: true
I20260709 13:03:14.873256 28722 raft_consensus.cc:2468] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 0.
I20260709 13:03:14.873747 28649 raft_consensus.cc:2468] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 0.
I20260709 13:03:14.874464 28539 leader_election.cc:304] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:14.875391 28786 raft_consensus.cc:2804] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:14.875821 28786 raft_consensus.cc:493] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.876163 28786 raft_consensus.cc:3060] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.881470 28786 raft_consensus.cc:515] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:14.883459 28786 leader_election.cc:290] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 election: Requested vote from peers c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:14.884349 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:14.884783 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:14.885236 28722 raft_consensus.cc:3060] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.885509 28649 raft_consensus.cc:3060] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.887612 28786 consensus_queue.cc:1048] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:14.890431 28793 raft_consensus.cc:493] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.890976 28793 raft_consensus.cc:515] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.892467 28649 raft_consensus.cc:2468] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 1.
I20260709 13:03:14.891812 28722 raft_consensus.cc:2468] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 1.
I20260709 13:03:14.893356 28793 leader_election.cc:290] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:14.894774 28538 leader_election.cc:304] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:14.895188 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626ec34cdc60466bb40187641f4b40d2" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" is_pre_election: true
I20260709 13:03:14.895617 28786 raft_consensus.cc:2804] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:14.895978 28573 raft_consensus.cc:2468] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 0.
I20260709 13:03:14.896430 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626ec34cdc60466bb40187641f4b40d2" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26" is_pre_election: true
I20260709 13:03:14.897053 28649 raft_consensus.cc:2468] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 0.
I20260709 13:03:14.897238 28786 raft_consensus.cc:697] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:14.897416 28689 leader_election.cc:304] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:14.898356 28793 raft_consensus.cc:2804] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:14.899106 28793 raft_consensus.cc:493] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:14.898437 28786 consensus_queue.cc:237] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:14.899519 28793 raft_consensus.cc:3060] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.910120 28793 raft_consensus.cc:515] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.914412 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626ec34cdc60466bb40187641f4b40d2" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:14.915311 28573 raft_consensus.cc:3060] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.924856 28573 raft_consensus.cc:2468] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 1.
I20260709 13:03:14.926534 28689 leader_election.cc:304] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:14.928643 28772 raft_consensus.cc:2804] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:14.929770 28827 consensus_queue.cc:1048] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:14.930867 28772 raft_consensus.cc:697] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 LEADER]: Becoming Leader. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Running, Role: LEADER
I20260709 13:03:14.931780 28793 leader_election.cc:290] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 1 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:14.933454 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "626ec34cdc60466bb40187641f4b40d2" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:14.934172 28649 raft_consensus.cc:3060] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.934218 28772 consensus_queue.cc:237] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:14.943743 28649 raft_consensus.cc:2468] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 1.
I20260709 13:03:14.955894 28472 catalog_manager.cc:5697] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.956079 28468 catalog_manager.cc:5697] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf reported cstate change: term changed from 0 to 1, leader changed from <none> to c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131). New cstate: current_term: 1 leader_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:15.015112 28801 consensus_queue.cc:1048] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.026939 28770 consensus_queue.cc:1048] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.054067 28786 raft_consensus.cc:493] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.054557 28786 raft_consensus.cc:515] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.056815 28786 leader_election.cc:290] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:15.057811 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8dc76668dcf4cfd91850962987de661" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" is_pre_election: true
I20260709 13:03:15.057904 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8dc76668dcf4cfd91850962987de661" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26" is_pre_election: true
I20260709 13:03:15.058416 28722 raft_consensus.cc:2468] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 0.
I20260709 13:03:15.058481 28648 raft_consensus.cc:2468] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 0.
I20260709 13:03:15.059450 28539 leader_election.cc:304] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:15.060158 28786 raft_consensus.cc:2804] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:15.060515 28786 raft_consensus.cc:493] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.060784 28786 raft_consensus.cc:3060] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:15.065865 28786 raft_consensus.cc:515] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.067639 28786 leader_election.cc:290] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 election: Requested vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:15.068553 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8dc76668dcf4cfd91850962987de661" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:15.068766 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8dc76668dcf4cfd91850962987de661" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:15.069119 28648 raft_consensus.cc:3060] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:15.069370 28722 raft_consensus.cc:3060] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:15.075121 28648 raft_consensus.cc:2468] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 1.
I20260709 13:03:15.075139 28722 raft_consensus.cc:2468] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 1.
I20260709 13:03:15.076297 28538 leader_election.cc:304] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:15.076604 28770 consensus_queue.cc:1048] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.077173 28786 raft_consensus.cc:2804] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:15.077713 28786 raft_consensus.cc:697] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:15.078567 28786 consensus_queue.cc:237] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.089984 28468 catalog_manager.cc:5697] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:15.090648 28771 consensus_queue.cc:1048] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.112325 28786 consensus_queue.cc:1048] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.112408 28772 consensus_queue.cc:1048] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.119262 28648 raft_consensus.cc:3060] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:15.128592 28827 consensus_queue.cc:1048] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:15.136304 28828 consensus_queue.cc:1048] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:15.217489 28573 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:15.223816 28771 consensus_queue.cc:1048] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.227622 28827 consensus_queue.cc:1048] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.235548 28770 consensus_queue.cc:1048] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.244168 28827 consensus_queue.cc:1048] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:15.312991 28648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097"
dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
mode: GRACEFUL
new_leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:45844
I20260709 13:03:15.313599 28648 raft_consensus.cc:606] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Received request to transfer leadership to TS b0ee9adadeea4d7c8446f04a6f0a3c87
I20260709 13:03:15.320449 28649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0065b221305f415fbed169dba9a764c7"
dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
mode: GRACEFUL
new_leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:45844
I20260709 13:03:15.321043 28649 raft_consensus.cc:606] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Received request to transfer leadership to TS b0ee9adadeea4d7c8446f04a6f0a3c87
I20260709 13:03:15.321713 28793 consensus_queue.cc:1048] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.333590 28772 consensus_queue.cc:1048] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.350525 28723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d76f50c03914a21812cb7a0af509fb5"
dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
mode: GRACEFUL
new_leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:57580
I20260709 13:03:15.351171 28723 raft_consensus.cc:606] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 LEADER]: Received request to transfer leadership to TS b0ee9adadeea4d7c8446f04a6f0a3c87
I20260709 13:03:15.369211 28573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
mode: GRACEFUL
new_leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
 from {username='slave'} at 127.0.0.1:48954
I20260709 13:03:15.369814 28573 raft_consensus.cc:606] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Received request to transfer leadership to TS 47af5783a79641d4bcd2f254ce2b8c26
I20260709 13:03:15.377545 28573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
mode: GRACEFUL
new_leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
 from {username='slave'} at 127.0.0.1:48954
I20260709 13:03:15.378165 28573 raft_consensus.cc:606] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Received request to transfer leadership to TS 47af5783a79641d4bcd2f254ce2b8c26
I20260709 13:03:15.387987 28573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8dc76668dcf4cfd91850962987de661"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
mode: GRACEFUL
new_leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
 from {username='slave'} at 127.0.0.1:48954
I20260709 13:03:15.388576 28573 raft_consensus.cc:606] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Received request to transfer leadership to TS 47af5783a79641d4bcd2f254ce2b8c26
I20260709 13:03:15.457432 28827 consensus_queue.cc:1048] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.469296 28786 consensus_queue.cc:1048] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:15.592989 28771 raft_consensus.cc:993] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26: : Instructing follower b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:15.593482 28770 raft_consensus.cc:1081] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Signalling peer b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:15.594966 28573 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0065b221305f415fbed169dba9a764c7"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:48946
I20260709 13:03:15.595521 28573 raft_consensus.cc:493] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:15.595896 28573 raft_consensus.cc:3060] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.600857 28573 raft_consensus.cc:515] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.602723 28573 leader_election.cc:290] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 2 election: Requested vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:15.603780 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065b221305f415fbed169dba9a764c7" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:15.603839 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065b221305f415fbed169dba9a764c7" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:15.604434 28723 raft_consensus.cc:3060] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.604501 28649 raft_consensus.cc:3055] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:15.604894 28649 raft_consensus.cc:740] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:15.605624 28649 consensus_queue.cc:260] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.606539 28649 raft_consensus.cc:3060] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.610303 28723 raft_consensus.cc:2468] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 2.
I20260709 13:03:15.610283 28786 consensus_queue.cc:1048] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:15.611614 28539 leader_election.cc:304] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:15.612685 28786 raft_consensus.cc:2804] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:15.612862 28649 raft_consensus.cc:2468] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 2.
I20260709 13:03:15.613188 28786 raft_consensus.cc:697] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:15.614194 28786 consensus_queue.cc:237] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.618791 28827 raft_consensus.cc:993] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87: : Instructing follower 47af5783a79641d4bcd2f254ce2b8c26 to start an election
I20260709 13:03:15.619266 28827 raft_consensus.cc:1081] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Signalling peer 47af5783a79641d4bcd2f254ce2b8c26 to start an election
I20260709 13:03:15.621099 28649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8dc76668dcf4cfd91850962987de661"
dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
 from {username='slave'} at 127.0.0.1:45824
I20260709 13:03:15.622008 28649 raft_consensus.cc:493] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:15.622502 28649 raft_consensus.cc:3060] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.622017 28468 catalog_manager.cc:5697] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 1 to 2, leader changed from 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130) to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:15.626644 28827 consensus_queue.cc:1048] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:15.631635 28649 raft_consensus.cc:515] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.634742 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8dc76668dcf4cfd91850962987de661" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:15.635001 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8dc76668dcf4cfd91850962987de661" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:15.635344 28573 raft_consensus.cc:3055] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:15.635712 28573 raft_consensus.cc:740] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:15.636548 28573 consensus_queue.cc:260] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.637423 28649 leader_election.cc:290] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 2 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:15.637682 28573 raft_consensus.cc:3060] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.650013 28573 raft_consensus.cc:2468] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 2.
I20260709 13:03:15.651463 28615 leader_election.cc:304] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: c0e27b43534a4feeb11707f73bd7ccbf
I20260709 13:03:15.652331 28770 raft_consensus.cc:2804] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:15.652814 28770 raft_consensus.cc:697] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Becoming Leader. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:15.653638 28770 consensus_queue.cc:237] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.654558 28769 raft_consensus.cc:993] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87: : Instructing follower 47af5783a79641d4bcd2f254ce2b8c26 to start an election
I20260709 13:03:15.654983 28769 raft_consensus.cc:1081] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Signalling peer 47af5783a79641d4bcd2f254ce2b8c26 to start an election
I20260709 13:03:15.656697 28649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88"
dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
 from {username='slave'} at 127.0.0.1:45824
I20260709 13:03:15.657256 28649 raft_consensus.cc:493] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:15.657625 28649 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.665194 28649 raft_consensus.cc:515] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.668063 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:15.667719 28649 leader_election.cc:290] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 2 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:15.668287 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:15.668843 28573 raft_consensus.cc:3055] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:15.668980 28723 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.669225 28573 raft_consensus.cc:740] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:15.670073 28573 consensus_queue.cc:260] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.671280 28573 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.674178 28723 raft_consensus.cc:2468] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 2.
I20260709 13:03:15.675225 28614 leader_election.cc:304] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:15.675904 28801 raft_consensus.cc:2804] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:15.676402 28801 raft_consensus.cc:697] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Becoming Leader. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:15.677268 28573 raft_consensus.cc:2468] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 2.
I20260709 13:03:15.677151 28801 consensus_queue.cc:237] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.677386 28472 catalog_manager.cc:5697] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 reported cstate change: term changed from 1 to 2, leader changed from b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129) to 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:15.690910 28469 catalog_manager.cc:5697] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 reported cstate change: term changed from 1 to 2, leader changed from b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129) to 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:15.740793 28828 raft_consensus.cc:993] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf: : Instructing follower b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:15.741282 28772 raft_consensus.cc:1081] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 LEADER]: Signalling peer b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:15.742815 28573 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d76f50c03914a21812cb7a0af509fb5"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:48948
I20260709 13:03:15.743305 28573 raft_consensus.cc:493] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:15.743578 28573 raft_consensus.cc:3060] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.748317 28573 raft_consensus.cc:515] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.750011 28573 leader_election.cc:290] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 2 election: Requested vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:15.751004 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d76f50c03914a21812cb7a0af509fb5" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:15.751091 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d76f50c03914a21812cb7a0af509fb5" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:15.751647 28723 raft_consensus.cc:3055] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:15.751680 28649 raft_consensus.cc:3060] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.752038 28723 raft_consensus.cc:740] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Running, Role: LEADER
I20260709 13:03:15.752741 28723 consensus_queue.cc:260] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.753893 28723 raft_consensus.cc:3060] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.756884 28770 raft_consensus.cc:993] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26: : Instructing follower b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:15.757337 28801 raft_consensus.cc:1081] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Signalling peer b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:15.758034 28649 raft_consensus.cc:2468] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 2.
I20260709 13:03:15.759055 28573 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:48946
I20260709 13:03:15.759191 28538 leader_election.cc:304] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:15.759856 28573 raft_consensus.cc:493] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:15.760267 28573 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.760341 28827 raft_consensus.cc:2804] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:15.760341 28723 raft_consensus.cc:2468] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 2.
I20260709 13:03:15.761001 28827 raft_consensus.cc:697] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:15.761830 28827 consensus_queue.cc:237] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.765754 28573 raft_consensus.cc:515] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.767467 28573 leader_election.cc:290] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 2 election: Requested vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:15.768168 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:15.768507 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:15.768868 28649 raft_consensus.cc:3055] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:15.769012 28723 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.769361 28649 raft_consensus.cc:740] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:15.769080 28469 catalog_manager.cc:5697] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 1 to 2, leader changed from c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131) to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:15.770282 28649 consensus_queue.cc:260] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.771701 28649 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.776105 28723 raft_consensus.cc:2468] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 2.
I20260709 13:03:15.777503 28539 leader_election.cc:304] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:15.778302 28827 raft_consensus.cc:2804] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:15.778761 28827 raft_consensus.cc:697] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:15.779115 28649 raft_consensus.cc:2468] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 2.
I20260709 13:03:15.779373 28827 consensus_queue.cc:237] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.786010 28469 catalog_manager.cc:5697] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 1 to 2, leader changed from 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130) to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:15.835973 28786 raft_consensus.cc:993] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87: : Instructing follower 47af5783a79641d4bcd2f254ce2b8c26 to start an election
I20260709 13:03:15.836449 28827 raft_consensus.cc:1081] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Signalling peer 47af5783a79641d4bcd2f254ce2b8c26 to start an election
I20260709 13:03:15.838104 28649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e"
dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
 from {username='slave'} at 127.0.0.1:45824
I20260709 13:03:15.838743 28649 raft_consensus.cc:493] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:15.839152 28649 raft_consensus.cc:3060] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.844198 28649 raft_consensus.cc:515] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.846088 28649 leader_election.cc:290] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 2 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:15.846988 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:15.847355 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:15.847540 28573 raft_consensus.cc:3055] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:15.847971 28723 raft_consensus.cc:3060] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.847931 28573 raft_consensus.cc:740] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:15.848871 28573 consensus_queue.cc:260] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.849807 28573 raft_consensus.cc:3060] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:15.853065 28723 raft_consensus.cc:2468] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 2.
I20260709 13:03:15.854116 28614 leader_election.cc:304] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:15.854952 28801 raft_consensus.cc:2804] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:15.854944 28573 raft_consensus.cc:2468] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 2.
I20260709 13:03:15.855659 28801 raft_consensus.cc:697] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Becoming Leader. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:15.856485 28801 consensus_queue.cc:237] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:15.863018 28469 catalog_manager.cc:5697] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 reported cstate change: term changed from 1 to 2, leader changed from b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129) to 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:16.069173 28573 raft_consensus.cc:1275] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Refusing update from remote peer 47af5783a79641d4bcd2f254ce2b8c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.071256 28801 consensus_queue.cc:1048] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.083252 28723 raft_consensus.cc:1275] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Refusing update from remote peer 47af5783a79641d4bcd2f254ce2b8c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.085196 28801 consensus_queue.cc:1048] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.095777 28649 raft_consensus.cc:1275] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.097309 28786 consensus_queue.cc:1048] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.108194 28723 raft_consensus.cc:1275] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.110209 28786 consensus_queue.cc:1048] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.145462 28649 raft_consensus.cc:1275] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.147320 28786 consensus_queue.cc:1048] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.155580 28723 raft_consensus.cc:1275] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.157194 28827 consensus_queue.cc:1048] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.208175 28573 raft_consensus.cc:1275] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Refusing update from remote peer 47af5783a79641d4bcd2f254ce2b8c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.209805 28801 consensus_queue.cc:1048] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.210026 28723 raft_consensus.cc:3060] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:16.218037 28723 raft_consensus.cc:1275] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Refusing update from remote peer 47af5783a79641d4bcd2f254ce2b8c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.222687 28801 consensus_queue.cc:1048] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.330194 28723 raft_consensus.cc:1275] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.331621 28827 consensus_queue.cc:1048] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.341466 28649 raft_consensus.cc:1275] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.343722 28769 consensus_queue.cc:1048] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:16.471903 28723 raft_consensus.cc:1275] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Refusing update from remote peer 47af5783a79641d4bcd2f254ce2b8c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.473613 28801 consensus_queue.cc:1048] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.482489 28573 raft_consensus.cc:1275] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Refusing update from remote peer 47af5783a79641d4bcd2f254ce2b8c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:16.484177 28771 consensus_queue.cc:1048] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:18.434747 27934 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:03:18.436020 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:18.462886 28573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
mode: GRACEFUL
new_leader_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:03:18.463730 28573 raft_consensus.cc:606] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Received request to transfer leadership to TS c0e27b43534a4feeb11707f73bd7ccbf
I20260709 13:03:18.466912 28573 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
mode: GRACEFUL
new_leader_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:03:18.467509 28573 raft_consensus.cc:606] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Received request to transfer leadership to TS c0e27b43534a4feeb11707f73bd7ccbf
I20260709 13:03:18.469028 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:03:18.469666 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:18.768170 28868 raft_consensus.cc:993] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87: : Instructing follower c0e27b43534a4feeb11707f73bd7ccbf to start an election
I20260709 13:03:18.768760 28868 raft_consensus.cc:1081] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Signalling peer c0e27b43534a4feeb11707f73bd7ccbf to start an election
I20260709 13:03:18.771099 28723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421"
dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
 from {username='slave'} at 127.0.0.1:57576
I20260709 13:03:18.771770 28723 raft_consensus.cc:493] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:18.772235 28723 raft_consensus.cc:3060] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:18.778820 28723 raft_consensus.cc:515] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:18.780611 28723 leader_election.cc:290] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 2 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:18.781842 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:18.781991 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:18.782341 28649 raft_consensus.cc:3060] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:18.782639 28573 raft_consensus.cc:3055] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:18.783016 28573 raft_consensus.cc:740] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:18.783758 28573 consensus_queue.cc:260] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:18.784895 28573 raft_consensus.cc:3060] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:18.788403 28649 raft_consensus.cc:2468] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 2.
I20260709 13:03:18.789685 28687 leader_election.cc:304] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:18.790494 28772 raft_consensus.cc:2804] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:18.791371 28573 raft_consensus.cc:2468] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 2.
I20260709 13:03:18.791596 28772 raft_consensus.cc:697] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 LEADER]: Becoming Leader. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Running, Role: LEADER
I20260709 13:03:18.792744 28772 consensus_queue.cc:237] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:18.797966 28868 raft_consensus.cc:993] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87: : Instructing follower c0e27b43534a4feeb11707f73bd7ccbf to start an election
I20260709 13:03:18.798439 28868 raft_consensus.cc:1081] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Signalling peer c0e27b43534a4feeb11707f73bd7ccbf to start an election
I20260709 13:03:18.800235 28723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097"
dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
 from {username='slave'} at 127.0.0.1:57576
I20260709 13:03:18.800743 28723 raft_consensus.cc:493] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:18.801110 28723 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:18.805009 28472 catalog_manager.cc:5697] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf reported cstate change: term changed from 1 to 2, leader changed from b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129) to c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131). New cstate: current_term: 2 leader_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:18.806806 28723 raft_consensus.cc:515] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:18.808629 28723 leader_election.cc:290] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 3 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:18.809661 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:18.810176 28573 raft_consensus.cc:3055] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:18.810232 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:18.810640 28573 raft_consensus.cc:740] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:18.810880 28649 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:18.811645 28573 consensus_queue.cc:260] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:18.812888 28573 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:18.817905 28649 raft_consensus.cc:2468] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 3.
I20260709 13:03:18.819417 28687 leader_election.cc:304] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:18.820326 28772 raft_consensus.cc:2804] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:18.820003 28573 raft_consensus.cc:2468] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 3.
I20260709 13:03:18.820889 28772 raft_consensus.cc:697] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 LEADER]: Becoming Leader. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Running, Role: LEADER
I20260709 13:03:18.821931 28772 consensus_queue.cc:237] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:18.830235 28472 catalog_manager.cc:5697] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf reported cstate change: term changed from 2 to 3, leader changed from b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129) to c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131). New cstate: current_term: 3 leader_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:19.317001 28574 raft_consensus.cc:1275] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Refusing update from remote peer c0e27b43534a4feeb11707f73bd7ccbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:19.318609 28772 consensus_queue.cc:1048] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:19.332762 28649 raft_consensus.cc:1275] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Refusing update from remote peer c0e27b43534a4feeb11707f73bd7ccbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:19.334345 28879 consensus_queue.cc:1048] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:19.368974 28574 raft_consensus.cc:1275] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Refusing update from remote peer c0e27b43534a4feeb11707f73bd7ccbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:19.370677 28879 consensus_queue.cc:1048] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:19.380744 28649 raft_consensus.cc:1275] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Refusing update from remote peer c0e27b43534a4feeb11707f73bd7ccbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:19.382761 28772 consensus_queue.cc:1048] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:19.470744 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:19.481880 28723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421"
dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
mode: GRACEFUL
new_leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:57580
I20260709 13:03:19.482435 28723 raft_consensus.cc:606] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 LEADER]: Received request to transfer leadership to TS b0ee9adadeea4d7c8446f04a6f0a3c87
I20260709 13:03:19.488888 28723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097"
dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
mode: GRACEFUL
new_leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:57580
I20260709 13:03:19.489395 28723 raft_consensus.cc:606] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 LEADER]: Received request to transfer leadership to TS b0ee9adadeea4d7c8446f04a6f0a3c87
I20260709 13:03:19.502671 28649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88"
dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
mode: GRACEFUL
new_leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:45844
I20260709 13:03:19.503209 28649 raft_consensus.cc:606] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Received request to transfer leadership to TS b0ee9adadeea4d7c8446f04a6f0a3c87
I20260709 13:03:19.509331 28649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e"
dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
mode: GRACEFUL
new_leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:45844
I20260709 13:03:19.509814 28649 raft_consensus.cc:606] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Received request to transfer leadership to TS b0ee9adadeea4d7c8446f04a6f0a3c87
I20260709 13:03:19.514885 28649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f"
dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
mode: GRACEFUL
new_leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:45844
I20260709 13:03:19.515448 28649 raft_consensus.cc:606] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Received request to transfer leadership to TS b0ee9adadeea4d7c8446f04a6f0a3c87
I20260709 13:03:19.836413 28888 raft_consensus.cc:993] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26: : Instructing follower b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:19.837222 28801 raft_consensus.cc:1081] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Signalling peer b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:19.839665 28574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:48946
I20260709 13:03:19.840927 28574 raft_consensus.cc:493] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:19.841631 28574 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.848315 28574 raft_consensus.cc:515] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:19.850363 28574 leader_election.cc:290] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 3 election: Requested vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:19.852568 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:19.852787 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:19.853401 28649 raft_consensus.cc:3055] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:19.853514 28722 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.853704 28649 raft_consensus.cc:740] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:19.854779 28649 consensus_queue.cc:260] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:19.856110 28649 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.860416 28722 raft_consensus.cc:2468] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 3.
I20260709 13:03:19.861796 28539 leader_election.cc:304] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:19.862681 28649 raft_consensus.cc:2468] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 3.
I20260709 13:03:19.862803 28867 raft_consensus.cc:2804] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:19.864140 28867 raft_consensus.cc:697] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:19.865332 28867 consensus_queue.cc:237] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:19.875129 28469 catalog_manager.cc:5697] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 2 to 3, leader changed from 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130) to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 3 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:19.887792 28772 raft_consensus.cc:993] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf: : Instructing follower b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:19.888845 28895 raft_consensus.cc:1081] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 LEADER]: Signalling peer b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:19.890931 28574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:48948
I20260709 13:03:19.891556 28574 raft_consensus.cc:493] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:19.892010 28574 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:19.897379 28574 raft_consensus.cc:515] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:19.899160 28574 leader_election.cc:290] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 4 election: Requested vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:19.900066 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:19.900354 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecfce63e8b94e33825cfa5d68a3c097" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:19.900724 28649 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:19.900906 28722 raft_consensus.cc:3055] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:19.901281 28722 raft_consensus.cc:740] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Running, Role: LEADER
I20260709 13:03:19.902235 28722 consensus_queue.cc:260] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:19.903748 28722 raft_consensus.cc:3060] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:19.907696 28649 raft_consensus.cc:2468] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 4.
I20260709 13:03:19.909163 28538 leader_election.cc:304] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:19.910022 28867 raft_consensus.cc:2804] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:19.910455 28867 raft_consensus.cc:697] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 4 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:19.910733 28722 raft_consensus.cc:2468] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 4.
I20260709 13:03:19.911473 28867 consensus_queue.cc:237] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:19.919143 28469 catalog_manager.cc:5697] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 3 to 4, leader changed from c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131) to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 4 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:19.977355 28772 raft_consensus.cc:993] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf: : Instructing follower b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:19.977941 28895 raft_consensus.cc:1081] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 LEADER]: Signalling peer b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:19.979449 28574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:48948
I20260709 13:03:19.979880 28574 raft_consensus.cc:493] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:19.980219 28574 raft_consensus.cc:3060] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.991603 28574 raft_consensus.cc:515] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:19.993124 28888 raft_consensus.cc:993] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26: : Instructing follower b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:19.993723 28801 raft_consensus.cc:1081] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Signalling peer b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:19.993726 28574 leader_election.cc:290] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 3 election: Requested vote from peers c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:19.994751 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:19.995137 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c7407fbedd4acc99084e230bcf1421" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:19.995277 28649 raft_consensus.cc:3060] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.995384 28574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:48946
I20260709 13:03:19.995710 28722 raft_consensus.cc:3055] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:19.995947 28574 raft_consensus.cc:493] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:19.996166 28722 raft_consensus.cc:740] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Running, Role: LEADER
I20260709 13:03:19.996417 28574 raft_consensus.cc:3060] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:19.997087 28722 consensus_queue.cc:260] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:19.998018 28722 raft_consensus.cc:3060] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.002460 28649 raft_consensus.cc:2468] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 3.
I20260709 13:03:20.003790 28538 leader_election.cc:304] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:20.003531 28574 raft_consensus.cc:515] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:20.004387 28722 raft_consensus.cc:2468] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 3.
I20260709 13:03:20.005084 28890 raft_consensus.cc:2804] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:20.006137 28574 leader_election.cc:290] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 3 election: Requested vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:20.006081 28890 raft_consensus.cc:697] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:20.008447 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:20.009971 28649 raft_consensus.cc:3055] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:20.009653 28890 consensus_queue.cc:237] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } }
I20260709 13:03:20.010604 28649 raft_consensus.cc:740] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:20.011878 28649 consensus_queue.cc:260] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:20.013151 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7d879ac8ed474ba5021fbf3fb2d35e" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:20.013798 28649 raft_consensus.cc:3060] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.014060 28722 raft_consensus.cc:3060] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:20.022981 28469 catalog_manager.cc:5697] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 2 to 3, leader changed from c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131) to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 3 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:20.026072 28649 raft_consensus.cc:2468] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 3.
I20260709 13:03:20.026085 28722 raft_consensus.cc:2468] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 3.
I20260709 13:03:20.027395 28538 leader_election.cc:304] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, b0ee9adadeea4d7c8446f04a6f0a3c87; no voters: 
I20260709 13:03:20.028402 28898 raft_consensus.cc:2804] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:20.028905 28898 raft_consensus.cc:697] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:20.030365 28898 consensus_queue.cc:237] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:20.039462 28469 catalog_manager.cc:5697] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 2 to 3, leader changed from 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130) to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 3 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:20.099494 28888 raft_consensus.cc:993] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26: : Instructing follower b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:20.100042 28801 raft_consensus.cc:1081] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Signalling peer b0ee9adadeea4d7c8446f04a6f0a3c87 to start an election
I20260709 13:03:20.101574 28574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
 from {username='slave'} at 127.0.0.1:48946
I20260709 13:03:20.102051 28574 raft_consensus.cc:493] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:20.102316 28574 raft_consensus.cc:3060] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:20.107138 28574 raft_consensus.cc:515] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:20.109233 28574 leader_election.cc:290] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 2 election: Requested vote from peers 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:20.110143 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:20.110327 28722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f" candidate_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:20.110694 28649 raft_consensus.cc:3055] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:20.110915 28722 raft_consensus.cc:3060] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:20.111037 28649 raft_consensus.cc:740] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:20.111848 28649 consensus_queue.cc:260] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:20.112810 28649 raft_consensus.cc:3060] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:20.115477 28722 raft_consensus.cc:2468] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 2.
I20260709 13:03:20.116595 28539 leader_election.cc:304] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0ee9adadeea4d7c8446f04a6f0a3c87, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:20.117538 28898 raft_consensus.cc:2804] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:20.117666 28649 raft_consensus.cc:2468] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ee9adadeea4d7c8446f04a6f0a3c87 in term 2.
I20260709 13:03:20.118010 28898 raft_consensus.cc:697] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Becoming Leader. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:20.119046 28898 consensus_queue.cc:237] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:20.126346 28469 catalog_manager.cc:5697] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 reported cstate change: term changed from 1 to 2, leader changed from 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130) to b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:20.285831 28649 raft_consensus.cc:1275] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.287109 28898 consensus_queue.cc:1048] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.299052 28722 raft_consensus.cc:1275] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.300949 28867 consensus_queue.cc:1048] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.475283 28649 raft_consensus.cc:1275] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.476676 28867 consensus_queue.cc:1048] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.486291 28722 raft_consensus.cc:1275] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.488111 28867 consensus_queue.cc:1048] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.534345 28722 raft_consensus.cc:1275] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 4 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:20.535650 28867 consensus_queue.cc:1048] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:20.538683 28649 raft_consensus.cc:1275] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 4 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:20.540485 28867 consensus_queue.cc:1048] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:20.551258 28648 raft_consensus.cc:1275] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.553542 28898 consensus_queue.cc:1048] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.564980 28722 raft_consensus.cc:1275] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:20.566092 28867 consensus_queue.cc:1048] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.588207 28649 raft_consensus.cc:1275] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:20.589483 28867 consensus_queue.cc:1048] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:20.598737 28722 raft_consensus.cc:1275] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Refusing update from remote peer b0ee9adadeea4d7c8446f04a6f0a3c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:20.601080 28898 consensus_queue.cc:1048] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:22.524227 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:22.532668 28574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0065b221305f415fbed169dba9a764c7"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
mode: GRACEFUL
new_leader_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:03:22.533449 28574 raft_consensus.cc:606] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Received request to transfer leadership to TS c0e27b43534a4feeb11707f73bd7ccbf
I20260709 13:03:22.536267 28574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
mode: GRACEFUL
new_leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:03:22.536808 28574 raft_consensus.cc:606] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Received request to transfer leadership to TS 47af5783a79641d4bcd2f254ce2b8c26
I20260709 13:03:22.539331 28574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44db1cf806fa4650a3552062cf11ca43"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
mode: GRACEFUL
new_leader_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:03:22.539834 28574 raft_consensus.cc:606] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Received request to transfer leadership to TS c0e27b43534a4feeb11707f73bd7ccbf
I20260709 13:03:22.542320 28574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88"
dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
mode: GRACEFUL
new_leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:03:22.542853 28574 raft_consensus.cc:606] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 LEADER]: Received request to transfer leadership to TS 47af5783a79641d4bcd2f254ce2b8c26
I20260709 13:03:22.543889 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:03:22.544474 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:22.716282 28918 raft_consensus.cc:993] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87: : Instructing follower 47af5783a79641d4bcd2f254ce2b8c26 to start an election
I20260709 13:03:22.716861 28917 raft_consensus.cc:1081] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Signalling peer 47af5783a79641d4bcd2f254ce2b8c26 to start an election
I20260709 13:03:22.718532 28648 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f"
dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
 from {username='slave'} at 127.0.0.1:45824
I20260709 13:03:22.719043 28648 raft_consensus.cc:493] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:22.719374 28648 raft_consensus.cc:3060] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:22.724443 28648 raft_consensus.cc:515] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.726432 28648 leader_election.cc:290] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 3 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:22.727358 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:22.728056 28574 raft_consensus.cc:3055] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:22.728288 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23d3c7d3a61e4b66b8e50ed6ce0c8f0f" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:22.728408 28574 raft_consensus.cc:740] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:22.728823 28723 raft_consensus.cc:3060] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:22.729247 28574 consensus_queue.cc:260] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.730275 28574 raft_consensus.cc:3060] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:22.733744 28723 raft_consensus.cc:2468] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 3.
I20260709 13:03:22.734932 28614 leader_election.cc:304] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:22.735749 28916 raft_consensus.cc:2804] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:22.735843 28574 raft_consensus.cc:2468] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 3.
I20260709 13:03:22.736891 28916 raft_consensus.cc:697] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 LEADER]: Becoming Leader. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:22.737815 28916 consensus_queue.cc:237] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.744784 28469 catalog_manager.cc:5697] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 reported cstate change: term changed from 2 to 3, leader changed from b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129) to 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:22.794945 28918 raft_consensus.cc:993] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87: : Instructing follower 47af5783a79641d4bcd2f254ce2b8c26 to start an election
I20260709 13:03:22.795575 28917 raft_consensus.cc:1081] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 LEADER]: Signalling peer 47af5783a79641d4bcd2f254ce2b8c26 to start an election
I20260709 13:03:22.797204 28648 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88"
dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
 from {username='slave'} at 127.0.0.1:45824
I20260709 13:03:22.797659 28648 raft_consensus.cc:493] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:22.797917 28648 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:22.802711 28648 raft_consensus.cc:515] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.804273 28648 leader_election.cc:290] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 4 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131:45279)
I20260709 13:03:22.805099 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:22.805168 28723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62cd4db841a54f90bc187b657a2adb88" candidate_uuid: "47af5783a79641d4bcd2f254ce2b8c26" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
I20260709 13:03:22.805657 28574 raft_consensus.cc:3055] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:22.805706 28723 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:22.806102 28574 raft_consensus.cc:740] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:22.806860 28574 consensus_queue.cc:260] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.807744 28574 raft_consensus.cc:3060] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:22.810149 28723 raft_consensus.cc:2468] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 4.
I20260709 13:03:22.811094 28614 leader_election.cc:304] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:22.811721 28916 raft_consensus.cc:2804] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:22.812245 28916 raft_consensus.cc:697] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 4 LEADER]: Becoming Leader. State: Replica: 47af5783a79641d4bcd2f254ce2b8c26, State: Running, Role: LEADER
I20260709 13:03:22.812630 28574 raft_consensus.cc:2468] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47af5783a79641d4bcd2f254ce2b8c26 in term 4.
I20260709 13:03:22.813097 28916 consensus_queue.cc:237] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.820751 28469 catalog_manager.cc:5697] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 reported cstate change: term changed from 3 to 4, leader changed from b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129) to 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130). New cstate: current_term: 4 leader_uuid: "47af5783a79641d4bcd2f254ce2b8c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:22.924057 28917 raft_consensus.cc:993] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87: : Instructing follower c0e27b43534a4feeb11707f73bd7ccbf to start an election
I20260709 13:03:22.924829 28919 raft_consensus.cc:993] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87: : Instructing follower c0e27b43534a4feeb11707f73bd7ccbf to start an election
I20260709 13:03:22.924853 28918 raft_consensus.cc:1081] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Signalling peer c0e27b43534a4feeb11707f73bd7ccbf to start an election
I20260709 13:03:22.925542 28917 raft_consensus.cc:1081] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Signalling peer c0e27b43534a4feeb11707f73bd7ccbf to start an election
I20260709 13:03:22.926698 28722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44db1cf806fa4650a3552062cf11ca43"
dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
 from {username='slave'} at 127.0.0.1:57576
I20260709 13:03:22.927112 28723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0065b221305f415fbed169dba9a764c7"
dest_uuid: "c0e27b43534a4feeb11707f73bd7ccbf"
 from {username='slave'} at 127.0.0.1:57576
I20260709 13:03:22.927215 28722 raft_consensus.cc:493] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:22.927762 28723 raft_consensus.cc:493] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:22.928166 28722 raft_consensus.cc:3060] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:22.928231 28723 raft_consensus.cc:3060] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:22.932608 28722 raft_consensus.cc:515] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.932713 28723 raft_consensus.cc:515] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.934317 28723 leader_election.cc:290] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 3 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:22.934476 28722 leader_election.cc:290] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 2 election: Requested vote from peers b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231), 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293)
I20260709 13:03:22.935721 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065b221305f415fbed169dba9a764c7" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:22.935870 28648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065b221305f415fbed169dba9a764c7" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:22.936275 28649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44db1cf806fa4650a3552062cf11ca43" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47af5783a79641d4bcd2f254ce2b8c26"
I20260709 13:03:22.936380 28574 raft_consensus.cc:3055] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:22.937003 28574 raft_consensus.cc:740] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:22.936779 28648 raft_consensus.cc:3060] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:22.936779 28573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44db1cf806fa4650a3552062cf11ca43" candidate_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87"
I20260709 13:03:22.936892 28649 raft_consensus.cc:3060] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:22.937886 28573 raft_consensus.cc:3055] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:22.938422 28573 raft_consensus.cc:740] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b0ee9adadeea4d7c8446f04a6f0a3c87, State: Running, Role: LEADER
I20260709 13:03:22.938320 28574 consensus_queue.cc:260] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.939386 28573 consensus_queue.cc:260] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.939860 28574 raft_consensus.cc:3060] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:22.940598 28573 raft_consensus.cc:3060] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:22.945498 28648 raft_consensus.cc:2468] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 3.
I20260709 13:03:22.945858 28649 raft_consensus.cc:2468] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 2.
I20260709 13:03:22.947319 28687 leader_election.cc:304] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:22.948005 28573 raft_consensus.cc:2468] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 2.
I20260709 13:03:22.948441 28772 raft_consensus.cc:2804] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:22.948734 28574 raft_consensus.cc:2468] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e27b43534a4feeb11707f73bd7ccbf in term 3.
I20260709 13:03:22.948669 28687 leader_election.cc:304] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47af5783a79641d4bcd2f254ce2b8c26, c0e27b43534a4feeb11707f73bd7ccbf; no voters: 
I20260709 13:03:22.950312 28772 raft_consensus.cc:697] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 LEADER]: Becoming Leader. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Running, Role: LEADER
I20260709 13:03:22.950867 28927 raft_consensus.cc:2804] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:22.951403 28927 raft_consensus.cc:697] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 LEADER]: Becoming Leader. State: Replica: c0e27b43534a4feeb11707f73bd7ccbf, State: Running, Role: LEADER
I20260709 13:03:22.951274 28772 consensus_queue.cc:237] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.952365 28927 consensus_queue.cc:237] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } }
I20260709 13:03:22.960237 28469 catalog_manager.cc:5697] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf reported cstate change: term changed from 2 to 3, leader changed from b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129) to c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131). New cstate: current_term: 3 leader_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:22.973374 28469 catalog_manager.cc:5697] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf reported cstate change: term changed from 1 to 2, leader changed from b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129) to c0e27b43534a4feeb11707f73bd7ccbf (127.27.71.131). New cstate: current_term: 2 leader_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.175251 28722 raft_consensus.cc:1275] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Refusing update from remote peer 47af5783a79641d4bcd2f254ce2b8c26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:23.176584 28916 consensus_queue.cc:1048] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:23.187011 28574 raft_consensus.cc:1275] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Refusing update from remote peer 47af5783a79641d4bcd2f254ce2b8c26: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:23.189095 28916 consensus_queue.cc:1048] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:23.278360 28722 raft_consensus.cc:1275] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 4 FOLLOWER]: Refusing update from remote peer 47af5783a79641d4bcd2f254ce2b8c26: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:23.279495 28916 consensus_queue.cc:1048] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e27b43534a4feeb11707f73bd7ccbf" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:23.289840 28574 raft_consensus.cc:1275] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 4 FOLLOWER]: Refusing update from remote peer 47af5783a79641d4bcd2f254ce2b8c26: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:23.291255 28937 consensus_queue.cc:1048] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:23.386636 28649 raft_consensus.cc:1275] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Refusing update from remote peer c0e27b43534a4feeb11707f73bd7ccbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:23.387997 28772 consensus_queue.cc:1048] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:23.397356 28574 raft_consensus.cc:1275] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Refusing update from remote peer c0e27b43534a4feeb11707f73bd7ccbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:23.399345 28772 consensus_queue.cc:1048] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:23.425734 28574 raft_consensus.cc:1275] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Refusing update from remote peer c0e27b43534a4feeb11707f73bd7ccbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:23.427222 28772 consensus_queue.cc:1048] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ee9adadeea4d7c8446f04a6f0a3c87" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 43231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:23.435642 28649 raft_consensus.cc:1275] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Refusing update from remote peer c0e27b43534a4feeb11707f73bd7ccbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:23.437844 28927 consensus_queue.cc:1048] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "47af5783a79641d4bcd2f254ce2b8c26" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:23.545444 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:23.551467 27934 tablet_server.cc:179] TabletServer@127.27.71.129:0 shutting down...
W20260709 13:03:23.580467 28615 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260709 13:03:23.584282 28615 consensus_peers.cc:597] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 -> Peer b0ee9adadeea4d7c8446f04a6f0a3c87 (127.27.71.129:43231): Couldn't send request to peer b0ee9adadeea4d7c8446f04a6f0a3c87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:23.585819 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:23.586524 27934 tablet_replica.cc:333] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87: stopping tablet replica
I20260709 13:03:23.587113 27934 raft_consensus.cc:2243] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.587569 27934 raft_consensus.cc:2272] T 626ec34cdc60466bb40187641f4b40d2 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.589552 27934 tablet_replica.cc:333] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87: stopping tablet replica
I20260709 13:03:23.590144 27934 raft_consensus.cc:2243] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.590829 27934 raft_consensus.cc:2272] T c0c7407fbedd4acc99084e230bcf1421 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.592553 27934 tablet_replica.cc:333] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87: stopping tablet replica
I20260709 13:03:23.593066 27934 raft_consensus.cc:2243] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.593734 27934 raft_consensus.cc:2272] T ab7d879ac8ed474ba5021fbf3fb2d35e P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.595389 27934 tablet_replica.cc:333] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87: stopping tablet replica
I20260709 13:03:23.595907 27934 raft_consensus.cc:2243] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.596374 27934 raft_consensus.cc:2272] T 0065b221305f415fbed169dba9a764c7 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.598506 27934 tablet_replica.cc:333] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87: stopping tablet replica
I20260709 13:03:23.599303 27934 raft_consensus.cc:2243] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.599772 27934 raft_consensus.cc:2272] T 62cd4db841a54f90bc187b657a2adb88 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.601747 27934 tablet_replica.cc:333] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87: stopping tablet replica
I20260709 13:03:23.602314 27934 raft_consensus.cc:2243] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.602749 27934 raft_consensus.cc:2272] T 44db1cf806fa4650a3552062cf11ca43 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.604997 27934 tablet_replica.cc:333] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87: stopping tablet replica
I20260709 13:03:23.605499 27934 raft_consensus.cc:2243] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.605885 27934 raft_consensus.cc:2272] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.607985 27934 tablet_replica.cc:333] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87: stopping tablet replica
I20260709 13:03:23.608485 27934 raft_consensus.cc:2243] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.609162 27934 raft_consensus.cc:2272] T 0ecfce63e8b94e33825cfa5d68a3c097 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.610977 27934 tablet_replica.cc:333] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87: stopping tablet replica
I20260709 13:03:23.611675 27934 raft_consensus.cc:2243] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.612241 27934 raft_consensus.cc:2272] T d8dc76668dcf4cfd91850962987de661 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.614696 27934 tablet_replica.cc:333] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87: stopping tablet replica
I20260709 13:03:23.615398 27934 raft_consensus.cc:2243] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.616415 27934 raft_consensus.cc:2272] T 7d76f50c03914a21812cb7a0af509fb5 P b0ee9adadeea4d7c8446f04a6f0a3c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.646376 27934 tablet_server.cc:196] TabletServer@127.27.71.129:0 shutdown complete.
I20260709 13:03:23.667222 27934 tablet_server.cc:179] TabletServer@127.27.71.130:0 shutting down...
I20260709 13:03:23.690464 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:23.691128 27934 tablet_replica.cc:333] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26: stopping tablet replica
I20260709 13:03:23.691689 27934 raft_consensus.cc:2243] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.692178 27934 raft_consensus.cc:2272] T 626ec34cdc60466bb40187641f4b40d2 P 47af5783a79641d4bcd2f254ce2b8c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.693938 27934 tablet_replica.cc:333] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26: stopping tablet replica
I20260709 13:03:23.694476 27934 raft_consensus.cc:2243] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.695236 27934 raft_consensus.cc:2272] T 62cd4db841a54f90bc187b657a2adb88 P 47af5783a79641d4bcd2f254ce2b8c26 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.697423 27934 tablet_replica.cc:333] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26: stopping tablet replica
I20260709 13:03:23.697947 27934 raft_consensus.cc:2243] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.698643 27934 raft_consensus.cc:2272] T d8dc76668dcf4cfd91850962987de661 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.700340 27934 tablet_replica.cc:333] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26: stopping tablet replica
I20260709 13:03:23.700832 27934 raft_consensus.cc:2243] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.701231 27934 raft_consensus.cc:2272] T 7d76f50c03914a21812cb7a0af509fb5 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.702857 27934 tablet_replica.cc:333] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26: stopping tablet replica
I20260709 13:03:23.703337 27934 raft_consensus.cc:2243] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.703738 27934 raft_consensus.cc:2272] T 0ecfce63e8b94e33825cfa5d68a3c097 P 47af5783a79641d4bcd2f254ce2b8c26 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.705403 27934 tablet_replica.cc:333] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26: stopping tablet replica
I20260709 13:03:23.705891 27934 raft_consensus.cc:2243] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.706600 27934 raft_consensus.cc:2272] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.709156 27934 tablet_replica.cc:333] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26: stopping tablet replica
I20260709 13:03:23.709702 27934 raft_consensus.cc:2243] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.710050 27934 raft_consensus.cc:2272] T 44db1cf806fa4650a3552062cf11ca43 P 47af5783a79641d4bcd2f254ce2b8c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.712282 27934 tablet_replica.cc:333] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26: stopping tablet replica
I20260709 13:03:23.712718 27934 raft_consensus.cc:2243] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.713049 27934 raft_consensus.cc:2272] T ab7d879ac8ed474ba5021fbf3fb2d35e P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.714493 27934 tablet_replica.cc:333] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26: stopping tablet replica
I20260709 13:03:23.714900 27934 raft_consensus.cc:2243] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.715269 27934 raft_consensus.cc:2272] T 0065b221305f415fbed169dba9a764c7 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.717460 27934 tablet_replica.cc:333] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26: stopping tablet replica
I20260709 13:03:23.717931 27934 raft_consensus.cc:2243] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.718328 27934 raft_consensus.cc:2272] T c0c7407fbedd4acc99084e230bcf1421 P 47af5783a79641d4bcd2f254ce2b8c26 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.739244 27934 tablet_server.cc:196] TabletServer@127.27.71.130:0 shutdown complete.
W20260709 13:03:23.753229 28687 consensus_peers.cc:597] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf -> Peer 47af5783a79641d4bcd2f254ce2b8c26 (127.27.71.130:35293): Couldn't send request to peer 47af5783a79641d4bcd2f254ce2b8c26. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:23.760542 27934 tablet_server.cc:179] TabletServer@127.27.71.131:0 shutting down...
I20260709 13:03:23.785715 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:23.786370 27934 tablet_replica.cc:333] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf: stopping tablet replica
I20260709 13:03:23.786942 27934 raft_consensus.cc:2243] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.787734 27934 raft_consensus.cc:2272] T 626ec34cdc60466bb40187641f4b40d2 P c0e27b43534a4feeb11707f73bd7ccbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.789562 27934 tablet_replica.cc:333] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf: stopping tablet replica
I20260709 13:03:23.790040 27934 raft_consensus.cc:2243] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.790388 27934 raft_consensus.cc:2272] T ab7d879ac8ed474ba5021fbf3fb2d35e P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.792276 27934 tablet_replica.cc:333] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf: stopping tablet replica
I20260709 13:03:23.792739 27934 raft_consensus.cc:2243] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.793077 27934 raft_consensus.cc:2272] T c0c7407fbedd4acc99084e230bcf1421 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.794595 27934 tablet_replica.cc:333] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf: stopping tablet replica
I20260709 13:03:23.795042 27934 raft_consensus.cc:2243] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.795765 27934 raft_consensus.cc:2272] T 0065b221305f415fbed169dba9a764c7 P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.798357 27934 tablet_replica.cc:333] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf: stopping tablet replica
I20260709 13:03:23.798954 27934 raft_consensus.cc:2243] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.799293 27934 raft_consensus.cc:2272] T 62cd4db841a54f90bc187b657a2adb88 P c0e27b43534a4feeb11707f73bd7ccbf [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.801352 27934 tablet_replica.cc:333] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf: stopping tablet replica
I20260709 13:03:23.801803 27934 raft_consensus.cc:2243] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.802182 27934 raft_consensus.cc:2272] T d8dc76668dcf4cfd91850962987de661 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.803634 27934 tablet_replica.cc:333] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf: stopping tablet replica
I20260709 13:03:23.804183 27934 raft_consensus.cc:2243] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.804576 27934 raft_consensus.cc:2272] T 0ecfce63e8b94e33825cfa5d68a3c097 P c0e27b43534a4feeb11707f73bd7ccbf [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.806181 27934 tablet_replica.cc:333] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf: stopping tablet replica
I20260709 13:03:23.806679 27934 raft_consensus.cc:2243] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.807106 27934 raft_consensus.cc:2272] T 7d76f50c03914a21812cb7a0af509fb5 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.808904 27934 tablet_replica.cc:333] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf: stopping tablet replica
I20260709 13:03:23.809382 27934 raft_consensus.cc:2243] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:23.809789 27934 raft_consensus.cc:2272] T 23d3c7d3a61e4b66b8e50ed6ce0c8f0f P c0e27b43534a4feeb11707f73bd7ccbf [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.811678 27934 tablet_replica.cc:333] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf: stopping tablet replica
I20260709 13:03:23.812177 27934 raft_consensus.cc:2243] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.813000 27934 raft_consensus.cc:2272] T 44db1cf806fa4650a3552062cf11ca43 P c0e27b43534a4feeb11707f73bd7ccbf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.841457 27934 tablet_server.cc:196] TabletServer@127.27.71.131:0 shutdown complete.
I20260709 13:03:23.861333 27934 master.cc:562] Master@127.27.71.190:36419 shutting down...
I20260709 13:03:23.883483 27934 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:23.884119 27934 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:23.884486 27934 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4edbbce1d5244c858bfefc32c43b1dc9: stopping tablet replica
I20260709 13:03:23.905503 27934 master.cc:584] Master@127.27.71.190:36419 shutdown complete.
47af5783a79641d4bcd2f254ce2b8c26  4
b0ee9adadeea4d7c8446f04a6f0a3c87  5
c0e27b43534a4feeb11707f73bd7ccbf  1
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10915 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:03:23.948369 27934 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.71.190:37537
I20260709 13:03:23.949568 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:23.955523 28947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:23.955555 28946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:23.956418 28949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:23.956849 27934 server_base.cc:1061] running on GCE node
I20260709 13:03:23.957672 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:23.957844 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:23.957955 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602203957944 us; error 0 us; skew 500 ppm
I20260709 13:03:23.958396 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:23.960903 27934 webserver.cc:533] Webserver started at http://127.27.71.190:37203/ using document root <none> and password file <none>
I20260709 13:03:23.961351 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:23.961511 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:23.961726 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:23.962809 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/master-0-root/instance:
uuid: "9c100c1aaf1b493089df815bded98302"
format_stamp: "Formatted at 2026-07-09 13:03:23 on dist-test-slave-mlt7"
I20260709 13:03:23.967651 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:03:23.971118 28954 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:23.972082 27934 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:03:23.972375 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/master-0-root
uuid: "9c100c1aaf1b493089df815bded98302"
format_stamp: "Formatted at 2026-07-09 13:03:23 on dist-test-slave-mlt7"
I20260709 13:03:23.972770 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:23.982585 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:23.984002 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:24.024294 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.190:37537
I20260709 13:03:24.024413 29005 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.190:37537 every 8 connection(s)
I20260709 13:03:24.028729 29006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.039047 29006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Bootstrap starting.
I20260709 13:03:24.043900 29006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:24.048326 29006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: No bootstrap required, opened a new log
I20260709 13:03:24.050635 29006 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c100c1aaf1b493089df815bded98302" member_type: VOTER }
I20260709 13:03:24.051005 29006 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:24.051285 29006 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c100c1aaf1b493089df815bded98302, State: Initialized, Role: FOLLOWER
I20260709 13:03:24.051966 29006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c100c1aaf1b493089df815bded98302" member_type: VOTER }
I20260709 13:03:24.052412 29006 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:03:24.052667 29006 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:03:24.052937 29006 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:24.057221 29006 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c100c1aaf1b493089df815bded98302" member_type: VOTER }
I20260709 13:03:24.057740 29006 leader_election.cc:304] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c100c1aaf1b493089df815bded98302; no voters: 
I20260709 13:03:24.058911 29006 leader_election.cc:290] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:03:24.059329 29009 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:24.060734 29009 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [term 1 LEADER]: Becoming Leader. State: Replica: 9c100c1aaf1b493089df815bded98302, State: Running, Role: LEADER
I20260709 13:03:24.061337 29009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c100c1aaf1b493089df815bded98302" member_type: VOTER }
I20260709 13:03:24.062044 29006 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:03:24.065980 29010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c100c1aaf1b493089df815bded98302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c100c1aaf1b493089df815bded98302" member_type: VOTER } }
I20260709 13:03:24.066941 29011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c100c1aaf1b493089df815bded98302. Latest consensus state: current_term: 1 leader_uuid: "9c100c1aaf1b493089df815bded98302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c100c1aaf1b493089df815bded98302" member_type: VOTER } }
I20260709 13:03:24.067363 29010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:24.067461 29011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:24.069401 29016 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:03:24.074697 29016 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:03:24.077709 27934 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:03:24.083741 29016 catalog_manager.cc:1383] Generated new cluster ID: 50f02ecca41d4cec8683dd4dd6dcace3
I20260709 13:03:24.084069 29016 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:03:24.099435 29016 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:03:24.100772 29016 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:03:24.113272 29016 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Generated new TSK 0
I20260709 13:03:24.113893 29016 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:03:24.145089 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:24.152184 29027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:24.152695 29028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:24.153813 29030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:24.154088 27934 server_base.cc:1061] running on GCE node
I20260709 13:03:24.155061 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:24.155241 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:24.155359 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602204155348 us; error 0 us; skew 500 ppm
I20260709 13:03:24.155824 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:24.158208 27934 webserver.cc:533] Webserver started at http://127.27.71.129:37587/ using document root <none> and password file <none>
I20260709 13:03:24.158694 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:24.158852 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:24.159068 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:24.160236 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/instance:
uuid: "a936878f59c840a6bd176a1d2d7347b1"
format_stamp: "Formatted at 2026-07-09 13:03:24 on dist-test-slave-mlt7"
I20260709 13:03:24.165205 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:03:24.168956 29035 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:24.169806 27934 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 13:03:24.170118 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root
uuid: "a936878f59c840a6bd176a1d2d7347b1"
format_stamp: "Formatted at 2026-07-09 13:03:24 on dist-test-slave-mlt7"
I20260709 13:03:24.170440 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:24.187541 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:24.188907 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:24.190444 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:24.193583 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:24.193826 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:24.194108 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:24.194334 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:24.240564 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.129:41631
I20260709 13:03:24.240705 29097 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.129:41631 every 8 connection(s)
I20260709 13:03:24.245445 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:24.252804 29102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:24.254845 29103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:24.256585 29105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:24.257633 27934 server_base.cc:1061] running on GCE node
I20260709 13:03:24.258920 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:24.259104 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:24.259358 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602204259325 us; error 0 us; skew 500 ppm
I20260709 13:03:24.259972 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:24.262601 27934 webserver.cc:533] Webserver started at http://127.27.71.130:35347/ using document root <none> and password file <none>
I20260709 13:03:24.263226 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:24.263468 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:24.263890 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:24.264891 29098 heartbeater.cc:344] Connected to a master server at 127.27.71.190:37537
I20260709 13:03:24.265296 29098 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:24.265327 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/instance:
uuid: "730186a8c70f43f2a57d80163bbc37d2"
format_stamp: "Formatted at 2026-07-09 13:03:24 on dist-test-slave-mlt7"
I20260709 13:03:24.266129 29098 heartbeater.cc:507] Master 127.27.71.190:37537 requested a full tablet report, sending...
I20260709 13:03:24.268388 28971 ts_manager.cc:194] Registered new tserver with Master: a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:03:24.270678 28971 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58570
I20260709 13:03:24.270819 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:03:24.275408 29110 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:24.276372 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:03:24.276706 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root
uuid: "730186a8c70f43f2a57d80163bbc37d2"
format_stamp: "Formatted at 2026-07-09 13:03:24 on dist-test-slave-mlt7"
I20260709 13:03:24.276988 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:24.299726 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:24.300961 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:24.302429 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:24.304672 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:24.304888 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:24.305132 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:24.305337 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:24.344612 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.130:35337
I20260709 13:03:24.344734 29172 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.130:35337 every 8 connection(s)
I20260709 13:03:24.349450 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:24.357473 29176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:24.358670 29177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:24.361755 27934 server_base.cc:1061] running on GCE node
W20260709 13:03:24.362531 29179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:24.363459 29173 heartbeater.cc:344] Connected to a master server at 127.27.71.190:37537
I20260709 13:03:24.363533 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:24.363860 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:24.363960 29173 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:24.364113 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602204364090 us; error 0 us; skew 500 ppm
I20260709 13:03:24.364899 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:24.364878 29173 heartbeater.cc:507] Master 127.27.71.190:37537 requested a full tablet report, sending...
I20260709 13:03:24.367264 28971 ts_manager.cc:194] Registered new tserver with Master: 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:24.368003 27934 webserver.cc:533] Webserver started at http://127.27.71.131:40815/ using document root <none> and password file <none>
I20260709 13:03:24.368510 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:24.368721 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:24.368993 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:24.368939 28971 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58574
I20260709 13:03:24.370514 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/instance:
uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
format_stamp: "Formatted at 2026-07-09 13:03:24 on dist-test-slave-mlt7"
I20260709 13:03:24.375422 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:03:24.378819 29184 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:24.379679 27934 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:03:24.380038 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root
uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
format_stamp: "Formatted at 2026-07-09 13:03:24 on dist-test-slave-mlt7"
I20260709 13:03:24.380366 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:24.401947 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:24.403158 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:24.404769 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:24.407109 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:24.407347 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:24.407593 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:24.407768 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:24.446525 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.131:45851
I20260709 13:03:24.446688 29246 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.131:45851 every 8 connection(s)
I20260709 13:03:24.460276 29247 heartbeater.cc:344] Connected to a master server at 127.27.71.190:37537
I20260709 13:03:24.460637 29247 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:24.461675 29247 heartbeater.cc:507] Master 127.27.71.190:37537 requested a full tablet report, sending...
I20260709 13:03:24.464169 28971 ts_manager.cc:194] Registered new tserver with Master: 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:24.464802 27934 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015000364s
I20260709 13:03:24.465629 28971 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58588
I20260709 13:03:24.492558 28971 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58590:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:03:24.498539 28971 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:03:24.697142 29137 tablet_service.cc:1511] Processing CreateTablet for tablet a385068bbb144e6fa0e8e9974398b1b9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:24.697762 29138 tablet_service.cc:1511] Processing CreateTablet for tablet 9e3e05d4f95940f5a4cac560d26d0ec8 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:24.698638 29137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a385068bbb144e6fa0e8e9974398b1b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.702286 29136 tablet_service.cc:1511] Processing CreateTablet for tablet 44d88fbf78394409bcacfadb40f377d6 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:24.703622 29136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44d88fbf78394409bcacfadb40f377d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.708940 29134 tablet_service.cc:1511] Processing CreateTablet for tablet d5d7d7e7e55b46f89b835614737ee91b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:24.710234 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5d7d7e7e55b46f89b835614737ee91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.712476 29133 tablet_service.cc:1511] Processing CreateTablet for tablet cdb0c0ac78fc4de7a353a995edd44346 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:24.713164 29135 tablet_service.cc:1511] Processing CreateTablet for tablet be73fc18460b4609ad4da2946e3acb87 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:24.713786 29133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdb0c0ac78fc4de7a353a995edd44346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.714504 29135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be73fc18460b4609ad4da2946e3acb87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.716993 29132 tablet_service.cc:1511] Processing CreateTablet for tablet d2ee8b4c1bf649b9b89ad6243f3890a8 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:24.718484 29132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ee8b4c1bf649b9b89ad6243f3890a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.719871 29212 tablet_service.cc:1511] Processing CreateTablet for tablet 9e3e05d4f95940f5a4cac560d26d0ec8 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:24.721252 29212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e3e05d4f95940f5a4cac560d26d0ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.721127 29209 tablet_service.cc:1511] Processing CreateTablet for tablet be73fc18460b4609ad4da2946e3acb87 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:24.722429 29209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be73fc18460b4609ad4da2946e3acb87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.725067 29211 tablet_service.cc:1511] Processing CreateTablet for tablet a385068bbb144e6fa0e8e9974398b1b9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:24.726532 29211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a385068bbb144e6fa0e8e9974398b1b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.726263 29207 tablet_service.cc:1511] Processing CreateTablet for tablet cdb0c0ac78fc4de7a353a995edd44346 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:24.726991 29056 tablet_service.cc:1511] Processing CreateTablet for tablet e96bceabcbf747e0acec327bc1abc729 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:24.727588 29207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdb0c0ac78fc4de7a353a995edd44346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.728413 29056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e96bceabcbf747e0acec327bc1abc729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.728638 29210 tablet_service.cc:1511] Processing CreateTablet for tablet 44d88fbf78394409bcacfadb40f377d6 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:24.729091 29062 tablet_service.cc:1511] Processing CreateTablet for tablet a385068bbb144e6fa0e8e9974398b1b9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:24.730362 29062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a385068bbb144e6fa0e8e9974398b1b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.733047 29061 tablet_service.cc:1511] Processing CreateTablet for tablet 44d88fbf78394409bcacfadb40f377d6 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:24.734366 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44d88fbf78394409bcacfadb40f377d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.737068 29060 tablet_service.cc:1511] Processing CreateTablet for tablet be73fc18460b4609ad4da2946e3acb87 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:24.738410 29060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be73fc18460b4609ad4da2946e3acb87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.741129 29058 tablet_service.cc:1511] Processing CreateTablet for tablet cdb0c0ac78fc4de7a353a995edd44346 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:24.742413 29058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdb0c0ac78fc4de7a353a995edd44346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.725067 29208 tablet_service.cc:1511] Processing CreateTablet for tablet d5d7d7e7e55b46f89b835614737ee91b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:24.743572 29063 tablet_service.cc:1511] Processing CreateTablet for tablet 9e3e05d4f95940f5a4cac560d26d0ec8 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:24.744153 29057 tablet_service.cc:1511] Processing CreateTablet for tablet d2ee8b4c1bf649b9b89ad6243f3890a8 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:24.744899 29063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e3e05d4f95940f5a4cac560d26d0ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.746271 29206 tablet_service.cc:1511] Processing CreateTablet for tablet d2ee8b4c1bf649b9b89ad6243f3890a8 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:24.747635 29206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ee8b4c1bf649b9b89ad6243f3890a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.749058 29059 tablet_service.cc:1511] Processing CreateTablet for tablet d5d7d7e7e55b46f89b835614737ee91b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:24.750465 29059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5d7d7e7e55b46f89b835614737ee91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.758517 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44d88fbf78394409bcacfadb40f377d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.773469 29138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e3e05d4f95940f5a4cac560d26d0ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.770963 29057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ee8b4c1bf649b9b89ad6243f3890a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.782413 29131 tablet_service.cc:1511] Processing CreateTablet for tablet e96bceabcbf747e0acec327bc1abc729 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:24.783877 29131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e96bceabcbf747e0acec327bc1abc729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.784139 29208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5d7d7e7e55b46f89b835614737ee91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.784332 29212 tablet_service.cc:1511] Processing CreateTablet for tablet e96bceabcbf747e0acec327bc1abc729 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:24.785585 29212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e96bceabcbf747e0acec327bc1abc729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.806813 29262 tablet_bootstrap.cc:492] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:24.807876 29055 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca58eb3f3b147bc941a1c21818d370c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:24.809253 29055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca58eb3f3b147bc941a1c21818d370c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.824496 29206 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca58eb3f3b147bc941a1c21818d370c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:24.828512 29054 tablet_service.cc:1511] Processing CreateTablet for tablet a9e4dbd865b648aaa1bce128c9de27af (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:24.829454 29135 tablet_service.cc:1511] Processing CreateTablet for tablet a9e4dbd865b648aaa1bce128c9de27af (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:24.831176 29134 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca58eb3f3b147bc941a1c21818d370c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:24.839407 29054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e4dbd865b648aaa1bce128c9de27af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.857229 29263 tablet_bootstrap.cc:492] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:24.872392 29211 tablet_service.cc:1511] Processing CreateTablet for tablet a9e4dbd865b648aaa1bce128c9de27af (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:24.873009 29061 tablet_service.cc:1511] Processing CreateTablet for tablet 344aa5f73f0f4124b1512832a0bca2c9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:24.873548 29131 tablet_service.cc:1511] Processing CreateTablet for tablet 344aa5f73f0f4124b1512832a0bca2c9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:24.874549 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 344aa5f73f0f4124b1512832a0bca2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.869443 29206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca58eb3f3b147bc941a1c21818d370c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.875658 29211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e4dbd865b648aaa1bce128c9de27af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.876025 29263 tablet_bootstrap.cc:654] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:24.878091 29135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e4dbd865b648aaa1bce128c9de27af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.882422 29131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 344aa5f73f0f4124b1512832a0bca2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.884148 29262 tablet_bootstrap.cc:654] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:24.887609 29207 tablet_service.cc:1511] Processing CreateTablet for tablet 344aa5f73f0f4124b1512832a0bca2c9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:24.888196 29264 tablet_bootstrap.cc:492] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:24.888990 29207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 344aa5f73f0f4124b1512832a0bca2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.890965 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca58eb3f3b147bc941a1c21818d370c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.908980 29138 tablet_service.cc:1511] Processing CreateTablet for tablet e8195256ce88462cab8bb734ddbb65ad (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:24.910799 29212 tablet_service.cc:1511] Processing CreateTablet for tablet e8195256ce88462cab8bb734ddbb65ad (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:24.912345 29212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8195256ce88462cab8bb734ddbb65ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.935091 29264 tablet_bootstrap.cc:654] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:24.908979 29056 tablet_service.cc:1511] Processing CreateTablet for tablet e8195256ce88462cab8bb734ddbb65ad (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:24.936519 29056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8195256ce88462cab8bb734ddbb65ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.936599 29210 tablet_service.cc:1511] Processing CreateTablet for tablet d8373166c3664170b540f729a4e3c93f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:24.938014 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8373166c3664170b540f729a4e3c93f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.940382 29138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8195256ce88462cab8bb734ddbb65ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.947394 29208 tablet_service.cc:1511] Processing CreateTablet for tablet cc31aa85851e41609957a765b2d40774 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:24.947734 29057 tablet_service.cc:1511] Processing CreateTablet for tablet d8373166c3664170b540f729a4e3c93f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:24.948776 29208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc31aa85851e41609957a765b2d40774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.950795 29057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8373166c3664170b540f729a4e3c93f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.951745 29130 tablet_service.cc:1511] Processing CreateTablet for tablet 8c8a2336e86c4846bf4e69ec61ba3fe0 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:24.953089 29130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c8a2336e86c4846bf4e69ec61ba3fe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.956954 29136 tablet_service.cc:1511] Processing CreateTablet for tablet cc31aa85851e41609957a765b2d40774 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:24.958271 29136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc31aa85851e41609957a765b2d40774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.961345 29137 tablet_service.cc:1511] Processing CreateTablet for tablet d8373166c3664170b540f729a4e3c93f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:24.962752 29137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8373166c3664170b540f729a4e3c93f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.966785 29054 tablet_service.cc:1511] Processing CreateTablet for tablet cc31aa85851e41609957a765b2d40774 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:24.967437 29060 tablet_service.cc:1511] Processing CreateTablet for tablet 8c8a2336e86c4846bf4e69ec61ba3fe0 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:24.968286 29054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc31aa85851e41609957a765b2d40774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.968943 29060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c8a2336e86c4846bf4e69ec61ba3fe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.977185 29212 tablet_service.cc:1511] Processing CreateTablet for tablet 8c8a2336e86c4846bf4e69ec61ba3fe0 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:24.978643 29212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c8a2336e86c4846bf4e69ec61ba3fe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.980290 29055 tablet_service.cc:1511] Processing CreateTablet for tablet 12e21412fcd24f95a8e3635fc5bbe138 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:24.983072 29055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12e21412fcd24f95a8e3635fc5bbe138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:24.995211 29209 tablet_service.cc:1511] Processing CreateTablet for tablet 12e21412fcd24f95a8e3635fc5bbe138 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:24.996757 29209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12e21412fcd24f95a8e3635fc5bbe138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.001681 29262 tablet_bootstrap.cc:492] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:25.002242 29262 ts_tablet_manager.cc:1403] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.196s	user 0.009s	sys 0.005s
I20260709 13:03:25.001268 29133 tablet_service.cc:1511] Processing CreateTablet for tablet 12e21412fcd24f95a8e3635fc5bbe138 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:25.004823 29133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12e21412fcd24f95a8e3635fc5bbe138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.018345 29264 tablet_bootstrap.cc:492] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:25.018908 29264 ts_tablet_manager.cc:1403] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.150s	user 0.009s	sys 0.024s
I20260709 13:03:25.021974 29264 raft_consensus.cc:359] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.022819 29264 raft_consensus.cc:385] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.023156 29264 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.024243 29264 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.061864 29210 tablet_service.cc:1511] Processing CreateTablet for tablet ac412099b9f64d4e8bcceebe1c7ffe87 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:25.063479 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac412099b9f64d4e8bcceebe1c7ffe87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.064000 29263 tablet_bootstrap.cc:492] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:25.064487 29263 ts_tablet_manager.cc:1403] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.214s	user 0.013s	sys 0.009s
I20260709 13:03:25.066129 29131 tablet_service.cc:1511] Processing CreateTablet for tablet cb329a9e56064f1480f8f42e09ff6679 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:25.067449 29131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb329a9e56064f1480f8f42e09ff6679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.067796 29263 raft_consensus.cc:359] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.068687 29263 raft_consensus.cc:385] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.069119 29263 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.070106 29263 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:25.073334 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:25.075690 29130 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdfda5dcdd94083afe1620564f9d376 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:25.077399 29130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdfda5dcdd94083afe1620564f9d376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.081945 29268 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.082086 29098 heartbeater.cc:499] Master 127.27.71.190:37537 was elected leader, sending a full tablet report...
I20260709 13:03:25.082453 29268 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.085862 29211 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdfda5dcdd94083afe1620564f9d376 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:25.087234 29211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdfda5dcdd94083afe1620564f9d376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.087718 29264 ts_tablet_manager.cc:1434] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.013s
I20260709 13:03:25.088759 29264 tablet_bootstrap.cc:492] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:25.100616 29131 tablet_service.cc:1511] Processing CreateTablet for tablet c0c33ea0ff53432c8be5e364242b23b3 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:25.100425 29263 ts_tablet_manager.cc:1434] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.010s
I20260709 13:03:25.103461 29131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c33ea0ff53432c8be5e364242b23b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.104403 29262 raft_consensus.cc:359] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.106696 29134 tablet_service.cc:1511] Processing CreateTablet for tablet b09437a3ae8d46379ccdbb1829b14f27 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:25.107081 29262 raft_consensus.cc:385] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.107429 29208 tablet_service.cc:1511] Processing CreateTablet for tablet b09437a3ae8d46379ccdbb1829b14f27 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:25.108116 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b09437a3ae8d46379ccdbb1829b14f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.107884 29262 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.110288 29057 tablet_service.cc:1511] Processing CreateTablet for tablet ac412099b9f64d4e8bcceebe1c7ffe87 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:25.110602 29055 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdfda5dcdd94083afe1620564f9d376 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:25.111846 29057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac412099b9f64d4e8bcceebe1c7ffe87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.112345 29054 tablet_service.cc:1511] Processing CreateTablet for tablet b09437a3ae8d46379ccdbb1829b14f27 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:25.106330 29209 tablet_service.cc:1511] Processing CreateTablet for tablet cb329a9e56064f1480f8f42e09ff6679 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:25.112923 29061 tablet_service.cc:1511] Processing CreateTablet for tablet c0c33ea0ff53432c8be5e364242b23b3 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:25.121001 29056 tablet_service.cc:1511] Processing CreateTablet for tablet cb329a9e56064f1480f8f42e09ff6679 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:25.124632 29054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b09437a3ae8d46379ccdbb1829b14f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.127396 29264 tablet_bootstrap.cc:654] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:25.128233 29055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdfda5dcdd94083afe1620564f9d376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.128233 29210 tablet_service.cc:1511] Processing CreateTablet for tablet c0c33ea0ff53432c8be5e364242b23b3 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:25.130472 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c33ea0ff53432c8be5e364242b23b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.129202 29138 tablet_service.cc:1511] Processing CreateTablet for tablet ac412099b9f64d4e8bcceebe1c7ffe87 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:25.134510 29208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b09437a3ae8d46379ccdbb1829b14f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.140170 29247 heartbeater.cc:499] Master 127.27.71.190:37537 was elected leader, sending a full tablet report...
I20260709 13:03:25.140578 29209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb329a9e56064f1480f8f42e09ff6679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.144749 29138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac412099b9f64d4e8bcceebe1c7ffe87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.144883 29263 tablet_bootstrap.cc:492] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:25.145680 29060 tablet_service.cc:1511] Processing CreateTablet for tablet 79cdfc6a725145348b70034422ef477b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:25.148952 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c33ea0ff53432c8be5e364242b23b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.152212 29056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb329a9e56064f1480f8f42e09ff6679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.154045 29136 tablet_service.cc:1511] Processing CreateTablet for tablet 79cdfc6a725145348b70034422ef477b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:25.155014 29060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79cdfc6a725145348b70034422ef477b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.155371 29136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79cdfc6a725145348b70034422ef477b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.157672 29268 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:25.166038 29133 tablet_service.cc:1511] Processing CreateTablet for tablet c54509bf16d642138312814fefa3f6dc (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:25.112540 29262 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.172719 29133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c54509bf16d642138312814fefa3f6dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.172900 29137 tablet_service.cc:1511] Processing CreateTablet for tablet 2e40d8d1f334428db02ee54c8467d058 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:25.174288 29137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e40d8d1f334428db02ee54c8467d058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.177098 29207 tablet_service.cc:1511] Processing CreateTablet for tablet 2e40d8d1f334428db02ee54c8467d058 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:25.178416 29207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e40d8d1f334428db02ee54c8467d058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.182946 29211 tablet_service.cc:1511] Processing CreateTablet for tablet c54509bf16d642138312814fefa3f6dc (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:25.182969 29212 tablet_service.cc:1511] Processing CreateTablet for tablet 79cdfc6a725145348b70034422ef477b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:25.184677 29212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79cdfc6a725145348b70034422ef477b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.188467 29211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c54509bf16d642138312814fefa3f6dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.192247 29263 tablet_bootstrap.cc:654] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:25.197930 29135 tablet_service.cc:1511] Processing CreateTablet for tablet 128a7406f55d4400bfd5fadcbc14d065 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:25.199291 29135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 128a7406f55d4400bfd5fadcbc14d065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.223769 29132 tablet_service.cc:1511] Processing CreateTablet for tablet c59eab5922dc4c07a3a86e915e3cab7f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:25.224280 29206 tablet_service.cc:1511] Processing CreateTablet for tablet 128a7406f55d4400bfd5fadcbc14d065 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:25.225370 29132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59eab5922dc4c07a3a86e915e3cab7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.225633 29206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 128a7406f55d4400bfd5fadcbc14d065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.225775 29129 tablet_service.cc:1511] Processing CreateTablet for tablet 40b726ca47da439b9be7ad6d531144fe (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:25.227381 29129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40b726ca47da439b9be7ad6d531144fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.239419 29173 heartbeater.cc:499] Master 127.27.71.190:37537 was elected leader, sending a full tablet report...
I20260709 13:03:25.239840 29205 tablet_service.cc:1511] Processing CreateTablet for tablet c59eab5922dc4c07a3a86e915e3cab7f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:25.241165 29205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59eab5922dc4c07a3a86e915e3cab7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.245036 29063 tablet_service.cc:1511] Processing CreateTablet for tablet c54509bf16d642138312814fefa3f6dc (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:25.245149 29204 tablet_service.cc:1511] Processing CreateTablet for tablet 40b726ca47da439b9be7ad6d531144fe (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:25.246408 29063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c54509bf16d642138312814fefa3f6dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.246503 29204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40b726ca47da439b9be7ad6d531144fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.248534 29134 tablet_service.cc:1511] Processing CreateTablet for tablet c24ba590bc46427dbac6cd4a4da67ee8 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:25.249852 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24ba590bc46427dbac6cd4a4da67ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.252786 29264 tablet_bootstrap.cc:492] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:25.254703 29264 ts_tablet_manager.cc:1403] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.166s	user 0.019s	sys 0.006s
I20260709 13:03:25.256245 29203 tablet_service.cc:1511] Processing CreateTablet for tablet c24ba590bc46427dbac6cd4a4da67ee8 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:25.257597 29203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24ba590bc46427dbac6cd4a4da67ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.265509 29059 tablet_service.cc:1511] Processing CreateTablet for tablet c59eab5922dc4c07a3a86e915e3cab7f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:25.265738 29058 tablet_service.cc:1511] Processing CreateTablet for tablet 40b726ca47da439b9be7ad6d531144fe (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:25.266273 29062 tablet_service.cc:1511] Processing CreateTablet for tablet c24ba590bc46427dbac6cd4a4da67ee8 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:25.267158 29059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59eab5922dc4c07a3a86e915e3cab7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.268738 29062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24ba590bc46427dbac6cd4a4da67ee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.273051 29058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40b726ca47da439b9be7ad6d531144fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.276758 29264 raft_consensus.cc:359] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.276876 29054 tablet_service.cc:1511] Processing CreateTablet for tablet 2e40d8d1f334428db02ee54c8467d058 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:25.277544 29264 raft_consensus.cc:385] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.277899 29264 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.278246 29054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e40d8d1f334428db02ee54c8467d058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.279804 29055 tablet_service.cc:1511] Processing CreateTablet for tablet 813e480732fd4061837e784d4d768b1f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:25.288681 29055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 813e480732fd4061837e784d4d768b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.283381 29264 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.293247 29264 ts_tablet_manager.cc:1434] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:03:25.294277 29264 tablet_bootstrap.cc:492] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:25.284919 29061 tablet_service.cc:1511] Processing CreateTablet for tablet 128a7406f55d4400bfd5fadcbc14d065 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:25.300599 29263 tablet_bootstrap.cc:492] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:25.301172 29263 ts_tablet_manager.cc:1403] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.157s	user 0.016s	sys 0.003s
I20260709 13:03:25.304086 29263 raft_consensus.cc:359] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.304802 29263 raft_consensus.cc:385] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.305171 29263 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.310992 29268 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.311556 29268 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.311074 29263 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.313657 29268 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:25.315323 29263 ts_tablet_manager.cc:1434] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:03:25.310267 29264 tablet_bootstrap.cc:654] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:25.324556 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 128a7406f55d4400bfd5fadcbc14d065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.325956 29138 tablet_service.cc:1511] Processing CreateTablet for tablet 813e480732fd4061837e784d4d768b1f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:25.328508 29273 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.329216 29273 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.332453 29262 ts_tablet_manager.cc:1434] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.330s	user 0.013s	sys 0.010s
I20260709 13:03:25.333499 29262 tablet_bootstrap.cc:492] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:25.346936 29211 tablet_service.cc:1511] Processing CreateTablet for tablet 813e480732fd4061837e784d4d768b1f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:25.347525 29273 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:25.348466 29211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 813e480732fd4061837e784d4d768b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.350046 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:25.351562 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:25.351763 29063 tablet_service.cc:1511] Processing CreateTablet for tablet e250871b4ad74e079a538cdb04065b00 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:25.353083 29063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e250871b4ad74e079a538cdb04065b00. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:25.353355 29038 leader_election.cc:343] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:25.354014 29038 leader_election.cc:343] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.364831 29263 tablet_bootstrap.cc:492] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:25.369019 29210 tablet_service.cc:1511] Processing CreateTablet for tablet e250871b4ad74e079a538cdb04065b00 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:25.370368 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e250871b4ad74e079a538cdb04065b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.327770 29138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 813e480732fd4061837e784d4d768b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.396797 29263 tablet_bootstrap.cc:654] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:25.417531 29270 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.418061 29270 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.436403 29063 tablet_service.cc:1511] Processing CreateTablet for tablet 348ae3050d97486fa402a5da2bbdce46 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:25.437889 29063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 348ae3050d97486fa402a5da2bbdce46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.441560 29207 tablet_service.cc:1511] Processing CreateTablet for tablet cddea8e9cff6495ba269c65b0edf8b6c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:25.443053 29207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cddea8e9cff6495ba269c65b0edf8b6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.450809 29059 tablet_service.cc:1511] Processing CreateTablet for tablet cddea8e9cff6495ba269c65b0edf8b6c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:25.458696 29270 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:25.470548 29270 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.476975 29059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cddea8e9cff6495ba269c65b0edf8b6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.485086 29211 tablet_service.cc:1511] Processing CreateTablet for tablet 348ae3050d97486fa402a5da2bbdce46 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:25.486624 29211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 348ae3050d97486fa402a5da2bbdce46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.491858 29264 tablet_bootstrap.cc:492] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:25.496536 29264 ts_tablet_manager.cc:1403] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.202s	user 0.028s	sys 0.023s
I20260709 13:03:25.499231 29264 raft_consensus.cc:359] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.499861 29264 raft_consensus.cc:385] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.501479 29264 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.488107 29270 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.508842 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:25.510783 29113 leader_election.cc:343] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.512269 29270 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:03:25.519814 29262 tablet_bootstrap.cc:654] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:25.502210 29264 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.529181 29138 tablet_service.cc:1511] Processing CreateTablet for tablet e250871b4ad74e079a538cdb04065b00 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:25.530084 29063 tablet_service.cc:1511] Processing CreateTablet for tablet a9f522a55d4b4051a75e0bf87cd2619e (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:25.530577 29138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e250871b4ad74e079a538cdb04065b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.530848 29203 tablet_service.cc:1511] Processing CreateTablet for tablet a9f522a55d4b4051a75e0bf87cd2619e (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:25.531351 29063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f522a55d4b4051a75e0bf87cd2619e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.532138 29203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f522a55d4b4051a75e0bf87cd2619e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.536044 29206 tablet_service.cc:1511] Processing CreateTablet for tablet 88cabbc6d2424a53a76857f415831c36 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:25.537403 29206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88cabbc6d2424a53a76857f415831c36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.537892 29134 tablet_service.cc:1511] Processing CreateTablet for tablet cddea8e9cff6495ba269c65b0edf8b6c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:25.539165 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cddea8e9cff6495ba269c65b0edf8b6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.540068 29132 tablet_service.cc:1511] Processing CreateTablet for tablet 348ae3050d97486fa402a5da2bbdce46 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:25.540895 29061 tablet_service.cc:1511] Processing CreateTablet for tablet 88cabbc6d2424a53a76857f415831c36 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:25.542280 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88cabbc6d2424a53a76857f415831c36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.542165 29137 tablet_service.cc:1511] Processing CreateTablet for tablet a9f522a55d4b4051a75e0bf87cd2619e (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:25.541842 29263 tablet_bootstrap.cc:492] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:25.543485 29137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f522a55d4b4051a75e0bf87cd2619e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.547048 29263 ts_tablet_manager.cc:1403] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.182s	user 0.019s	sys 0.029s
I20260709 13:03:25.544355 29129 tablet_service.cc:1511] Processing CreateTablet for tablet 88cabbc6d2424a53a76857f415831c36 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:25.548870 29129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88cabbc6d2424a53a76857f415831c36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.550563 29263 raft_consensus.cc:359] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.551352 29263 raft_consensus.cc:385] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.556396 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:25.557178 29147 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:25.558828 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:25.559504 29073 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:25.560665 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:25.562350 29264 ts_tablet_manager.cc:1434] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.065s	user 0.004s	sys 0.002s
I20260709 13:03:25.563452 29264 tablet_bootstrap.cc:492] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:25.570063 29132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 348ae3050d97486fa402a5da2bbdce46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.551672 29263 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
W20260709 13:03:25.576045 28955 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 46192.5us
I20260709 13:03:25.576448 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:25.588500 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:03:25.589144 29039 leader_election.cc:343] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.589382 29273 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:25.589751 29273 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.590101 29273 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:25.590167 29268 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:25.590313 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:25.590566 29268 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.590930 29268 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:25.591450 29059 tablet_service.cc:1511] Processing CreateTablet for tablet 43a52106dbee45398d36a633f281eef6 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:25.591890 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:25.593694 29059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43a52106dbee45398d36a633f281eef6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:25.593766 29188 leader_election.cc:343] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.594518 29135 tablet_service.cc:1511] Processing CreateTablet for tablet 43a52106dbee45398d36a633f281eef6 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:25.595798 29135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43a52106dbee45398d36a633f281eef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.589638 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:25.597451 29269 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:25.594537 29188 leader_election.cc:343] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.604445 29263 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.606254 29204 tablet_service.cc:1511] Processing CreateTablet for tablet 43a52106dbee45398d36a633f281eef6 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:25.607652 29204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43a52106dbee45398d36a633f281eef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.612637 29263 ts_tablet_manager.cc:1434] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.065s	user 0.004s	sys 0.004s
I20260709 13:03:25.611873 29273 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.617385 29264 tablet_bootstrap.cc:654] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:25.621114 29263 tablet_bootstrap.cc:492] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:25.622677 29273 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:25.635362 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:25.623396 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:25.634879 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:25.624948 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:25.637091 29205 tablet_service.cc:1511] Processing CreateTablet for tablet 3aaa941bea174ae08dc8aad56e0425f2 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
W20260709 13:03:25.637441 29113 leader_election.cc:343] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.625962 29268 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.634879 29063 tablet_service.cc:1511] Processing CreateTablet for tablet 3aaa941bea174ae08dc8aad56e0425f2 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
W20260709 13:03:25.638157 29185 leader_election.cc:343] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.638402 29205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aaa941bea174ae08dc8aad56e0425f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.638778 29063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aaa941bea174ae08dc8aad56e0425f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.638607 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:25.639492 29270 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:25.640769 29185 leader_election.cc:343] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.641188 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:25.641999 29270 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:25.655606 29269 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.655202 29061 tablet_service.cc:1511] Processing CreateTablet for tablet e68315e143764938be60ec716705c15f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:25.656119 29269 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.658099 29268 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:25.658639 29210 tablet_service.cc:1511] Processing CreateTablet for tablet e68315e143764938be60ec716705c15f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:25.659750 29262 tablet_bootstrap.cc:492] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:25.659847 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:25.660373 29262 ts_tablet_manager.cc:1403] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.327s	user 0.014s	sys 0.008s
I20260709 13:03:25.658206 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e68315e143764938be60ec716705c15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.660846 29269 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:25.655630 29073 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:25.664788 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:25.665853 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:25.659979 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e68315e143764938be60ec716705c15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.666600 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:25.668263 29148 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 1: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:03:25.669235 29273 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:25.669685 29134 tablet_service.cc:1511] Processing CreateTablet for tablet e68315e143764938be60ec716705c15f (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:25.671056 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e68315e143764938be60ec716705c15f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:25.674274 29038 leader_election.cc:343] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.663609 29262 raft_consensus.cc:359] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.679991 29262 raft_consensus.cc:385] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:03:25.679723 29039 leader_election.cc:343] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.680446 29262 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.681188 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:25.682997 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:25.683620 29269 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:03:25.684580 29038 leader_election.cc:343] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.685065 29038 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:25.685860 29269 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:25.693341 29059 tablet_service.cc:1511] Processing CreateTablet for tablet d506d5cb9c9943c49f64dbb8fa09e5c9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:25.694689 29059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d506d5cb9c9943c49f64dbb8fa09e5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.698374 29205 tablet_service.cc:1511] Processing CreateTablet for tablet d506d5cb9c9943c49f64dbb8fa09e5c9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:25.700158 29138 tablet_service.cc:1511] Processing CreateTablet for tablet 3aaa941bea174ae08dc8aad56e0425f2 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:25.701933 29138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aaa941bea174ae08dc8aad56e0425f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.703969 29263 tablet_bootstrap.cc:654] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:25.707437 29054 tablet_service.cc:1511] Processing CreateTablet for tablet 082c896dd04e451198eba8392e70fba7 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:25.708819 29054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 082c896dd04e451198eba8392e70fba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.709748 29207 tablet_service.cc:1511] Processing CreateTablet for tablet 082c896dd04e451198eba8392e70fba7 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:25.710976 29205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d506d5cb9c9943c49f64dbb8fa09e5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.719523 29207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 082c896dd04e451198eba8392e70fba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.732013 29134 tablet_service.cc:1511] Processing CreateTablet for tablet d506d5cb9c9943c49f64dbb8fa09e5c9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:25.733044 29056 tablet_service.cc:1511] Processing CreateTablet for tablet 3842b36ffd134cd28c657475df48e741 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:25.733645 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d506d5cb9c9943c49f64dbb8fa09e5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.734453 29056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3842b36ffd134cd28c657475df48e741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.732156 29206 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1cace028c94f79adfce2be604fb35a (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:25.730605 29138 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1cace028c94f79adfce2be604fb35a (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:25.738299 29138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1cace028c94f79adfce2be604fb35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.739260 29264 tablet_bootstrap.cc:492] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:25.681277 29262 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.741489 29206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1cace028c94f79adfce2be604fb35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.744382 29137 tablet_service.cc:1511] Processing CreateTablet for tablet 3842b36ffd134cd28c657475df48e741 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:25.744802 29059 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1cace028c94f79adfce2be604fb35a (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:25.745806 29137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3842b36ffd134cd28c657475df48e741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.746217 29059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1cace028c94f79adfce2be604fb35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.751603 29210 tablet_service.cc:1511] Processing CreateTablet for tablet 3842b36ffd134cd28c657475df48e741 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:25.752588 29135 tablet_service.cc:1511] Processing CreateTablet for tablet 082c896dd04e451198eba8392e70fba7 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:25.753954 29135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 082c896dd04e451198eba8392e70fba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.753064 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3842b36ffd134cd28c657475df48e741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.765601 29263 tablet_bootstrap.cc:492] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:25.765125 29054 tablet_service.cc:1511] Processing CreateTablet for tablet e9334a7c40c744cbbff4dec0358f721b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:25.774902 29054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9334a7c40c744cbbff4dec0358f721b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.775403 29134 tablet_service.cc:1511] Processing CreateTablet for tablet e9334a7c40c744cbbff4dec0358f721b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:25.776942 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9334a7c40c744cbbff4dec0358f721b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.777010 29206 tablet_service.cc:1511] Processing CreateTablet for tablet e9334a7c40c744cbbff4dec0358f721b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:25.766136 29263 ts_tablet_manager.cc:1403] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.000s
I20260709 13:03:25.778432 29206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9334a7c40c744cbbff4dec0358f721b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.784737 29262 ts_tablet_manager.cc:1434] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.124s	user 0.007s	sys 0.001s
I20260709 13:03:25.785987 29262 tablet_bootstrap.cc:492] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:25.791956 29264 ts_tablet_manager.cc:1403] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.203s	user 0.019s	sys 0.019s
I20260709 13:03:25.782157 29263 raft_consensus.cc:359] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.793040 29263 raft_consensus.cc:385] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.793509 29263 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.795179 29264 raft_consensus.cc:359] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.799631 29134 tablet_service.cc:1511] Processing CreateTablet for tablet 2fae9bf99a434b83a73451cee1c56e6a (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:25.802516 29205 tablet_service.cc:1511] Processing CreateTablet for tablet 2fae9bf99a434b83a73451cee1c56e6a (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:25.803990 29205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fae9bf99a434b83a73451cee1c56e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.804030 29264 raft_consensus.cc:385] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.804499 29264 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.805657 29262 tablet_bootstrap.cc:654] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:25.805353 29264 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.808974 29061 tablet_service.cc:1511] Processing CreateTablet for tablet 2fae9bf99a434b83a73451cee1c56e6a (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:25.810407 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fae9bf99a434b83a73451cee1c56e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.812150 29263 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.815886 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fae9bf99a434b83a73451cee1c56e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.815788 29263 ts_tablet_manager.cc:1434] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.000s
I20260709 13:03:25.807863 29264 ts_tablet_manager.cc:1434] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.000s
I20260709 13:03:25.846002 29134 tablet_service.cc:1511] Processing CreateTablet for tablet 984bea8952114e3f9c0a045e7f557e37 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:25.847137 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984bea8952114e3f9c0a045e7f557e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.847514 29262 tablet_bootstrap.cc:492] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:25.848090 29262 ts_tablet_manager.cc:1403] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.062s	user 0.021s	sys 0.022s
I20260709 13:03:25.852192 29262 raft_consensus.cc:359] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.852890 29262 raft_consensus.cc:385] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.853194 29262 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.853508 29205 tablet_service.cc:1511] Processing CreateTablet for tablet 984bea8952114e3f9c0a045e7f557e37 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:25.856130 29059 tablet_service.cc:1511] Processing CreateTablet for tablet 01d4b885cd86484fbaa6c528fc2ed1bc (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:25.857381 29059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d4b885cd86484fbaa6c528fc2ed1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.856695 29205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984bea8952114e3f9c0a045e7f557e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.857843 29137 tablet_service.cc:1511] Processing CreateTablet for tablet 01d4b885cd86484fbaa6c528fc2ed1bc (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:25.859040 29137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d4b885cd86484fbaa6c528fc2ed1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.854099 29263 tablet_bootstrap.cc:492] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:25.854550 29264 tablet_bootstrap.cc:492] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:25.854740 29056 tablet_service.cc:1511] Processing CreateTablet for tablet 984bea8952114e3f9c0a045e7f557e37 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:25.867477 29056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984bea8952114e3f9c0a045e7f557e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.867256 29207 tablet_service.cc:1511] Processing CreateTablet for tablet 01d4b885cd86484fbaa6c528fc2ed1bc (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:25.868536 29207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d4b885cd86484fbaa6c528fc2ed1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.854962 29262 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.873417 29262 ts_tablet_manager.cc:1434] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:03:25.874541 29262 tablet_bootstrap.cc:492] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:25.882038 29263 tablet_bootstrap.cc:654] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:25.890568 29264 tablet_bootstrap.cc:654] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:25.896862 29262 tablet_bootstrap.cc:654] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:25.902626 29263 tablet_bootstrap.cc:492] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:25.903131 29263 ts_tablet_manager.cc:1403] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.004s
I20260709 13:03:25.903569 29061 tablet_service.cc:1511] Processing CreateTablet for tablet 77b0625d42ef413aae4ec112d9ba450b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:25.904284 29273 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.905232 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77b0625d42ef413aae4ec112d9ba450b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.904961 29273 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.908959 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:25.910012 29269 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.910650 29269 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:25.910985 29111 leader_election.cc:343] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.912389 29273 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:25.912840 29269 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:25.913197 29273 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.913924 29273 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.915424 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:25.916781 29273 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:25.917347 29038 leader_election.cc:343] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.919476 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:25.912230 29263 raft_consensus.cc:359] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.920369 29263 raft_consensus.cc:385] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.920961 29263 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.921561 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:25.922461 29263 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:25.923687 29111 leader_election.cc:343] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:25.924439 29039 leader_election.cc:343] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:25.929807 29039 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:25.930843 29269 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:25.932533 29134 tablet_service.cc:1511] Processing CreateTablet for tablet 77b0625d42ef413aae4ec112d9ba450b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:25.934098 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77b0625d42ef413aae4ec112d9ba450b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.935489 29207 tablet_service.cc:1511] Processing CreateTablet for tablet 77b0625d42ef413aae4ec112d9ba450b (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:25.936806 29207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77b0625d42ef413aae4ec112d9ba450b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.936612 29054 tablet_service.cc:1511] Processing CreateTablet for tablet 5acfd220cbb54cd181c08050de276f65 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:25.937808 29054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5acfd220cbb54cd181c08050de276f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.950510 29263 ts_tablet_manager.cc:1434] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 13:03:25.955451 29135 tablet_service.cc:1511] Processing CreateTablet for tablet 5acfd220cbb54cd181c08050de276f65 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:25.961112 29135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5acfd220cbb54cd181c08050de276f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.966336 29263 tablet_bootstrap.cc:492] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:25.969313 29205 tablet_service.cc:1511] Processing CreateTablet for tablet 5acfd220cbb54cd181c08050de276f65 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:25.967785 29056 tablet_service.cc:1511] Processing CreateTablet for tablet 3c777774468e477ca1ab0352dff56c9c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:25.970577 29205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5acfd220cbb54cd181c08050de276f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.970697 29056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c777774468e477ca1ab0352dff56c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.974316 29138 tablet_service.cc:1511] Processing CreateTablet for tablet 3c777774468e477ca1ab0352dff56c9c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:25.975620 29138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c777774468e477ca1ab0352dff56c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.959733 29264 tablet_bootstrap.cc:492] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:25.978608 29059 tablet_service.cc:1511] Processing CreateTablet for tablet b55b583a63b64ca6832b73f813985e49 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:25.979375 29264 ts_tablet_manager.cc:1403] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.125s	user 0.012s	sys 0.002s
I20260709 13:03:25.981254 29210 tablet_service.cc:1511] Processing CreateTablet for tablet 3c777774468e477ca1ab0352dff56c9c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:25.982703 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c777774468e477ca1ab0352dff56c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.984331 29059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b55b583a63b64ca6832b73f813985e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.970669 29262 tablet_bootstrap.cc:492] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:25.985145 29262 ts_tablet_manager.cc:1403] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.111s	user 0.038s	sys 0.040s
I20260709 13:03:25.986208 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:25.986847 29221 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:25.987798 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:25.992239 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:25.993269 29292 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:25.993098 29264 raft_consensus.cc:359] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.993767 29292 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:25.994274 29292 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:25.994526 29262 raft_consensus.cc:359] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.995291 29262 raft_consensus.cc:385] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:25.995038 29132 tablet_service.cc:1511] Processing CreateTablet for tablet b55b583a63b64ca6832b73f813985e49 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:25.995580 29262 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:25.996377 29132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b55b583a63b64ca6832b73f813985e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:25.996409 29262 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.997774 29273 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:25.998818 29262 ts_tablet_manager.cc:1434] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:03:25.998376 29273 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:25.999811 29262 tablet_bootstrap.cc:492] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:25.993901 29264 raft_consensus.cc:385] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.000720 29273 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.013350 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.018759 29206 tablet_service.cc:1511] Processing CreateTablet for tablet b55b583a63b64ca6832b73f813985e49 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:26.023608 29073 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:26.024197 29206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b55b583a63b64ca6832b73f813985e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.024950 29111 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:26.025866 29273 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:26.026232 29273 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.019032 29061 tablet_service.cc:1511] Processing CreateTablet for tablet b27f1b9ed9964227ac908c2897d935e9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:26.019590 29134 tablet_service.cc:1511] Processing CreateTablet for tablet b27f1b9ed9964227ac908c2897d935e9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:26.026590 29273 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.027341 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b27f1b9ed9964227ac908c2897d935e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.014712 29264 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.028674 29264 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.027341 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b27f1b9ed9964227ac908c2897d935e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.035375 29273 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.041350 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:26.041914 29073 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.045034 29270 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.045559 29270 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.048614 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.049049 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:26.049443 29071 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:26.049582 29263 tablet_bootstrap.cc:654] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.049849 29148 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:03:26.051080 29188 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:26.047807 29270 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:26.052544 29270 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.052776 29273 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.045143 29264 ts_tablet_manager.cc:1434] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.001s
I20260709 13:03:26.051306 29073 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:26.057062 29264 tablet_bootstrap.cc:492] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:26.057016 29205 tablet_service.cc:1511] Processing CreateTablet for tablet b27f1b9ed9964227ac908c2897d935e9 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:26.057449 29111 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:26.055454 29292 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.058404 29205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b27f1b9ed9964227ac908c2897d935e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.059743 29292 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.055531 29262 tablet_bootstrap.cc:654] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:26.038342 29113 leader_election.cc:343] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.063843 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:26.060420 29273 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:26.065372 29059 tablet_service.cc:1511] Processing CreateTablet for tablet ee1e380ba6084a0e9d9978fa9983edc1 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
W20260709 13:03:26.067054 29111 leader_election.cc:343] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.068938 29134 tablet_service.cc:1511] Processing CreateTablet for tablet ee1e380ba6084a0e9d9978fa9983edc1 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:26.069869 29059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee1e380ba6084a0e9d9978fa9983edc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.070086 29207 tablet_service.cc:1511] Processing CreateTablet for tablet ee1e380ba6084a0e9d9978fa9983edc1 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:26.070720 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee1e380ba6084a0e9d9978fa9983edc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.071539 29207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee1e380ba6084a0e9d9978fa9983edc1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:26.078264 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:26.089941 29061 tablet_service.cc:1511] Processing CreateTablet for tablet ea48540c1af24bf3b55edec5c52ec133 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:26.091101 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea48540c1af24bf3b55edec5c52ec133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.092705 29135 tablet_service.cc:1511] Processing CreateTablet for tablet ea48540c1af24bf3b55edec5c52ec133 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:26.094065 29135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea48540c1af24bf3b55edec5c52ec133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.094709 29210 tablet_service.cc:1511] Processing CreateTablet for tablet ea48540c1af24bf3b55edec5c52ec133 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:26.095853 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea48540c1af24bf3b55edec5c52ec133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.100293 29056 tablet_service.cc:1511] Processing CreateTablet for tablet b9574de46fcd490e84208e41907200b0 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:26.075356 29273 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:26.101686 29056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9574de46fcd490e84208e41907200b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.101686 29273 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.107368 29137 tablet_service.cc:1511] Processing CreateTablet for tablet b9574de46fcd490e84208e41907200b0 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:26.108678 29137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9574de46fcd490e84208e41907200b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.110484 29270 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:26.065699 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:03:26.112807 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 75051.1us
I20260709 13:03:26.112867 29293 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:26.114408 29264 tablet_bootstrap.cc:654] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.114612 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:26.116811 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:26.116883 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:26.117424 29221 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.138573 29221 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
W20260709 13:03:26.128159 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 77178.8us
W20260709 13:03:26.129140 29113 leader_election.cc:343] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:26.141952 29113 leader_election.cc:343] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.142736 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:26.142552 29061 tablet_service.cc:1511] Processing CreateTablet for tablet 023c90d48e094012bc60b1e600fffa14 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:26.143620 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 023c90d48e094012bc60b1e600fffa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.146217 29207 tablet_service.cc:1511] Processing CreateTablet for tablet b9574de46fcd490e84208e41907200b0 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:26.147330 29293 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:26.147817 29293 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:26.149081 29293 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.152872 29263 tablet_bootstrap.cc:492] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:26.153373 29263 ts_tablet_manager.cc:1403] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.187s	user 0.013s	sys 0.032s
I20260709 13:03:26.158268 29135 tablet_service.cc:1511] Processing CreateTablet for tablet 023c90d48e094012bc60b1e600fffa14 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:26.159610 29135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 023c90d48e094012bc60b1e600fffa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.165465 29210 tablet_service.cc:1511] Processing CreateTablet for tablet 023c90d48e094012bc60b1e600fffa14 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:26.166416 29207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9574de46fcd490e84208e41907200b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.166965 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 023c90d48e094012bc60b1e600fffa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.167652 29061 tablet_service.cc:1511] Processing CreateTablet for tablet a57ab99b58a448de88023e7166f78ce2 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:26.168756 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a57ab99b58a448de88023e7166f78ce2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.176347 29264 tablet_bootstrap.cc:492] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:26.176254 29263 raft_consensus.cc:359] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.176944 29264 ts_tablet_manager.cc:1403] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.120s	user 0.014s	sys 0.010s
I20260709 13:03:26.177073 29263 raft_consensus.cc:385] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.177441 29263 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.181213 28969 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.182458 29263 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.184742 29263 ts_tablet_manager.cc:1434] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 13:03:26.185689 29263 tablet_bootstrap.cc:492] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:26.192699 29262 tablet_bootstrap.cc:492] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:26.193264 29262 ts_tablet_manager.cc:1403] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.194s	user 0.010s	sys 0.004s
I20260709 13:03:26.193629 29135 tablet_service.cc:1511] Processing CreateTablet for tablet a57ab99b58a448de88023e7166f78ce2 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:26.194702 29135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a57ab99b58a448de88023e7166f78ce2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.196255 29262 raft_consensus.cc:359] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.197039 29262 raft_consensus.cc:385] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.197412 29262 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.199692 29262 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.200522 29263 tablet_bootstrap.cc:654] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.201946 29262 ts_tablet_manager.cc:1434] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:03:26.202944 29262 tablet_bootstrap.cc:492] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:26.205147 29205 tablet_service.cc:1511] Processing CreateTablet for tablet a57ab99b58a448de88023e7166f78ce2 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:26.206462 29205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a57ab99b58a448de88023e7166f78ce2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.209438 29262 tablet_bootstrap.cc:654] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.215029 29056 tablet_service.cc:1511] Processing CreateTablet for tablet 2c02e27bf25948329604215b11034f44 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:26.215618 29137 tablet_service.cc:1511] Processing CreateTablet for tablet 2c02e27bf25948329604215b11034f44 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:26.216813 29056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c02e27bf25948329604215b11034f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.216872 29137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c02e27bf25948329604215b11034f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.223074 29210 tablet_service.cc:1511] Processing CreateTablet for tablet 2c02e27bf25948329604215b11034f44 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:26.224229 29210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c02e27bf25948329604215b11034f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.224221 29059 tablet_service.cc:1511] Processing CreateTablet for tablet 9a29f89bc1cb471d94fee8d14f56314c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:26.225421 29059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a29f89bc1cb471d94fee8d14f56314c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.227517 29132 tablet_service.cc:1511] Processing CreateTablet for tablet 9a29f89bc1cb471d94fee8d14f56314c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:26.228721 29132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a29f89bc1cb471d94fee8d14f56314c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.229772 29206 tablet_service.cc:1511] Processing CreateTablet for tablet 9a29f89bc1cb471d94fee8d14f56314c (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:26.230963 29206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a29f89bc1cb471d94fee8d14f56314c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.230813 29061 tablet_service.cc:1511] Processing CreateTablet for tablet d7517b9e93904801886467474f000b2d (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:26.231858 29061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7517b9e93904801886467474f000b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.245609 29263 tablet_bootstrap.cc:492] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:26.246147 29263 ts_tablet_manager.cc:1403] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.001s
I20260709 13:03:26.249135 29263 raft_consensus.cc:359] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.249588 29134 tablet_service.cc:1511] Processing CreateTablet for tablet d7517b9e93904801886467474f000b2d (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:26.250725 29134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7517b9e93904801886467474f000b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.179991 29264 raft_consensus.cc:359] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.252462 29264 raft_consensus.cc:385] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.252754 29264 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.253444 29264 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.254098 29135 tablet_service.cc:1511] Processing CreateTablet for tablet d0107c7a468e44f6b9ebd1ced2ae1ed1 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:26.255175 29135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0107c7a468e44f6b9ebd1ced2ae1ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.255656 29264 ts_tablet_manager.cc:1434] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.078s	user 0.004s	sys 0.000s
I20260709 13:03:26.255518 29203 tablet_service.cc:1511] Processing CreateTablet for tablet d7517b9e93904801886467474f000b2d (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:26.257783 29203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7517b9e93904801886467474f000b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.263202 29204 tablet_service.cc:1511] Processing CreateTablet for tablet d0107c7a468e44f6b9ebd1ced2ae1ed1 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:26.264448 29204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0107c7a468e44f6b9ebd1ced2ae1ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.249840 29263 raft_consensus.cc:385] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.272578 29263 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.273366 29263 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.274475 29059 tablet_service.cc:1511] Processing CreateTablet for tablet d0107c7a468e44f6b9ebd1ced2ae1ed1 (DEFAULT_TABLE table=test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:26.287844 29059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0107c7a468e44f6b9ebd1ced2ae1ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.293702 29264 tablet_bootstrap.cc:492] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:26.294775 29269 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.295286 29269 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.297776 29269 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.299973 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:26.300632 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:26.300634 29221 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
W20260709 13:03:26.302419 29039 leader_election.cc:343] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.302953 29038 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:26.304260 29269 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:26.304647 29269 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.304975 29269 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.310045 29264 tablet_bootstrap.cc:654] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.311054 29263 ts_tablet_manager.cc:1434] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.000s
I20260709 13:03:26.312155 29263 tablet_bootstrap.cc:492] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:26.314733 29262 tablet_bootstrap.cc:492] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:26.315227 29262 ts_tablet_manager.cc:1403] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.112s	user 0.021s	sys 0.020s
I20260709 13:03:26.316695 29269 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.318228 29262 raft_consensus.cc:359] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.318928 29262 raft_consensus.cc:385] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.319025 29263 tablet_bootstrap.cc:654] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.319269 29262 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.319473 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:26.320196 29262 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.321127 29148 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.321940 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:26.322443 29262 ts_tablet_manager.cc:1434] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:03:26.322489 29221 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.323575 29262 tablet_bootstrap.cc:492] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:26.327147 28971 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.328215 29148 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:26.329656 29221 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:26.329957 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:26.330295 29262 tablet_bootstrap.cc:654] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.320624 29269 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.332229 29269 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:26.332633 29269 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:26.333439 29269 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.337455 29264 tablet_bootstrap.cc:492] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:26.338114 29264 ts_tablet_manager.cc:1403] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.045s	user 0.005s	sys 0.009s
I20260709 13:03:26.342195 29264 raft_consensus.cc:359] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.343164 29264 raft_consensus.cc:385] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.343674 29264 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.344823 29264 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.348316 29264 ts_tablet_manager.cc:1434] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.005s
I20260709 13:03:26.349694 29264 tablet_bootstrap.cc:492] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:26.356071 29294 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.356776 29294 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.357600 29264 tablet_bootstrap.cc:654] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.360610 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.361249 29071 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:26.362504 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:26.363349 29270 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:26.363726 29270 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.364078 29270 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.364207 29294 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:26.369400 29273 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.370409 29264 tablet_bootstrap.cc:492] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:26.370036 29273 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.370987 29264 ts_tablet_manager.cc:1403] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.005s
I20260709 13:03:26.371083 29294 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.371609 29294 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.372252 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:26.372877 29148 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:26.373642 29294 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:26.374969 29264 raft_consensus.cc:359] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.375836 29264 raft_consensus.cc:385] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.376204 29264 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.376979 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:26.377074 29264 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.378540 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.378825 29262 tablet_bootstrap.cc:492] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:26.379336 29262 ts_tablet_manager.cc:1403] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.056s	user 0.033s	sys 0.014s
I20260709 13:03:26.379541 29264 ts_tablet_manager.cc:1434] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 13:03:26.380520 29264 tablet_bootstrap.cc:492] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:26.380458 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.381098 29073 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:26.382396 29262 raft_consensus.cc:359] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.383144 29262 raft_consensus.cc:385] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.383535 29262 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.384311 29262 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.384709 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:26.386514 29262 ts_tablet_manager.cc:1434] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:26.387698 29262 tablet_bootstrap.cc:492] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:26.388749 29264 tablet_bootstrap.cc:654] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:26.387562 29113 leader_election.cc:343] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.394119 29262 tablet_bootstrap.cc:654] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.401249 29264 tablet_bootstrap.cc:492] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:26.379231 29071 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:26.401713 29273 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.402815 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:26.404486 29273 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:26.404902 29273 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.405295 29273 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.406920 29270 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.407730 29262 tablet_bootstrap.cc:492] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:26.382439 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:26.408278 29262 ts_tablet_manager.cc:1403] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 13:03:26.408782 29294 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:26.409176 29294 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.409507 29294 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.411505 29262 raft_consensus.cc:359] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:26.412016 29188 leader_election.cc:343] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.412266 29262 raft_consensus.cc:385] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.412657 29262 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.413501 29262 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.416813 29262 ts_tablet_manager.cc:1434] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:03:26.417781 29262 tablet_bootstrap.cc:492] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:26.418650 29264 ts_tablet_manager.cc:1403] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.008s
I20260709 13:03:26.422125 29264 raft_consensus.cc:359] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.423017 29264 raft_consensus.cc:385] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.423358 29264 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.423688 29262 tablet_bootstrap.cc:654] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.425220 29294 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.428321 29264 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.428665 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:26.429651 29071 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.430858 29264 ts_tablet_manager.cc:1434] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 13:03:26.431802 29264 tablet_bootstrap.cc:492] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:26.434581 28971 catalog_manager.cc:5697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.439203 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:26.439867 29148 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.415102 29273 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.443375 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:26.444316 29073 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.445156 29262 tablet_bootstrap.cc:492] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:26.445663 29262 ts_tablet_manager.cc:1403] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.028s	user 0.007s	sys 0.009s
I20260709 13:03:26.447381 29148 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:26.448431 29262 raft_consensus.cc:359] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.449268 29262 raft_consensus.cc:385] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.449574 29262 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.449893 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:26.450434 29072 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.450374 29262 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.452849 29262 ts_tablet_manager.cc:1434] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:26.443360 29264 tablet_bootstrap.cc:654] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.456203 29262 tablet_bootstrap.cc:492] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:26.459089 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:26.457302 29294 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:26.463526 29072 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:26.466814 29270 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:26.465620 29294 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:26.467478 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:26.466339 29071 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:26.472150 29293 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.473248 29293 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.473904 29270 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:26.474366 29270 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:26.468029 29294 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:26.474987 29263 tablet_bootstrap.cc:492] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:26.475433 29263 ts_tablet_manager.cc:1403] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.164s	user 0.035s	sys 0.061s
I20260709 13:03:26.475536 29293 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.477080 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:26.478461 29262 tablet_bootstrap.cc:654] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:26.478806 29113 leader_election.cc:343] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.478490 29263 raft_consensus.cc:359] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.479822 29263 raft_consensus.cc:385] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.480250 29263 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.480813 29273 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.480902 29263 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.480958 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.483356 29294 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.483954 29072 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:26.485558 29111 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:26.488242 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:26.489933 29273 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:26.490247 29263 ts_tablet_manager.cc:1434] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.003s
I20260709 13:03:26.491343 29263 tablet_bootstrap.cc:492] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:26.490311 29221 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.492702 29273 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.494102 29273 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.496284 29264 tablet_bootstrap.cc:492] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:26.495409 29270 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.496752 29264 ts_tablet_manager.cc:1403] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.065s	user 0.009s	sys 0.006s
I20260709 13:03:26.496558 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:26.497303 29148 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.499838 29264 raft_consensus.cc:359] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.500604 29264 raft_consensus.cc:385] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.500891 29264 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.501368 29263 tablet_bootstrap.cc:654] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.501672 29264 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.503892 29264 ts_tablet_manager.cc:1434] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:26.504954 29264 tablet_bootstrap.cc:492] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:26.503988 29221 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:26.508936 29113 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:26.510193 29293 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:26.511381 29293 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:26.512243 29293 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.517339 29262 tablet_bootstrap.cc:492] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:26.518036 29262 ts_tablet_manager.cc:1403] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.062s	user 0.012s	sys 0.009s
I20260709 13:03:26.514170 29264 tablet_bootstrap.cc:654] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.528916 28969 catalog_manager.cc:5697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:26.518424 29073 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:26.533121 29262 raft_consensus.cc:359] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.533821 29262 raft_consensus.cc:385] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.534066 29262 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.534689 29262 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.536477 29148 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:26.536677 29262 ts_tablet_manager.cc:1434] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:03:26.537571 29262 tablet_bootstrap.cc:492] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:26.535768 29273 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.539492 29273 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.544785 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:26.547646 29262 tablet_bootstrap.cc:654] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.549465 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:26.550072 29073 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.555179 29072 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.557898 29262 tablet_bootstrap.cc:492] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:26.558365 29262 ts_tablet_manager.cc:1403] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.003s
I20260709 13:03:26.557168 28969 catalog_manager.cc:5697] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:26.558451 29113 leader_election.cc:343] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.561153 29262 raft_consensus.cc:359] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.561836 29262 raft_consensus.cc:385] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.562114 29262 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.565964 29264 tablet_bootstrap.cc:492] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:26.566538 29264 ts_tablet_manager.cc:1403] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.000s
I20260709 13:03:26.569412 29264 raft_consensus.cc:359] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.570072 29264 raft_consensus.cc:385] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.570336 29264 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.570976 29264 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.573099 29264 ts_tablet_manager.cc:1434] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:03:26.573625 29262 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.575876 29262 ts_tablet_manager.cc:1434] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:03:26.576854 29262 tablet_bootstrap.cc:492] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:26.577901 29264 tablet_bootstrap.cc:492] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:26.576930 29073 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:26.581313 29111 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:26.582446 29273 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:26.582899 28971 catalog_manager.cc:5697] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.583829 29273 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:26.584357 29273 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:26.585260 29273 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.586472 29264 tablet_bootstrap.cc:654] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.594120 29262 tablet_bootstrap.cc:654] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.604174 29263 tablet_bootstrap.cc:492] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:26.604537 29262 tablet_bootstrap.cc:492] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:26.604681 29263 ts_tablet_manager.cc:1403] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.114s	user 0.022s	sys 0.030s
I20260709 13:03:26.605003 29262 ts_tablet_manager.cc:1403] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.003s
I20260709 13:03:26.607744 29262 raft_consensus.cc:359] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.608423 29262 raft_consensus.cc:385] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.608713 29262 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.609406 29262 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.611323 29262 ts_tablet_manager.cc:1434] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:03:26.611622 29263 raft_consensus.cc:359] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.612229 29262 tablet_bootstrap.cc:492] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:26.612390 29263 raft_consensus.cc:385] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.612792 29263 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.612950 29264 tablet_bootstrap.cc:492] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:26.612349 28969 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:26.613391 29264 ts_tablet_manager.cc:1403] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.012s
I20260709 13:03:26.613623 29263 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.616288 29264 raft_consensus.cc:359] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.616681 29307 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.617002 29264 raft_consensus.cc:385] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.617291 29264 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.617221 29307 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.618144 29264 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.618909 29262 tablet_bootstrap.cc:654] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.619550 29307 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.620815 29263 ts_tablet_manager.cc:1434] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:03:26.621240 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:26.621770 29263 tablet_bootstrap.cc:492] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:26.621943 29148 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:26.623827 29039 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:26.624967 29269 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:26.625062 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:26.625526 29269 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.625852 29221 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:26.626008 29269 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.634060 29263 tablet_bootstrap.cc:654] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.641376 29264 ts_tablet_manager.cc:1434] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 13:03:26.642413 29264 tablet_bootstrap.cc:492] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:26.643401 29263 tablet_bootstrap.cc:492] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:26.644052 29263 ts_tablet_manager.cc:1403] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.000s
I20260709 13:03:26.646589 29221 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.646685 29263 raft_consensus.cc:359] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.647501 29263 raft_consensus.cc:385] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.647804 29263 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.648898 29264 tablet_bootstrap.cc:654] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.648567 29263 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.653885 29263 ts_tablet_manager.cc:1434] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:03:26.655094 29263 tablet_bootstrap.cc:492] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:26.661442 29262 tablet_bootstrap.cc:492] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:26.661924 29262 ts_tablet_manager.cc:1403] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.050s	user 0.020s	sys 0.020s
I20260709 13:03:26.664947 29262 raft_consensus.cc:359] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.665663 29262 raft_consensus.cc:385] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.665939 29262 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.666591 29262 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.667479 29264 tablet_bootstrap.cc:492] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:26.667948 29264 ts_tablet_manager.cc:1403] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.026s	user 0.003s	sys 0.007s
I20260709 13:03:26.669875 29263 tablet_bootstrap.cc:654] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.670433 29264 raft_consensus.cc:359] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.671164 29264 raft_consensus.cc:385] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.671423 29264 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.672602 29264 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.671778 29262 ts_tablet_manager.cc:1434] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:03:26.674471 29262 tablet_bootstrap.cc:492] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:26.675024 29264 ts_tablet_manager.cc:1434] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:03:26.675904 29264 tablet_bootstrap.cc:492] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:26.680738 29262 tablet_bootstrap.cc:654] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.681507 29264 tablet_bootstrap.cc:654] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.689145 29269 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.690474 29270 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.690966 29270 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.693286 29292 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:26.694439 29263 tablet_bootstrap.cc:492] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:26.694876 29263 ts_tablet_manager.cc:1403] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.004s
I20260709 13:03:26.695783 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.696442 29073 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:26.696831 29270 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:26.697803 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:26.698040 29263 raft_consensus.cc:359] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.698860 29263 raft_consensus.cc:385] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.699234 29263 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.699194 29292 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:26.700225 29263 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.700765 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:26.703265 29262 tablet_bootstrap.cc:492] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:26.703856 29262 ts_tablet_manager.cc:1403] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.002s
I20260709 13:03:26.707335 29262 raft_consensus.cc:359] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.708227 29262 raft_consensus.cc:385] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.708564 29262 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.710038 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:26.710599 29148 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.711851 29263 ts_tablet_manager.cc:1434] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 13:03:26.713029 29263 tablet_bootstrap.cc:492] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:26.718506 29148 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:26.719770 29039 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:26.720865 29307 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:26.721244 29263 tablet_bootstrap.cc:654] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.721357 29307 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:26.722537 29262 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.723817 29307 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.724495 29332 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.725387 29273 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:26.726194 29332 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.722051 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:26.730937 29222 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.729209 29262 ts_tablet_manager.cc:1434] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.001s
I20260709 13:03:26.732896 29262 tablet_bootstrap.cc:492] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:26.724967 29270 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.724890 29269 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.731389 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:26.730928 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:26.740201 29273 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.740711 29273 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.741073 29270 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:26.741911 29038 leader_election.cc:343] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.743058 29270 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:03:26.745647 29039 leader_election.cc:343] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.745815 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.746116 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:26.729055 29332 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.749971 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:26.751668 29148 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
W20260709 13:03:26.752549 29185 leader_election.cc:343] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.753273 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:26.753486 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.754056 29270 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:26.754417 29270 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.754730 29270 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:26.755882 29111 leader_election.cc:343] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.756767 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:26.757409 29273 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.757632 29332 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:26.773685 29341 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:03:26.787371 29262 tablet_bootstrap.cc:654] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:26.789909 29270 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.792184 29219 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:26.794525 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:26.795606 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:26.796780 29293 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:26.800343 29185 leader_election.cc:343] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.805857 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:26.806879 29270 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:26.800385 29222 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:26.836319 29273 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.837085 29273 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.841753 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.847947 29264 tablet_bootstrap.cc:492] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:26.848618 29264 ts_tablet_manager.cc:1403] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.173s	user 0.035s	sys 0.040s
I20260709 13:03:26.852370 29273 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.857203 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:26.850497 29293 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:26.849818 29111 leader_election.cc:343] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.858687 29270 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.860105 29270 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.862195 29270 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:26.865389 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:26.867127 29185 leader_election.cc:343] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.871176 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:26.872102 29146 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
W20260709 13:03:26.870783 29113 leader_election.cc:343] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:26.892105 29294 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:26.893608 29294 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:26.894145 29294 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:26.915170 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:26.923205 29315 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:26.923820 29315 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.924299 29315 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.940460 29293 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.940999 29293 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.926433 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:26.943081 29292 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.943291 29273 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:26.943557 29292 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.948761 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.944962 29273 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:03:26.948709 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.954708 29071 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:26.955047 29073 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:26.956907 29263 tablet_bootstrap.cc:492] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:26.958981 29307 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:26.952088 29270 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:03:26.949082 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 82934.3us
I20260709 13:03:26.956663 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:26.952913 29264 raft_consensus.cc:359] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.961796 29264 raft_consensus.cc:385] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:26.962005 29264 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:26.946002 29292 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.962743 29264 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.962325 29317 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:03:26.970162 29147 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:26.943092 29293 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:26.959005 29263 ts_tablet_manager.cc:1403] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.246s	user 0.025s	sys 0.029s
I20260709 13:03:26.971743 29273 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:26.980885 29264 ts_tablet_manager.cc:1434] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.132s	user 0.007s	sys 0.001s
I20260709 13:03:26.976440 29270 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:26.982127 29264 tablet_bootstrap.cc:492] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:26.981853 29270 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.982704 29315 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:26.989375 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:26.989970 29071 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:26.985934 29273 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.991781 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:26.992542 29317 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:26.992645 29273 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.992947 29317 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:26.993355 29317 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:26.987600 29270 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:27.000432 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:27.001199 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:27.001528 29263 raft_consensus.cc:359] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.003720 29263 raft_consensus.cc:385] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:27.004626 29263 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:27.001895 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:27.006115 29263 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.003798 28970 catalog_manager.cc:5697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:27.010923 29307 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.011817 29307 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.014969 29315 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:27.015249 29263 ts_tablet_manager.cc:1434] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.001s
I20260709 13:03:27.014026 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:27.016820 29263 tablet_bootstrap.cc:492] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:27.021111 29307 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:27.021713 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:27.025135 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:03:27.028295 29113 leader_election.cc:343] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:27.028996 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:27.028632 29270 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.029598 29270 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.028388 29315 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:03:27.031296 29039 leader_election.cc:343] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:27.032747 29185 leader_election.cc:343] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:27.022442 29293 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:03:27.038712 29262 tablet_bootstrap.cc:492] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:27.250218 29262 ts_tablet_manager.cc:1403] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.518s	user 0.043s	sys 0.069s
I20260709 13:03:27.252234 29262 raft_consensus.cc:359] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.252766 29262 raft_consensus.cc:385] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:27.253006 29262 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:27.253535 29262 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.254915 29262 ts_tablet_manager.cc:1434] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:03:27.255751 29262 tablet_bootstrap.cc:492] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:27.054869 29273 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:27.264129 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:27.069571 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:27.264954 29145 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:27.265228 29273 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:27.270583 29317 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.245144 29352 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.239s
I20260709 13:03:27.272083 29220 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:27.166985 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:27.280259 29222 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:27.119076 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:27.278394 29264 tablet_bootstrap.cc:654] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:27.055220 29349 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:27.281989 29349 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.282471 29349 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:27.282415 29072 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:27.102442 29315 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:27.287729 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:27.246901 29348 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.187s
I20260709 13:03:27.273396 29270 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:27.293787 29315 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.302449 29270 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:27.305271 29270 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.306061 29317 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:27.305650 29270 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:27.308765 29348 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.309396 29348 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.309340 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:27.318439 29073 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:27.320998 29263 tablet_bootstrap.cc:654] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:27.332022 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:27.332653 29071 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:27.340397 29071 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:27.347154 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:27.351187 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:03:27.351483 29188 leader_election.cc:343] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:27.351519 29341 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:27.355445 29145 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:27.327121 29294 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.359407 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:27.363314 29262 tablet_bootstrap.cc:654] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:27.365537 29219 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:27.385401 29360 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:27.386173 29360 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:27.387271 29360 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.394656 29264 tablet_bootstrap.cc:492] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:27.395835 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:27.397024 29264 ts_tablet_manager.cc:1403] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.415s	user 0.028s	sys 0.008s
I20260709 13:03:27.396097 29348 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:27.407512 29264 raft_consensus.cc:359] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.410844 29264 raft_consensus.cc:385] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:27.411314 29264 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:27.409403 29315 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:03:27.412021 29264 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.414116 29264 ts_tablet_manager.cc:1434] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.000s
I20260709 13:03:27.408360 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:27.408957 29317 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:27.415118 29264 tablet_bootstrap.cc:492] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:27.408682 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:27.407838 29349 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.435227 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:27.438089 29072 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:27.439137 29349 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:27.428370 29317 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:27.433804 29073 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:27.434734 29307 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:27.432278 29332 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.446599 29148 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:03:27.410043 29359 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.039s
I20260709 13:03:27.459057 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:27.453799 29307 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.466442 29292 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:27.466863 29292 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.467209 29292 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:27.460821 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:27.464205 29307 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:27.456741 29332 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.473555 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:27.469484 29294 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:27.461639 29270 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.494308 29270 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:27.498519 29332 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:27.498414 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:27.477636 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:03:27.481334 29113 leader_election.cc:343] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:27.486685 29293 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.497364 29349 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:27.499893 29072 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:27.506398 29269 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:03:27.515259 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:27.477381 29317 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.498448 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:27.507663 29349 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:27.516671 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:27.524343 29263 tablet_bootstrap.cc:492] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:27.524922 29263 ts_tablet_manager.cc:1403] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.508s	user 0.057s	sys 0.090s
I20260709 13:03:27.506875 29293 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.533962 29293 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:27.534670 29188 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:03:27.533382 29113 consensus_peers.cc:597] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:27.536777 29113 leader_election.cc:343] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:27.537422 29262 tablet_bootstrap.cc:492] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:27.537930 29262 ts_tablet_manager.cc:1403] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.282s	user 0.022s	sys 0.001s
I20260709 13:03:27.540009 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:27.540957 29222 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:27.541285 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:27.541930 29073 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:27.542371 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:27.543197 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:27.545388 29294 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:27.543823 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:27.546777 29357 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:27.548282 29357 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:27.550580 29147 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:27.555495 29113 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:27.549068 29357 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.567294 29264 tablet_bootstrap.cc:654] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:27.560462 29293 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:27.568087 29293 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:27.569417 29293 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:27.574119 29262 raft_consensus.cc:359] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.575084 29262 raft_consensus.cc:385] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:27.575439 29262 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:27.582170 29263 raft_consensus.cc:359] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.582886 29263 raft_consensus.cc:385] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:27.583411 29263 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:27.579288 29307 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.582679 29293 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.590602 29292 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.604535 29262 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.564502 29349 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.671797 29359 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:27.764891 29359 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:27.766777 29293 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.193473 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:28.194376 29220 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:28.286537 29263 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.293725 29262 ts_tablet_manager.cc:1434] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.755s	user 0.024s	sys 0.016s
W20260709 13:03:28.294167 29262 ts_tablet_manager.cc:1467] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2: Tablet startup took 755ms
W20260709 13:03:28.132524 29317 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.000s	sys 0.022s
W20260709 13:03:28.294509 29262 ts_tablet_manager.cc:1469] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2: Trace:
0709 13:03:24.945929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.955959 (+ 10030us) service_pool.cc:224] Handling call
0709 13:03:24.957809 (+  1850us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:03:24.957902 (+    93us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:03:25.022042 (+ 64140us) tablet_metadata.cc:665] Metadata flushed
0709 13:03:25.070398 (+ 48356us) tablet_replica.cc:206] Creating consensus instance
0709 13:03:25.070867 (+   469us) inbound_call.cc:177] Queueing success response
0709 13:03:27.255529 (+2184662us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:03:27.538207 (+282678us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:03:27.545552 (+  7345us) tablet_replica.cc:261] Starting instrumentation
0709 13:03:27.546070 (+   518us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":218,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":35676,"tablet-open.queue_time_us":2184760}
I20260709 13:03:28.341444 29349 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:28.363719 29262 tablet_bootstrap.cc:492] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:28.436834 29220 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:03:28.427098 29307 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.425437 29292 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:28.444803 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 133673us
I20260709 13:03:28.473017 29263 ts_tablet_manager.cc:1434] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.948s	user 0.005s	sys 0.019s
W20260709 13:03:28.473459 29263 ts_tablet_manager.cc:1467] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Tablet startup took 948ms
W20260709 13:03:28.473790 29263 ts_tablet_manager.cc:1469] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Trace:
0709 13:03:24.993978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:24.994140 (+   162us) service_pool.cc:224] Handling call
0709 13:03:24.996213 (+  2073us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:03:24.996317 (+   104us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:03:25.015745 (+ 19428us) tablet_metadata.cc:665] Metadata flushed
0709 13:03:25.043971 (+ 28226us) tablet_replica.cc:206] Creating consensus instance
0709 13:03:25.044339 (+   368us) inbound_call.cc:177] Queueing success response
0709 13:03:27.016510 (+1972171us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:03:27.525251 (+508741us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:03:27.580373 (+ 55122us) tablet_replica.cc:261] Starting instrumentation
0709 13:03:27.580852 (+   479us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":152606,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":141,"spinlock_wait_cycles":1344384,"tablet-open.queue_time_us":1972218,"thread_start_us":125184,"threads_started":1}
I20260709 13:03:28.486543 29263 tablet_bootstrap.cc:492] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:28.488095 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:28.494073 29220 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:28.495030 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:28.497407 29264 tablet_bootstrap.cc:492] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:28.497900 29264 ts_tablet_manager.cc:1403] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 1.083s	user 0.010s	sys 0.025s
I20260709 13:03:28.500043 29113 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:28.502562 29357 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:28.503929 29357 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:28.506569 29357 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.510257 29264 raft_consensus.cc:359] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.510950 29264 raft_consensus.cc:385] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:28.511216 29264 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:28.511772 29264 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.516801 29262 tablet_bootstrap.cc:654] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:28.517426 29068 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:03:28.504786 29113 leader_election.cc:343] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:28.519418 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 21) took 1007 ms. Trace:
I20260709 13:03:28.520828 29068 rpcz_store.cc:276] 0709 13:03:27.512534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.512674 (+   140us) service_pool.cc:224] Handling call
0709 13:03:28.519224 (+1006550us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400129654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:03:28.519359 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2302288640}
I20260709 13:03:28.525082 29264 ts_tablet_manager.cc:1434] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:03:28.526087 29264 tablet_bootstrap.cc:492] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:28.526163 29263 tablet_bootstrap.cc:654] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:28.534103 29264 tablet_bootstrap.cc:654] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:28.526057 28968 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 22) took 1177 ms. Trace:
I20260709 13:03:28.536576 28968 rpcz_store.cc:276] 0709 13:03:27.348634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.348806 (+   172us) service_pool.cc:224] Handling call
0709 13:03:28.525986 (+1177180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38344,"reported_tablets":5}
I20260709 13:03:28.545986 29263 tablet_bootstrap.cc:492] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:28.546526 29263 ts_tablet_manager.cc:1403] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.060s	user 0.012s	sys 0.009s
I20260709 13:03:28.549633 29263 raft_consensus.cc:359] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.552906 29146 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:28.553328 29264 tablet_bootstrap.cc:492] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:28.553777 29264 ts_tablet_manager.cc:1403] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.007s
I20260709 13:03:28.550307 29263 raft_consensus.cc:385] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:28.554631 29263 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:28.555483 29263 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.558328 29262 tablet_bootstrap.cc:492] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:28.558853 29262 ts_tablet_manager.cc:1403] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.197s	user 0.019s	sys 0.006s
I20260709 13:03:28.559290 29263 ts_tablet_manager.cc:1434] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:03:28.560312 29263 tablet_bootstrap.cc:492] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:28.563083 29262 raft_consensus.cc:359] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.567433 29262 raft_consensus.cc:385] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:28.567720 29262 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:28.568463 29262 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.570722 29262 ts_tablet_manager.cc:1434] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:03:28.571619 29262 tablet_bootstrap.cc:492] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:28.572404 29221 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:28.576368 29146 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:28.576964 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 22) took 1036 ms. Trace:
I20260709 13:03:28.577208 29264 raft_consensus.cc:359] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.577268 29146 rpcz_store.cc:276] 0709 13:03:27.540150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.542013 (+  1863us) service_pool.cc:224] Handling call
0709 13:03:28.552726 (+1010713us) spinlock_profiling.cc:243] Waited 8.88 ms on lock 0x7b54000ea028. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8ea924 00007f783bad3227 00007f783bad2ab8 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6
0709 13:03:28.576883 (+ 24157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20423552}
I20260709 13:03:28.577963 29264 raft_consensus.cc:385] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:28.578245 29264 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:28.578790 29264 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.586333 29263 tablet_bootstrap.cc:654] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:28.591284 29264 ts_tablet_manager.cc:1434] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:03:28.595068 29262 tablet_bootstrap.cc:654] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:28.643563 29389 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.650792 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 19) took 1191 ms. Trace:
I20260709 13:03:28.691326 29263 tablet_bootstrap.cc:492] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:28.704838 29390 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.705507 29357 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.724318 29377 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:28.725945 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:28.736936 29073 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:28.745266 29264 tablet_bootstrap.cc:492] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:28.750604 28970 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 22) took 1416 ms. Trace:
I20260709 13:03:28.753086 29359 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.753818 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:03:28.756574 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:28.841570 29221 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:28.844373 29389 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.848028 29068 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:28.848115 29390 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.850939 29390 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.762049 29071 rpcz_store.cc:276] 0709 13:03:27.464436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.469266 (+  4830us) service_pool.cc:224] Handling call
0709 13:03:27.488643 (+ 19377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:27.504577 (+ 15934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:27.506797 (+  2220us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:27.518845 (+ 12048us) log.cc:844] Serialized 26 byte log entry
0709 13:03:27.553243 (+ 34398us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:03:27.553947 (+   704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:27.556635 (+  2688us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:28.650448 (+1093813us) raft_consensus.cc:1685] finished
0709 13:03:28.650475 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:28.650705 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1360,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":8513,"thread_start_us":32431,"threads_started":1,"wal-append.queue_time_us":362746}
I20260709 13:03:28.852072 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:28.771148 29262 tablet_bootstrap.cc:492] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:28.853358 29356 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.810752 29393 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:28.853978 29356 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:28.854883 29038 leader_election.cc:343] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:28.854616 29393 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.854615 29293 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:28.856392 29356 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:28.858346 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:28.858990 29071 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:28.862664 29185 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:28.853979 29262 ts_tablet_manager.cc:1403] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.283s	user 0.054s	sys 0.020s
I20260709 13:03:28.866508 29262 raft_consensus.cc:359] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.870661 29262 raft_consensus.cc:385] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:28.868716 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:28.870860 29393 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.871589 29221 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 2: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:28.763056 29263 ts_tablet_manager.cc:1403] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.203s	user 0.014s	sys 0.005s
I20260709 13:03:28.872397 29382 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:28.874320 29382 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:28.874703 29382 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:28.875415 29263 raft_consensus.cc:359] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.876206 29263 raft_consensus.cc:385] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:28.876498 29263 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:28.877198 29263 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.878103 29262 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:28.878827 29262 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.879420 29219 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:28.881000 29262 ts_tablet_manager.cc:1434] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:03:28.882110 29262 tablet_bootstrap.cc:492] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:28.884516 29389 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.814390 29392 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:28.885475 28970 rpcz_store.cc:276] 0709 13:03:27.333823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:27.334653 (+   830us) service_pool.cc:224] Handling call
0709 13:03:28.750537 (+1415884us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":988304,"reported_tablets":4}
I20260709 13:03:28.886312 29392 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:28.887820 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 1.33278e+06us
I20260709 13:03:28.888109 29292 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:28.888868 29357 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.889420 29392 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.891693 29357 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.891834 29263 ts_tablet_manager.cc:1434] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:03:28.904055 29263 tablet_bootstrap.cc:492] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:28.904277 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:28.795555 29307 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.908385 29307 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.910367 29263 tablet_bootstrap.cc:654] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:28.911592 29382 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.762594 28968 catalog_manager.cc:5697] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:28.896759 29262 tablet_bootstrap.cc:654] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:28.898586 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:28.898907 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:28.912899 29222 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:28.914124 29292 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:28.914193 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:28.914726 29292 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.915089 29349 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:28.915356 29389 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:28.915441 29349 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:28.915988 29349 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:28.916989 29292 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.918757 28968 catalog_manager.cc:5697] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 2, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:28.921787 29307 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.915990 29389 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.923228 29349 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.924773 29389 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.927415 29349 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.932476 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:28.932928 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:28.933914 29039 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:28.935252 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:03:28.936654 29038 leader_election.cc:343] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:28.936735 29218 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:28.938026 29389 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:28.929591 29220 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:28.942621 29349 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:28.943871 29377 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:03:28.959659 29262 tablet_bootstrap.cc:492] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:28.956542 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:28.960052 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:28.960280 29262 ts_tablet_manager.cc:1403] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.078s	user 0.011s	sys 0.001s
I20260709 13:03:28.960624 29069 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:28.966995 29389 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:28.967023 29357 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:28.969148 29389 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.969600 29357 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.969777 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:28.970909 29068 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:28.971267 29389 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.971602 29357 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.979809 29262 raft_consensus.cc:359] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.980618 29262 raft_consensus.cc:385] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:28.980882 29262 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:28.982369 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:28.982926 29070 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:28.981501 29262 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.986668 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:28.986754 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:28.988528 29068 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:28.988688 29113 leader_election.cc:343] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:28.976055 29359 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:28.991777 29359 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:28.994132 29292 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:28.994608 29292 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.995020 29069 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:03:28.996405 29292 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:28.996275 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:28.997372 29359 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:28.998826 29359 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:28.999488 29264 tablet_bootstrap.cc:654] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.003675 29262 ts_tablet_manager.cc:1434] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 13:03:28.985937 29382 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:29.004725 29262 tablet_bootstrap.cc:492] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:28.999871 29068 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:03:29.007246 29188 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.008373 29377 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:29.008829 29377 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:29.009266 29263 tablet_bootstrap.cc:492] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:29.009495 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:29.010007 29263 ts_tablet_manager.cc:1403] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.106s	user 0.024s	sys 0.016s
I20260709 13:03:29.011746 29185 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:29.013818 29355 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:29.013849 29263 raft_consensus.cc:359] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.014366 29355 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:29.014647 29263 raft_consensus.cc:385] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.014967 29263 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.015637 29263 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.015306 29355 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.018766 29357 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:03:29.019767 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.019811 29357 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.021070 29263 ts_tablet_manager.cc:1434] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:03:29.019343 29293 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.023115 29293 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.023586 29357 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.020556 29068 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:29.025249 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:29.025318 29293 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.026551 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:29.031869 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:29.032661 29146 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:29.036653 29263 tablet_bootstrap.cc:492] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:29.036746 29218 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:29.037925 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:29.038738 29293 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:29.039202 29293 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:29.040555 29293 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.044149 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.050619 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.027827 29070 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:29.052721 29292 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:03:29.061472 29292 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.009780 29377 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.051503 29145 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:03:29.076098 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.109155 29144 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:29.105741 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.111716 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:29.114487 29292 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.121536 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.121857 29219 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:29.124539 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.124881 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.128806 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.130507 29039 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:29.132189 29389 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:29.132469 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:29.133061 29070 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.133004 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.134636 29068 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:29.135085 29357 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:29.135713 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.137025 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.116650 29293 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.134487 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.143363 29264 tablet_bootstrap.cc:492] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:29.143810 29264 ts_tablet_manager.cc:1403] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.399s	user 0.024s	sys 0.027s
I20260709 13:03:29.121052 29382 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:29.142545 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.147593 29264 raft_consensus.cc:359] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.148315 29377 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.150347 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:29.152704 29264 raft_consensus.cc:385] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.153177 29377 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.153514 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:29.154500 29264 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.157481 29263 tablet_bootstrap.cc:654] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.157454 29264 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.161566 29264 ts_tablet_manager.cc:1434] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:03:29.162556 29264 tablet_bootstrap.cc:492] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:29.169056 29039 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:29.170226 29392 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:29.170635 29392 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:29.171020 29392 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:29.172626 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.173292 29111 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.174741 29408 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:29.174831 29355 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:29.176011 29070 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:29.175854 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:29.176273 29270 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:29.177742 29307 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:29.178181 29389 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:29.179404 29039 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:29.180207 29307 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:29.181167 29039 leader_election.cc:343] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.181533 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:29.182240 29307 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:29.182689 29039 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:29.183422 29307 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:29.140941 29293 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.188175 29262 tablet_bootstrap.cc:654] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.185540 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:29.191367 29185 leader_election.cc:343] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.185115 29377 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:29.157892 29382 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.183862 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:29.194016 29146 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:03:29.194432 29307 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:29.197827 29382 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:29.197000 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.193161 29392 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.202006 29392 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.202853 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:29.203441 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:29.204291 29220 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 2: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:29.206421 29408 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:29.208123 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.209092 29359 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:29.209206 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:29.209496 29111 leader_election.cc:343] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.207665 29408 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:29.210081 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.211207 29359 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:29.210506 29263 tablet_bootstrap.cc:492] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:29.211941 29263 ts_tablet_manager.cc:1403] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.176s	user 0.003s	sys 0.012s
I20260709 13:03:29.206427 29293 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.211212 29408 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.214079 29111 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.218511 29293 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:29.219581 29263 raft_consensus.cc:359] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.220464 29263 raft_consensus.cc:385] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.220796 29263 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.221467 29263 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.221935 29264 tablet_bootstrap.cc:654] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.223644 29263 ts_tablet_manager.cc:1434] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:03:29.224571 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.225411 29293 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:29.224566 29263 tablet_bootstrap.cc:492] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:29.225857 29111 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.230098 29407 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:29.230926 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.231807 28971 catalog_manager.cc:5697] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:29.233021 28971 catalog_manager.cc:5697] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:29.234237 28971 catalog_manager.cc:5697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:29.236240 29407 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:29.239500 29070 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:29.239411 29382 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:29.242116 29262 tablet_bootstrap.cc:492] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:29.242645 29262 ts_tablet_manager.cc:1403] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.238s	user 0.013s	sys 0.017s
I20260709 13:03:29.248447 29262 raft_consensus.cc:359] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.255223 29262 raft_consensus.cc:385] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.255947 29262 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.256747 29262 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.259528 29262 ts_tablet_manager.cc:1434] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:03:29.262676 29262 tablet_bootstrap.cc:492] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:29.267906 29407 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.268282 29408 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.268648 29407 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.268931 29408 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.267479 29264 tablet_bootstrap.cc:492] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:29.270285 29264 ts_tablet_manager.cc:1403] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.108s	user 0.013s	sys 0.021s
I20260709 13:03:29.269665 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.275439 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.277135 29264 raft_consensus.cc:359] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.277515 29188 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.279650 29264 raft_consensus.cc:385] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.280014 29264 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.280361 29382 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:29.280382 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:29.280854 29264 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.279273 29263 tablet_bootstrap.cc:654] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.274521 29407 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.289955 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.290752 29145 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:29.296778 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.296905 29293 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:29.297304 29264 ts_tablet_manager.cc:1434] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:03:29.294480 29382 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.298157 29382 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.298297 29264 tablet_bootstrap.cc:492] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:29.300423 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:29.292695 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:29.303650 29113 leader_election.cc:343] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.304597 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:29.305330 29218 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:29.308524 29070 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:29.312711 29382 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:29.294054 29408 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.307724 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:29.318313 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.324237 29070 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:29.325708 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:29.328373 29408 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:29.333382 29408 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.334287 29408 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.328409 29382 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:29.337060 29382 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.337653 29382 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.337937 29262 tablet_bootstrap.cc:654] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.339433 28971 catalog_manager.cc:5697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.341310 28971 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.346550 29408 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.328331 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.348770 29407 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.350351 29407 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.352954 29293 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:29.355996 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:29.356809 29407 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:29.358050 29113 leader_election.cc:343] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.355304 29143 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
W20260709 13:03:29.356563 29111 leader_election.cc:343] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.361490 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:29.363250 29264 tablet_bootstrap.cc:654] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.364948 29407 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:29.366366 29407 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.367095 29407 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.367857 29382 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.370682 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:29.371592 29218 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.373674 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.377254 29382 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:29.371272 29408 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.380261 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:29.380710 29070 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.382968 28968 catalog_manager.cc:5697] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:29.384192 29218 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:29.384701 29377 raft_consensus.cc:2764] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:03:29.384970 28968 catalog_manager.cc:5697] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:29.386533 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:29.387269 29408 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:29.387490 29407 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.387912 29408 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:29.388885 29408 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.389642 29407 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.393359 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.393962 29039 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:29.394127 29073 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:29.395081 29307 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:29.395596 29307 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:29.396158 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:29.396461 29307 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.397203 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:03:29.397523 29113 leader_election.cc:343] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.398196 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:29.398273 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:29.398273 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:29.398852 29071 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.403165 29262 tablet_bootstrap.cc:492] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:29.403801 29262 ts_tablet_manager.cc:1403] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.141s	user 0.006s	sys 0.010s
I20260709 13:03:29.406356 29071 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:29.407528 29262 raft_consensus.cc:359] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.413398 29262 raft_consensus.cc:385] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.413892 29262 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.411978 29070 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:29.415370 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:29.415163 29262 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.411711 29263 tablet_bootstrap.cc:492] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:29.416251 29377 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:29.424672 29377 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:29.424803 29263 ts_tablet_manager.cc:1403] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.200s	user 0.019s	sys 0.009s
I20260709 13:03:29.425608 29377 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.408034 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:29.441273 29262 ts_tablet_manager.cc:1434] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
I20260709 13:03:29.442943 29262 tablet_bootstrap.cc:492] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:29.441872 29408 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:29.444401 29408 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.445255 29408 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:29.447669 29111 leader_election.cc:343] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.448657 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:29.449995 29293 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:03:29.450289 29188 leader_election.cc:343] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.450456 29293 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:03:29.451303 29188 leader_election.cc:343] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.451362 29293 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.452034 29263 raft_consensus.cc:359] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.452666 29263 raft_consensus.cc:385] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.452916 29263 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.454965 29263 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.464413 29263 ts_tablet_manager.cc:1434] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 13:03:29.457289 28970 catalog_manager.cc:5697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.474874 29270 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:29.477723 29263 tablet_bootstrap.cc:492] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:29.477156 29408 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.504843 29264 tablet_bootstrap.cc:492] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:29.505506 29264 ts_tablet_manager.cc:1403] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.207s	user 0.015s	sys 0.021s
I20260709 13:03:29.511006 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:29.511649 29070 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.512549 29408 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.516235 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:29.516680 29143 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:29.517853 29113 leader_election.cc:343] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.518182 29270 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:29.532451 29071 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:29.534993 29355 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:29.533239 29264 raft_consensus.cc:359] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.539570 29264 raft_consensus.cc:385] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.539983 29264 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.540652 29264 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.542029 29070 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:29.543143 29264 ts_tablet_manager.cc:1434] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.037s	user 0.003s	sys 0.003s
I20260709 13:03:29.544857 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:29.545722 29293 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:29.544257 29264 tablet_bootstrap.cc:492] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:29.545969 28970 catalog_manager.cc:5697] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.546427 28968 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.548569 29293 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:29.549772 29263 tablet_bootstrap.cc:654] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.550333 29293 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.555035 29377 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.555507 29377 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.557929 29377 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:29.566226 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.567446 29141 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:03:29.571218 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.576349 29188 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:29.577971 29270 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:29.578348 29270 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.578722 29270 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:29.580473 29355 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:29.587555 29262 tablet_bootstrap.cc:654] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.590662 29359 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:29.598735 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08000add00 after lost signal to thread 27937
I20260709 13:03:29.599520 28971 catalog_manager.cc:5697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:29.601480 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08001cb200 after lost signal to thread 29005
W20260709 13:03:29.610410 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08001c78e0 after lost signal to thread 29097
W20260709 13:03:29.611449 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08000e4ec0 after lost signal to thread 29172
W20260709 13:03:29.612428 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9640 after lost signal to thread 29246
I20260709 13:03:29.618466 29359 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:29.632917 29270 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.639997 29264 tablet_bootstrap.cc:654] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.641319 29392 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.641945 29392 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.644804 29392 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.606000 29293 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:03:29.645852 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:29.647326 29038 leader_election.cc:343] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.645768 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:03:29.650447 29039 leader_election.cc:343] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.651065 29039 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:29.652149 29389 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:29.652232 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:29.652391 29270 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:29.653959 29263 tablet_bootstrap.cc:492] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:29.654585 29263 ts_tablet_manager.cc:1403] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.177s	user 0.008s	sys 0.011s
I20260709 13:03:29.656270 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:29.659791 29143 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:29.662043 29263 raft_consensus.cc:359] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.662688 29263 raft_consensus.cc:385] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.662957 29263 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.670506 29263 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.693679 29263 ts_tablet_manager.cc:1434] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.004s
I20260709 13:03:29.695026 29263 tablet_bootstrap.cc:492] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:29.697739 29219 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.704115 29262 tablet_bootstrap.cc:492] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:29.704613 29262 ts_tablet_manager.cc:1403] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.262s	user 0.022s	sys 0.000s
I20260709 13:03:29.717473 29143 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:03:29.726259 29188 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:29.727434 29432 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:29.728006 29432 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:29.728976 29432 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.740450 29359 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:29.762804 29262 raft_consensus.cc:359] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.768857 29264 tablet_bootstrap.cc:492] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:29.769968 28969 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.775733 29262 raft_consensus.cc:385] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.775992 29264 ts_tablet_manager.cc:1403] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.232s	user 0.016s	sys 0.009s
I20260709 13:03:29.776209 29262 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.776417 29263 tablet_bootstrap.cc:654] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.782759 29262 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.783324 29264 raft_consensus.cc:359] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.784039 29264 raft_consensus.cc:385] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.784303 29264 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.784902 29264 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.786846 29264 ts_tablet_manager.cc:1434] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.000s
I20260709 13:03:29.787717 29264 tablet_bootstrap.cc:492] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:29.787761 29262 ts_tablet_manager.cc:1434] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.083s	user 0.009s	sys 0.000s
I20260709 13:03:29.788692 29262 tablet_bootstrap.cc:492] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:29.794485 29262 tablet_bootstrap.cc:654] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.793045 28971 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 2 to 3, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 3 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:29.797590 29143 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:29.797438 29389 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.798775 29264 tablet_bootstrap.cc:654] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.801234 29389 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.804229 29263 tablet_bootstrap.cc:492] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:29.805415 29392 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:29.804667 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.806535 29392 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:29.806681 29143 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:29.804666 29263 ts_tablet_manager.cc:1403] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.110s	user 0.012s	sys 0.003s
I20260709 13:03:29.807053 29392 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:29.808353 29392 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.809674 29263 raft_consensus.cc:359] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.809909 29392 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:29.810324 29263 raft_consensus.cc:385] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.810614 29263 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.811251 29263 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.813724 29389 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:29.814661 29263 ts_tablet_manager.cc:1434] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:29.814996 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:29.815667 29263 tablet_bootstrap.cc:492] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:29.818395 29389 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:29.819336 29389 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:29.820226 29389 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.814699 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:29.825305 29038 leader_election.cc:343] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.831250 29262 tablet_bootstrap.cc:492] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:29.832137 29262 ts_tablet_manager.cc:1403] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.000s
I20260709 13:03:29.835152 29262 raft_consensus.cc:359] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.835820 29262 raft_consensus.cc:385] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.836112 29262 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.837244 29263 tablet_bootstrap.cc:654] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.836836 29262 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.840297 29262 ts_tablet_manager.cc:1434] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:03:29.841300 29262 tablet_bootstrap.cc:492] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:29.845837 29389 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.849953 29264 tablet_bootstrap.cc:492] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:29.850577 29264 ts_tablet_manager.cc:1403] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.063s	user 0.028s	sys 0.006s
I20260709 13:03:29.853271 29262 tablet_bootstrap.cc:654] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:29.853065 29407 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:29.856375 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:29.856841 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:29.857046 29143 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.855101 29389 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:29.858610 29038 leader_election.cc:343] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:29.859197 29141 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.860214 29264 raft_consensus.cc:359] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.861328 29264 raft_consensus.cc:385] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.861699 29264 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.862504 29264 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.866297 29264 ts_tablet_manager.cc:1434] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:03:29.870609 29263 tablet_bootstrap.cc:492] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:29.871055 29263 ts_tablet_manager.cc:1403] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.000s
I20260709 13:03:29.940258 29432 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.874413 29263 raft_consensus.cc:359] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.943082 29432 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.948679 29071 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:29.956451 29407 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:29.959687 29377 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:29.960399 29377 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:29.963505 29377 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:29.972879 29355 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:29.973337 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.976779 29263 raft_consensus.cc:385] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:29.977305 29263 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:29.978322 29263 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:29.979506 29113 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:29.980548 29262 tablet_bootstrap.cc:492] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:29.981460 29262 ts_tablet_manager.cc:1403] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.140s	user 0.012s	sys 0.009s
I20260709 13:03:29.982662 29264 tablet_bootstrap.cc:492] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:29.990550 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:29.990892 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.991431 29141 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:29.992300 29073 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:03:29.992316 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:29.996089 29072 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:29.959690 29432 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:30.006179 29263 ts_tablet_manager.cc:1434] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.135s	user 0.006s	sys 0.003s
I20260709 13:03:30.007742 29263 tablet_bootstrap.cc:492] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:30.007212 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:30.012079 29382 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.017642 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:30.018852 29452 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:30.019331 29452 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.019764 29452 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.031211 29355 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:30.037120 29382 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:30.037194 29407 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:29.998319 29143 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:30.044345 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:30.051467 29392 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:30.052253 29392 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:30.053610 29392 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.056269 29262 raft_consensus.cc:359] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.057056 29262 raft_consensus.cc:385] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.057358 29262 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.059113 29262 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.066704 29262 ts_tablet_manager.cc:1434] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.085s	user 0.004s	sys 0.006s
W20260709 13:03:30.072693 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:30.092535 29262 tablet_bootstrap.cc:492] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:30.094002 29263 tablet_bootstrap.cc:654] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.096277 29264 tablet_bootstrap.cc:654] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.117511 29408 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:30.118448 29293 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:30.128527 29408 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.129043 29408 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.128429 28971 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:30.132633 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:30.133749 29408 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:30.134881 29113 leader_election.cc:343] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.137216 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:30.139051 29111 leader_election.cc:343] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.139506 29111 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:30.139701 29452 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.142951 29293 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:30.142813 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:30.143715 29070 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.153523 29452 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:30.155321 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:30.155948 29146 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.156765 29262 tablet_bootstrap.cc:654] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.175058 29073 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:03:30.175668 29073 pending_rounds.cc:85] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 1
I20260709 13:03:30.176100 29073 pending_rounds.cc:107] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:03:30.176524 29073 raft_consensus.cc:2889] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:30.180219 29452 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:30.179471 29146 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:30.191453 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:30.196756 29452 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:30.199618 29070 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:30.200039 29264 tablet_bootstrap.cc:492] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:30.200474 29264 ts_tablet_manager.cc:1403] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.218s	user 0.008s	sys 0.015s
I20260709 13:03:30.200753 29452 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:30.203778 29452 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.205305 29264 raft_consensus.cc:359] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.206269 29264 raft_consensus.cc:385] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.206645 29264 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.207473 29264 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.210901 29264 ts_tablet_manager.cc:1434] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:03:30.212424 29264 tablet_bootstrap.cc:492] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:30.223093 29262 tablet_bootstrap.cc:492] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:30.223778 29262 ts_tablet_manager.cc:1403] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.132s	user 0.022s	sys 0.006s
W20260709 13:03:30.233224 29457 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:03:30.233337 29146 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:03:30.237833 29452 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:30.239538 29262 raft_consensus.cc:359] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.242580 29262 raft_consensus.cc:385] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.242807 29262 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.246191 29263 tablet_bootstrap.cc:492] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:30.246675 29263 ts_tablet_manager.cc:1403] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.239s	user 0.016s	sys 0.016s
I20260709 13:03:30.244724 28971 catalog_manager.cc:5697] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:30.247610 29262 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.248481 29432 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.248947 29432 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.249917 29262 ts_tablet_manager.cc:1434] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:03:30.251042 29262 tablet_bootstrap.cc:492] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:30.253865 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:30.254380 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:30.255144 29143 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
W20260709 13:03:30.255509 29185 leader_election.cc:343] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.255774 29263 raft_consensus.cc:359] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.256465 29263 raft_consensus.cc:385] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.256752 29263 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.251051 29432 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:30.257565 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:30.260018 29452 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:30.260397 29452 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.260744 29452 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.261343 29263 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.260880 29264 tablet_bootstrap.cc:654] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.268240 29263 ts_tablet_manager.cc:1434] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.002s
I20260709 13:03:30.269045 29262 tablet_bootstrap.cc:654] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.269613 29263 tablet_bootstrap.cc:492] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:30.278288 29264 tablet_bootstrap.cc:492] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:30.278803 29264 ts_tablet_manager.cc:1403] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.067s	user 0.010s	sys 0.001s
I20260709 13:03:30.280018 29263 tablet_bootstrap.cc:654] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.281342 29264 raft_consensus.cc:359] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.282020 29264 raft_consensus.cc:385] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.282275 29264 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.288269 29452 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.290912 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:03:30.292177 29185 leader_election.cc:343] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.295651 29452 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:30.296865 29264 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.298256 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:30.298960 29146 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.299897 29264 ts_tablet_manager.cc:1434] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:03:30.310303 29262 tablet_bootstrap.cc:492] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:30.311337 29262 ts_tablet_manager.cc:1403] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.061s	user 0.015s	sys 0.013s
I20260709 13:03:30.311568 29263 tablet_bootstrap.cc:492] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:30.310863 29264 tablet_bootstrap.cc:492] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:30.312039 29263 ts_tablet_manager.cc:1403] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.005s
I20260709 13:03:30.314988 29263 raft_consensus.cc:359] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.318171 29262 raft_consensus.cc:359] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.320092 29262 raft_consensus.cc:385] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.320376 29262 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.321099 29262 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.321801 29264 tablet_bootstrap.cc:654] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.323158 29262 ts_tablet_manager.cc:1434] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:03:30.319187 29146 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:30.315802 29263 raft_consensus.cc:385] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.324134 29263 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.324203 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:30.325465 29452 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:30.325103 29263 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.326191 29452 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:30.327464 29263 ts_tablet_manager.cc:1434] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:03:30.329190 29262 tablet_bootstrap.cc:492] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:30.327059 29452 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.342130 29263 tablet_bootstrap.cc:492] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:30.343021 29264 tablet_bootstrap.cc:492] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:30.343525 29264 ts_tablet_manager.cc:1403] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.003s
I20260709 13:03:30.346680 29264 raft_consensus.cc:359] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.347398 29264 raft_consensus.cc:385] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.347721 29264 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.348719 29264 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.350929 29264 ts_tablet_manager.cc:1434] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:03:30.352557 29263 tablet_bootstrap.cc:654] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.353982 29262 tablet_bootstrap.cc:654] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.355137 29264 tablet_bootstrap.cc:492] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:30.371598 29264 tablet_bootstrap.cc:654] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.376679 29262 tablet_bootstrap.cc:492] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:30.377359 29262 ts_tablet_manager.cc:1403] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.014s
I20260709 13:03:30.380157 29262 raft_consensus.cc:359] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.382599 29262 raft_consensus.cc:385] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.383283 29262 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.384362 29262 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.387063 29263 tablet_bootstrap.cc:492] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:30.387527 29263 ts_tablet_manager.cc:1403] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.002s
I20260709 13:03:30.387796 28971 catalog_manager.cc:5697] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:30.390220 29264 tablet_bootstrap.cc:492] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:30.390667 29264 ts_tablet_manager.cc:1403] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.004s
I20260709 13:03:30.391021 29262 ts_tablet_manager.cc:1434] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 13:03:30.391685 29263 raft_consensus.cc:359] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.392354 29263 raft_consensus.cc:385] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.392613 29262 tablet_bootstrap.cc:492] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:30.392608 29263 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.393481 29263 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.395887 29264 raft_consensus.cc:359] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.396587 29264 raft_consensus.cc:385] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.396858 29264 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.397516 29264 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.400206 29263 ts_tablet_manager.cc:1434] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:03:30.401052 29262 tablet_bootstrap.cc:654] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.401156 29263 tablet_bootstrap.cc:492] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:30.404140 29264 ts_tablet_manager.cc:1434] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:03:30.405258 29264 tablet_bootstrap.cc:492] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:30.412257 29263 tablet_bootstrap.cc:654] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.413491 29264 tablet_bootstrap.cc:654] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.428720 29392 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.429867 29262 tablet_bootstrap.cc:492] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:30.429420 29392 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.430384 29262 ts_tablet_manager.cc:1403] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.010s
I20260709 13:03:30.433480 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:30.434105 29262 raft_consensus.cc:359] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.434578 29146 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:30.434830 29262 raft_consensus.cc:385] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.435184 29262 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.435905 29262 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.435576 29452 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.436919 29452 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.438088 29293 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.436921 29359 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.438591 29293 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.439254 29452 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:30.439100 29359 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.441177 29359 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:30.441969 29218 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.443104 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:30.444655 29185 leader_election.cc:343] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.445305 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:30.447197 29113 leader_election.cc:343] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.447676 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:30.448374 29072 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:30.452059 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:30.453151 29407 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:30.453527 29407 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.453887 29407 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.456177 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:30.450734 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:30.457142 29392 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:30.458747 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:30.460007 29293 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:30.461092 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:03:30.462452 29111 leader_election.cc:343] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.463492 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:30.457176 29146 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
W20260709 13:03:30.468830 29113 leader_election.cc:343] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.469501 29113 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:30.471119 29293 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:30.471805 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:30.472091 29392 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.473253 29382 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:30.473706 29382 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.474108 29382 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.480058 29262 ts_tablet_manager.cc:1434] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.003s
I20260709 13:03:30.481324 29262 tablet_bootstrap.cc:492] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:30.488915 29264 tablet_bootstrap.cc:492] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:30.489513 29264 ts_tablet_manager.cc:1403] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.002s
I20260709 13:03:30.492748 29264 raft_consensus.cc:359] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.493726 29382 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.500216 29264 raft_consensus.cc:385] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.499151 29392 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:30.499737 29263 tablet_bootstrap.cc:492] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:30.504019 29389 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.504297 29263 ts_tablet_manager.cc:1403] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.103s	user 0.044s	sys 0.021s
I20260709 13:03:30.504702 29389 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.497303 29407 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.503242 29262 tablet_bootstrap.cc:654] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.506963 29263 raft_consensus.cc:359] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.503535 29264 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.507637 29263 raft_consensus.cc:385] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.507969 29263 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.508077 29264 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.508684 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:30.508751 29389 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:30.509312 29073 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.508911 29263 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.511873 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:30.512596 29146 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:30.513720 29263 ts_tablet_manager.cc:1434] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
W20260709 13:03:30.516080 29038 consensus_peers.cc:597] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:30.517154 29407 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:30.518626 29382 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:30.522142 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:30.523949 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:30.524098 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:03:30.525913 29185 leader_election.cc:343] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.526826 29392 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.527765 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:30.528527 29146 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.528170 29392 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.531139 29392 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:30.533385 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:03:30.533670 29038 leader_election.cc:343] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.538480 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:30.538836 29452 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:30.539674 29039 leader_election.cc:343] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.539465 29452 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.540700 29219 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:30.542306 29452 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:30.543809 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:30.545068 29038 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:30.546180 29073 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:30.546792 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:30.547538 29188 leader_election.cc:343] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.547683 29072 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:30.549170 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:30.550338 29185 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:30.551837 29359 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:30.552431 29359 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:30.552940 29452 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:30.553682 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:30.553427 29359 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.549276 29389 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:30.553434 29452 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.559111 29264 ts_tablet_manager.cc:1434] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.018s
I20260709 13:03:30.560294 29264 tablet_bootstrap.cc:492] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:30.564436 29263 tablet_bootstrap.cc:492] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:30.557421 29392 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:30.563068 29389 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.563221 29452 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.568796 29389 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.568145 29392 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:30.570448 29113 leader_election.cc:343] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.569763 29146 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:30.570333 29392 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.572831 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:30.579159 29382 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:30.579849 29382 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:30.580699 29382 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.583720 29475 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.601961 29263 tablet_bootstrap.cc:654] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.604518 29264 tablet_bootstrap.cc:654] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.599553 28971 catalog_manager.cc:5697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:30.608896 29262 tablet_bootstrap.cc:492] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:30.590507 29475 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.609611 29389 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.622437 29262 ts_tablet_manager.cc:1403] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.141s	user 0.008s	sys 0.026s
I20260709 13:03:30.621162 29478 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:30.613637 29452 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.627400 29262 raft_consensus.cc:359] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.628216 29262 raft_consensus.cc:385] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.628504 29262 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.630404 29262 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.636703 29389 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:30.638547 29475 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:30.645625 29452 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:30.646576 29262 ts_tablet_manager.cc:1434] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.002s
I20260709 13:03:30.646359 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:30.651280 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:30.652288 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:30.652129 29392 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.653033 29219 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:30.653045 29072 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:30.652406 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:30.656816 29038 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:30.658732 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:03:30.658867 29188 leader_election.cc:343] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.659518 29218 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:30.659339 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:30.659381 29188 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:30.660038 29146 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.660521 29475 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:30.660990 29475 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.661481 29475 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.660449 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:30.663316 29143 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:30.665071 29377 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:03:30.667465 29039 leader_election.cc:343] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.669040 29038 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:30.673655 29478 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:30.681800 29262 tablet_bootstrap.cc:492] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:30.690742 29264 tablet_bootstrap.cc:492] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:30.691529 29264 ts_tablet_manager.cc:1403] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.131s	user 0.015s	sys 0.014s
I20260709 13:03:30.686620 28971 catalog_manager.cc:5697] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:30.691097 29392 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:30.691133 29475 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.695428 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:03:30.697656 29038 leader_election.cc:343] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.699934 29263 tablet_bootstrap.cc:492] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:30.700089 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:30.700438 29263 ts_tablet_manager.cc:1403] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.165s	user 0.014s	sys 0.002s
I20260709 13:03:30.700665 29142 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.702718 29475 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:30.704366 29146 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:30.705991 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:30.707113 29475 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:30.707572 29475 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:30.708400 29475 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.713594 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:30.714128 29142 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:30.715175 29218 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.719908 29264 raft_consensus.cc:359] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.721271 29264 raft_consensus.cc:385] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.721549 29264 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.723057 29263 raft_consensus.cc:359] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.744293 29263 raft_consensus.cc:385] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.744493 29264 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.744656 29263 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.749099 29263 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.754190 29263 ts_tablet_manager.cc:1434] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.005s
I20260709 13:03:30.754215 29452 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:03:30.757766 29262 tablet_bootstrap.cc:654] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.760385 29264 ts_tablet_manager.cc:1434] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.069s	user 0.013s	sys 0.002s
I20260709 13:03:30.765273 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:30.768558 28969 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:30.773862 29263 tablet_bootstrap.cc:492] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:30.774585 29479 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:30.774704 29264 tablet_bootstrap.cc:492] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:30.775183 29479 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:30.776121 29479 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.768038 29218 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:30.805975 29263 tablet_bootstrap.cc:654] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.808338 29264 tablet_bootstrap.cc:654] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.808463 28971 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:30.820694 29262 tablet_bootstrap.cc:492] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:30.821170 29262 ts_tablet_manager.cc:1403] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.142s	user 0.007s	sys 0.012s
I20260709 13:03:30.823637 29262 raft_consensus.cc:359] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.824324 29262 raft_consensus.cc:385] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.824592 29262 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.825196 29262 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.827515 29262 ts_tablet_manager.cc:1434] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:03:30.828522 29262 tablet_bootstrap.cc:492] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:30.829017 29452 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:30.833451 29264 tablet_bootstrap.cc:492] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:30.833871 29264 ts_tablet_manager.cc:1403] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.000s
I20260709 13:03:30.836014 29264 raft_consensus.cc:359] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.836763 29264 raft_consensus.cc:385] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.837041 29264 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.845345 29263 tablet_bootstrap.cc:492] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:30.845978 29263 ts_tablet_manager.cc:1403] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.080s	user 0.009s	sys 0.003s
I20260709 13:03:30.845402 29264 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.847612 29264 ts_tablet_manager.cc:1434] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.014s	user 0.001s	sys 0.005s
I20260709 13:03:30.847659 29262 tablet_bootstrap.cc:654] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.848582 29264 tablet_bootstrap.cc:492] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:30.849141 29263 raft_consensus.cc:359] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.849833 29263 raft_consensus.cc:385] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.850087 29263 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.850860 29263 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.853520 29264 tablet_bootstrap.cc:654] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.859001 29263 ts_tablet_manager.cc:1434] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:03:30.860000 29262 tablet_bootstrap.cc:492] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:30.860466 29262 ts_tablet_manager.cc:1403] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.003s
I20260709 13:03:30.863130 29382 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.863597 29262 raft_consensus.cc:359] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.863673 29382 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.864369 29262 raft_consensus.cc:385] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.864651 29262 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.865319 29262 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.866129 29382 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:30.866876 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:30.868407 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:30.870515 29188 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:30.871021 29263 tablet_bootstrap.cc:492] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:30.871901 29382 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:30.883616 29262 ts_tablet_manager.cc:1434] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:03:30.884701 29262 tablet_bootstrap.cc:492] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:30.891847 29262 tablet_bootstrap.cc:654] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.895547 29263 tablet_bootstrap.cc:654] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.898941 29382 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:30.906296 29264 tablet_bootstrap.cc:492] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:30.908128 29264 ts_tablet_manager.cc:1403] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.060s	user 0.020s	sys 0.004s
I20260709 13:03:30.920990 29452 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:30.925186 29262 tablet_bootstrap.cc:492] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:30.925722 29262 ts_tablet_manager.cc:1403] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.041s	user 0.007s	sys 0.007s
I20260709 13:03:30.925186 29263 tablet_bootstrap.cc:492] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:30.928694 29263 ts_tablet_manager.cc:1403] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.058s	user 0.017s	sys 0.011s
I20260709 13:03:30.930454 29263 raft_consensus.cc:359] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.930961 29263 raft_consensus.cc:385] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.931135 29263 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.931602 29263 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.932454 29262 raft_consensus.cc:359] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.933071 29262 raft_consensus.cc:385] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.933300 29262 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.933851 29262 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.936156 29262 ts_tablet_manager.cc:1434] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:03:30.937619 29262 tablet_bootstrap.cc:492] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:30.943305 29262 tablet_bootstrap.cc:654] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.944903 29264 raft_consensus.cc:359] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.945652 29264 raft_consensus.cc:385] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.945914 29264 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.946542 29264 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.948508 29264 ts_tablet_manager.cc:1434] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.000s
I20260709 13:03:30.949448 29264 tablet_bootstrap.cc:492] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:30.954495 29263 ts_tablet_manager.cc:1434] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.000s
I20260709 13:03:30.963600 29408 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:30.971364 29408 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.972611 29263 tablet_bootstrap.cc:492] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:30.974309 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:30.974846 29408 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:30.975013 29070 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:30.974434 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:30.975828 29264 tablet_bootstrap.cc:654] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:30.976830 29113 leader_election.cc:343] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:30.977943 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:30.978649 29408 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:30.979002 29408 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:30.979347 29408 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:30.981052 29407 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:30.981653 29263 tablet_bootstrap.cc:654] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:30.987653 29262 tablet_bootstrap.cc:492] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:30.988121 29262 ts_tablet_manager.cc:1403] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.051s	user 0.022s	sys 0.027s
I20260709 13:03:30.988485 29263 tablet_bootstrap.cc:492] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:30.988927 29263 ts_tablet_manager.cc:1403] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 13:03:30.990824 29262 raft_consensus.cc:359] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.991497 29262 raft_consensus.cc:385] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.991796 29262 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.992436 29262 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.991320 29263 raft_consensus.cc:359] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.993213 29263 raft_consensus.cc:385] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:30.993479 29263 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:30.994304 29262 ts_tablet_manager.cc:1434] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:03:30.994076 29263 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.994477 29408 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:30.995291 29262 tablet_bootstrap.cc:492] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:30.996030 29263 ts_tablet_manager.cc:1434] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:30.997542 29263 tablet_bootstrap.cc:492] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:31.002164 29262 tablet_bootstrap.cc:654] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.003769 29263 tablet_bootstrap.cc:654] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.013175 29408 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.014428 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:31.015264 29263 tablet_bootstrap.cc:492] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:31.015713 29263 ts_tablet_manager.cc:1403] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
W20260709 13:03:31.017099 29113 leader_election.cc:343] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:31.019191 29262 tablet_bootstrap.cc:492] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:31.019632 29262 ts_tablet_manager.cc:1403] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.000s
I20260709 13:03:31.022651 29392 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.022740 29263 raft_consensus.cc:359] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.023586 29263 raft_consensus.cc:385] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.023231 29392 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.023846 29263 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.024535 29263 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.025856 29392 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.025988 29264 tablet_bootstrap.cc:492] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:31.026330 29263 ts_tablet_manager.cc:1434] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.000s
I20260709 13:03:31.026993 29264 ts_tablet_manager.cc:1403] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.078s	user 0.005s	sys 0.011s
I20260709 13:03:31.027376 29263 tablet_bootstrap.cc:492] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:31.029505 29264 raft_consensus.cc:359] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.030179 29264 raft_consensus.cc:385] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.030465 29264 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.031261 29264 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.031548 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.033267 29264 ts_tablet_manager.cc:1434] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:03:31.033419 29263 tablet_bootstrap.cc:654] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.034318 29264 tablet_bootstrap.cc:492] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:31.037643 29293 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.038133 29293 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.040879 29293 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.042754 29264 tablet_bootstrap.cc:654] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.044656 29263 tablet_bootstrap.cc:492] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:31.045094 29263 ts_tablet_manager.cc:1403] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.007s
I20260709 13:03:31.045003 29262 raft_consensus.cc:359] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.045734 29262 raft_consensus.cc:385] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.045985 29262 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.046703 29262 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.048273 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.050221 29220 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.053370 29262 ts_tablet_manager.cc:1434] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 13:03:31.054001 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:31.054750 29143 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:31.055482 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:31.055658 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:31.058154 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:31.058941 29072 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.056888 29392 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:31.062117 29113 leader_election.cc:343] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:31.064828 29111 leader_election.cc:343] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:31.065244 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:31.065907 29293 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:31.066915 29392 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:31.067495 29262 tablet_bootstrap.cc:492] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:31.073279 29219 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.089330 29263 raft_consensus.cc:359] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.090059 29263 raft_consensus.cc:385] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.090327 29263 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.098510 29263 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.101732 29408 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:31.109442 29263 ts_tablet_manager.cc:1434] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.002s
I20260709 13:03:31.108523 29072 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:31.126869 29262 tablet_bootstrap.cc:654] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:31.127697 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:31.129861 29263 tablet_bootstrap.cc:492] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:31.131981 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:31.136425 29382 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:31.138414 29408 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:31.139248 29408 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:31.141855 29408 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.144109 29263 tablet_bootstrap.cc:654] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.146786 29264 tablet_bootstrap.cc:492] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:31.147404 29264 ts_tablet_manager.cc:1403] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.113s	user 0.028s	sys 0.007s
I20260709 13:03:31.155378 29264 raft_consensus.cc:359] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.160313 29264 raft_consensus.cc:385] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.160705 29264 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.161429 29264 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.172221 29264 ts_tablet_manager.cc:1434] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.002s
I20260709 13:03:31.176335 29262 tablet_bootstrap.cc:492] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:31.176837 29262 ts_tablet_manager.cc:1403] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.110s	user 0.008s	sys 0.008s
I20260709 13:03:31.162828 29382 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:03:31.179909 29264 tablet_bootstrap.cc:492] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:31.305598 29263 tablet_bootstrap.cc:492] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:31.200064 29262 raft_consensus.cc:359] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.306164 29263 ts_tablet_manager.cc:1403] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.181s	user 0.007s	sys 0.014s
I20260709 13:03:31.306361 29262 raft_consensus.cc:385] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.306649 29262 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.307261 29262 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.198089 29382 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.308441 29382 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.308926 29263 raft_consensus.cc:359] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.309566 29262 ts_tablet_manager.cc:1434] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.132s	user 0.039s	sys 0.028s
I20260709 13:03:31.309595 29263 raft_consensus.cc:385] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.310006 29263 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.310566 29382 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:31.310702 29262 tablet_bootstrap.cc:492] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:31.260484 29407 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.311648 29407 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.311029 29263 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.314734 29263 ts_tablet_manager.cc:1434] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:03:31.315317 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.262667 29408 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.230206 28969 catalog_manager.cc:5697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:31.316165 29220 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:31.316120 29408 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.317283 29262 tablet_bootstrap.cc:654] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.318648 29408 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.318912 29263 tablet_bootstrap.cc:492] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:31.319777 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.320322 29407 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.350092 29220 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:31.320752 29392 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.351166 29392 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.351585 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:31.352795 29407 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:31.353145 29407 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.353330 29392 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.353466 29407 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.321734 29264 tablet_bootstrap.cc:654] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.327253 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:31.356432 29143 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:31.341318 29219 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.343160 29262 tablet_bootstrap.cc:492] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:31.347143 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:31.357820 29262 ts_tablet_manager.cc:1403] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.047s	user 0.008s	sys 0.020s
I20260709 13:03:31.358035 29070 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:31.360401 29262 raft_consensus.cc:359] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.361137 29262 raft_consensus.cc:385] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.361404 29262 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.362054 29262 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.362798 29264 tablet_bootstrap.cc:492] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:31.363265 29264 ts_tablet_manager.cc:1403] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.184s	user 0.006s	sys 0.020s
I20260709 13:03:31.364365 29262 ts_tablet_manager.cc:1434] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:03:31.365383 29262 tablet_bootstrap.cc:492] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:31.348958 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:31.368249 29264 raft_consensus.cc:359] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.368984 29264 raft_consensus.cc:385] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.369268 29264 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.369993 29264 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.369357 29407 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.371940 29262 tablet_bootstrap.cc:654] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.372187 29264 ts_tablet_manager.cc:1434] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:03:31.373306 29264 tablet_bootstrap.cc:492] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:31.375164 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:31.376062 29219 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.379362 29263 tablet_bootstrap.cc:654] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.388708 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:31.389586 29070 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:31.392194 29264 tablet_bootstrap.cc:654] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.393810 29382 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.394289 29382 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.397189 29382 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:31.399513 29407 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.406822 29185 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:31.409143 29432 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:31.409662 29432 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.410099 29432 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.414330 29516 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.414880 29516 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.417361 29516 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.416574 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.420217 29264 tablet_bootstrap.cc:492] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:31.420351 29218 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:31.420683 29264 ts_tablet_manager.cc:1403] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.048s	user 0.008s	sys 0.010s
I20260709 13:03:31.423418 29264 raft_consensus.cc:359] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.424120 29264 raft_consensus.cc:385] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.424405 29264 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.425096 29264 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.426604 29264 ts_tablet_manager.cc:1434] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:03:31.427551 29264 tablet_bootstrap.cc:492] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:31.440778 29516 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:31.445948 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:31.446590 29264 tablet_bootstrap.cc:654] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.447139 29143 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:31.448330 29407 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.448930 29407 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.451584 29407 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.453526 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:31.457043 29113 leader_election.cc:343] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:31.459697 29516 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:31.460654 29264 tablet_bootstrap.cc:492] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:31.461107 29264 ts_tablet_manager.cc:1403] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.000s
I20260709 13:03:31.463205 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:31.463848 29264 raft_consensus.cc:359] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.464541 29264 raft_consensus.cc:385] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.464815 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:31.465430 29143 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:31.464831 29264 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.466773 29264 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.469137 29264 ts_tablet_manager.cc:1434] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:03:31.470103 29264 tablet_bootstrap.cc:492] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:31.470306 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:31.470855 29072 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.472854 29263 tablet_bootstrap.cc:492] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:31.473284 29263 ts_tablet_manager.cc:1403] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.155s	user 0.006s	sys 0.005s
I20260709 13:03:31.475952 29264 tablet_bootstrap.cc:654] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.477703 29516 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:31.476584 29263 raft_consensus.cc:359] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.479321 29263 raft_consensus.cc:385] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.479615 29263 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.480777 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:31.480921 29516 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:31.481326 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:31.482093 29392 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:31.480376 29263 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.482461 29392 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.482604 29516 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:31.482808 29392 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.483058 29516 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.483453 29516 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.483589 29263 ts_tablet_manager.cc:1434] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:03:31.492985 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:31.495841 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.496883 29432 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.499617 29432 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:31.499668 29392 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.505527 29392 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.505699 29262 tablet_bootstrap.cc:492] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:31.506053 29072 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:31.506278 29262 ts_tablet_manager.cc:1403] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.141s	user 0.036s	sys 0.026s
I20260709 13:03:31.502645 29219 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:31.491504 29516 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.508389 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:31.509323 29408 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:31.509829 29408 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:31.510651 29408 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.511510 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:31.511698 29516 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.513708 29262 raft_consensus.cc:359] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:31.514977 29038 leader_election.cc:343] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:31.516590 29038 leader_election.cc:343] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:31.517230 29264 tablet_bootstrap.cc:492] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:31.515799 29262 raft_consensus.cc:385] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.517666 29264 ts_tablet_manager.cc:1403] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.005s
I20260709 13:03:31.517598 29262 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.520618 29392 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.520865 29264 raft_consensus.cc:359] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.521551 29264 raft_consensus.cc:385] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.521178 29392 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.521823 29264 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.523509 29392 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.523351 29264 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.524952 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:31.525410 29264 ts_tablet_manager.cc:1434] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:03:31.525609 29221 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.525269 29262 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.529096 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.529785 29220 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:31.530234 29264 tablet_bootstrap.cc:492] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:31.536288 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:31.537181 29072 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.544261 29262 ts_tablet_manager.cc:1434] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:03:31.546099 29263 tablet_bootstrap.cc:492] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:31.544974 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:31.547124 29262 tablet_bootstrap.cc:492] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:31.547791 29407 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.548041 29264 tablet_bootstrap.cc:654] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.548364 29407 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.553413 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:31.554255 29142 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:31.553783 28971 catalog_manager.cc:5697] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:31.574753 29516 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:31.578183 29516 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.578155 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.578835 29516 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.582728 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:31.583343 29146 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:31.584525 29113 leader_election.cc:343] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:31.592418 29262 tablet_bootstrap.cc:654] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.575316 29407 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.600649 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:03:31.573279 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59922.4us
I20260709 13:03:31.602226 29408 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:31.607971 29111 leader_election.cc:343] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:31.599613 29516 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.611202 29264 tablet_bootstrap.cc:492] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:31.611711 29264 ts_tablet_manager.cc:1403] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.082s	user 0.013s	sys 0.035s
W20260709 13:03:31.612672 29111 leader_election.cc:343] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:31.613062 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:31.614348 29146 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:31.615299 29407 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.615794 29407 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.610579 29221 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:31.617008 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:31.617635 29218 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.617771 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:31.618842 29532 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:31.619796 29532 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:31.620301 29264 raft_consensus.cc:359] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.620944 29264 raft_consensus.cc:385] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.620955 29516 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.621340 29264 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.620748 29532 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.622676 29264 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.625133 29264 ts_tablet_manager.cc:1434] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:03:31.626109 29264 tablet_bootstrap.cc:492] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:31.631722 29359 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:31.634073 29218 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:31.634479 29263 tablet_bootstrap.cc:654] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.635361 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:31.637596 29516 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:31.639310 29359 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.639407 29516 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:31.639828 29359 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.640375 29516 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.642042 29359 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.644160 29479 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.644663 29479 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.648654 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:31.649443 29073 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:31.650633 29408 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:31.651216 29113 consensus_peers.cc:597] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:31.652099 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.652174 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.653017 29218 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:31.653087 29262 tablet_bootstrap.cc:492] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:31.653625 29262 ts_tablet_manager.cc:1403] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.107s	user 0.009s	sys 0.006s
W20260709 13:03:31.653653 29038 leader_election.cc:343] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:31.631345 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.618433 29407 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.654984 29264 tablet_bootstrap.cc:654] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:31.646806 29479 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.656056 29072 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:31.657450 29221 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:31.657960 29185 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:31.658403 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:31.658958 29382 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:31.659435 29382 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:31.659889 29527 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:31.660297 29527 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:31.660686 29527 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:31.660396 29382 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.666013 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:31.666898 29407 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:31.667272 29407 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.667593 29407 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.669907 29262 raft_consensus.cc:359] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.670637 29262 raft_consensus.cc:385] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.670928 29262 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.672910 29262 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.674021 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:31.674050 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:31.678064 29143 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.678916 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:31.679481 29142 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.681217 28968 catalog_manager.cc:5697] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:31.688403 29143 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:31.688611 29527 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.691352 29262 ts_tablet_manager.cc:1434] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.008s
I20260709 13:03:31.704780 29262 tablet_bootstrap.cc:492] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:31.710307 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:31.710307 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:31.711130 29145 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:31.711297 29143 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.716428 29527 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:31.717203 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:31.707393 29142 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:31.720844 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:31.735496 29111 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:03:31.740443 29111 leader_election.cc:343] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:31.720093 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 50871.3us
I20260709 13:03:31.750861 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:31.754484 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:31.755146 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:31.783106 29532 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:31.783869 29536 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:31.742312 29407 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.787783 29532 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.788036 29536 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:31.793815 29536 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:31.782155 29479 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:31.786425 28971 catalog_manager.cc:5697] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:31.795907 29532 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:31.822475 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 80990.1us
I20260709 13:03:31.818423 28968 catalog_manager.cc:5697] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:31.849057 29479 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:31.849756 29143 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:31.890276 29392 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:31.892040 29407 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.899103 29263 tablet_bootstrap.cc:492] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:31.900538 29392 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.900532 29263 ts_tablet_manager.cc:1403] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.355s	user 0.018s	sys 0.048s
I20260709 13:03:31.908571 29479 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.919075 29392 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:31.939633 29263 raft_consensus.cc:359] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.942330 29263 raft_consensus.cc:385] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:31.942759 29263 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:31.945042 29263 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.966641 29263 ts_tablet_manager.cc:1434] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.000s
I20260709 13:03:31.971015 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:31.944746 29536 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.978259 29263 tablet_bootstrap.cc:492] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:31.985833 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:31.986493 29221 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:31.988323 29038 leader_election.cc:343] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:31.988337 29220 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:31.992465 29407 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:31.993503 29392 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:31.994380 29392 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:31.994237 29532 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.994782 29392 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:31.995728 29392 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:31.996766 29392 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:32.006732 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:03:32.008529 29038 leader_election.cc:343] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:32.003654 29532 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:32.007884 29221 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:32.013469 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:32.015535 29407 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:32.016656 29407 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:32.018683 29407 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.021858 29262 tablet_bootstrap.cc:654] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:32.025480 29264 tablet_bootstrap.cc:492] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
W20260709 13:03:32.025954 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 201220us
I20260709 13:03:32.026721 29111 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:32.027472 29407 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:32.029790 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:32.026069 29264 ts_tablet_manager.cc:1403] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.400s	user 0.027s	sys 0.025s
I20260709 13:03:32.037474 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:32.054303 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:32.055190 29073 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:32.061551 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:32.077744 29382 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.078306 29382 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.095438 29382 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:32.097395 29263 tablet_bootstrap.cc:654] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:32.108556 29264 raft_consensus.cc:359] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.109557 29264 raft_consensus.cc:385] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:32.109896 29264 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:32.116144 29264 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.117050 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:03:32.119870 29113 leader_election.cc:343] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:32.120939 29264 ts_tablet_manager.cc:1434] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.083s	user 0.005s	sys 0.004s
I20260709 13:03:32.126674 29264 tablet_bootstrap.cc:492] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:32.121421 29536 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:32.124234 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:32.131186 29262 tablet_bootstrap.cc:492] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:32.133505 29262 ts_tablet_manager.cc:1403] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.439s	user 0.033s	sys 0.047s
W20260709 13:03:32.137542 29111 leader_election.cc:343] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:32.140370 29546 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:32.143695 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:32.144574 29148 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:32.141788 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:32.148702 29382 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:32.152091 29262 raft_consensus.cc:359] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.152714 29262 raft_consensus.cc:385] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:32.152935 29262 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:32.150974 28969 catalog_manager.cc:5697] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:32.135790 29142 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:32.157722 29262 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.135313 29073 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
W20260709 13:03:32.136561 29039 leader_election.cc:343] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:32.165158 29039 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:32.166864 29516 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:32.162024 29541 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:32.169577 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:32.170169 29392 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:32.176048 29392 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.176565 29392 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.180264 29111 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:32.160944 28971 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:32.191077 29262 ts_tablet_manager.cc:1434] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 13:03:32.178640 29392 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:32.193454 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:32.194128 29219 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:32.199563 29528 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:32.193069 29541 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:32.213709 29541 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:32.199811 29142 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:32.224999 29262 tablet_bootstrap.cc:492] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:32.228940 29541 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.236824 29185 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:32.238291 29537 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:32.240013 29479 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:03:32.240078 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:32.248469 29145 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:32.273407 29038 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:32.274305 29263 tablet_bootstrap.cc:492] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:32.274801 29263 ts_tablet_manager.cc:1403] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.305s	user 0.022s	sys 0.019s
I20260709 13:03:32.238052 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:32.281633 29516 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:32.284343 29516 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:32.287352 29392 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:32.285332 29516 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.300009 29528 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:32.302145 29263 raft_consensus.cc:359] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.302882 29263 raft_consensus.cc:385] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:32.303128 29263 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:32.303838 29263 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.313357 29556 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:32.319841 29264 tablet_bootstrap.cc:654] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:32.325896 29263 ts_tablet_manager.cc:1434] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 13:03:32.314010 29556 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.340960 29263 tablet_bootstrap.cc:492] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:32.333700 29556 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:32.377538 29262 tablet_bootstrap.cc:654] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:32.384200 29532 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:32.361045 29537 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:32.414645 29219 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:32.444905 29392 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.446528 29392 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.448714 29392 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:32.453343 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:32.453397 29544 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:03:32.468884 28971 catalog_manager.cc:5697] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:32.470212 29071 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:32.478134 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:32.479249 29221 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 1: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:03:32.474294 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:03:32.501387 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 82843.3us
I20260709 13:03:32.508666 29407 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.509275 29407 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.511523 29407 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:32.511873 29565 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:03:32.513257 29263 tablet_bootstrap.cc:654] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:32.481613 29556 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.531697 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:32.533370 29556 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:32.537467 29113 leader_election.cc:343] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:32.556061 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:32.556690 29218 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:32.556826 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:32.556507 29407 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:32.557502 29143 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:32.559212 29264 tablet_bootstrap.cc:492] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:32.559701 29264 ts_tablet_manager.cc:1403] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.435s	user 0.031s	sys 0.048s
I20260709 13:03:32.564296 29527 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:32.564838 29527 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.565032 29039 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:32.592775 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:32.596210 29111 leader_election.cc:343] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:32.596966 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:32.598121 29407 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:32.602115 29527 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:32.611073 29143 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:03:32.608016 29556 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:32.627130 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:32.632623 29532 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:32.633220 29532 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:32.634177 29532 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.657722 29262 tablet_bootstrap.cc:492] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:32.658190 29262 ts_tablet_manager.cc:1403] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.442s	user 0.025s	sys 0.022s
I20260709 13:03:32.659876 29264 raft_consensus.cc:359] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.660567 29264 raft_consensus.cc:385] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:32.660786 29264 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:32.661762 29407 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:03:32.674615 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:32.703648 29556 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:32.649535 29218 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:03:32.779129 29264 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:32.786125 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 100390us
W20260709 13:03:32.792966 29549 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.007s
W20260709 13:03:32.800243 29572 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.071s	user 0.003s	sys 0.014s
I20260709 13:03:32.800304 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:32.799710 29264 ts_tablet_manager.cc:1434] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.240s	user 0.007s	sys 0.002s
I20260709 13:03:32.806383 29262 raft_consensus.cc:359] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.809696 29262 raft_consensus.cc:385] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:32.810108 29262 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:32.810981 29262 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.813455 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:32.817963 29537 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:32.826390 28971 catalog_manager.cc:5697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:32.837467 29099 tablet.cc:2406] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:03:32.817104 29543 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 13:03:32.854782 29263 tablet_bootstrap.cc:492] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:32.855394 29263 ts_tablet_manager.cc:1403] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.515s	user 0.077s	sys 0.069s
I20260709 13:03:32.858282 29263 raft_consensus.cc:359] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.859000 29263 raft_consensus.cc:385] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:32.859283 29263 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:32.860021 29263 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.862344 29263 ts_tablet_manager.cc:1434] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:03:32.864459 29263 tablet_bootstrap.cc:492] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:32.870045 29556 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.873808 29556 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.874842 29544 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:32.876955 29556 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:32.877776 29219 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:32.878218 29264 tablet_bootstrap.cc:492] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:32.883035 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:32.888447 29541 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:32.889001 29541 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:32.890326 29263 tablet_bootstrap.cc:654] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:32.900081 29038 leader_election.cc:343] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:32.892716 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:32.891173 29541 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:32.918252 29262 ts_tablet_manager.cc:1434] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.260s	user 0.007s	sys 0.020s
I20260709 13:03:32.922632 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:32.923460 29139 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
W20260709 13:03:32.924973 29113 leader_election.cc:343] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:32.925216 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:32.928472 29111 leader_election.cc:343] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:32.928907 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:32.936211 29262 tablet_bootstrap.cc:492] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:32.950506 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:32.951821 29264 tablet_bootstrap.cc:654] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:32.953075 29544 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:32.953545 29544 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:32.953994 29544 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:32.958984 29581 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:32.960227 28968 catalog_manager.cc:5697] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 2, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:32.967250 29581 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:32.977991 29263 tablet_bootstrap.cc:492] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:32.978596 29263 ts_tablet_manager.cc:1403] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.114s	user 0.016s	sys 0.012s
I20260709 13:03:32.999018 29544 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.001564 29544 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:33.003888 29038 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.004128 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:33.006325 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:33.021294 29139 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:33.011214 29263 raft_consensus.cc:359] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.024667 29263 raft_consensus.cc:385] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.025106 29263 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.026561 29263 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.027280 29262 tablet_bootstrap.cc:654] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.027804 29220 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:33.078248 29220 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:33.082494 29264 tablet_bootstrap.cc:492] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:33.083006 29264 ts_tablet_manager.cc:1403] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.205s	user 0.021s	sys 0.013s
I20260709 13:03:33.085533 29264 raft_consensus.cc:359] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.086225 29264 raft_consensus.cc:385] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.086508 29264 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.087193 29264 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.088423 29038 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:33.091461 29139 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:33.093053 29516 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:33.093551 29516 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:33.094501 29516 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.099848 29262 tablet_bootstrap.cc:492] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:33.100456 29262 ts_tablet_manager.cc:1403] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.176s	user 0.009s	sys 0.008s
I20260709 13:03:33.100800 29264 ts_tablet_manager.cc:1434] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:03:33.106357 29264 tablet_bootstrap.cc:492] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:33.110117 29262 raft_consensus.cc:359] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.111071 29262 raft_consensus.cc:385] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.111410 29262 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.112228 29262 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.112829 29263 ts_tablet_manager.cc:1434] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.134s	user 0.007s	sys 0.005s
I20260709 13:03:33.120071 29262 ts_tablet_manager.cc:1434] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:03:33.121063 29262 tablet_bootstrap.cc:492] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:33.123427 29263 tablet_bootstrap.cc:492] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:33.129438 29264 tablet_bootstrap.cc:654] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.131247 28969 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:33.136634 29262 tablet_bootstrap.cc:654] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.146574 29264 tablet_bootstrap.cc:492] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:33.146620 29263 tablet_bootstrap.cc:654] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.147176 29264 ts_tablet_manager.cc:1403] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.000s
I20260709 13:03:33.150142 29264 raft_consensus.cc:359] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.150892 29264 raft_consensus.cc:385] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.151160 29264 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.154249 29262 tablet_bootstrap.cc:492] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:33.154762 29262 ts_tablet_manager.cc:1403] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.000s
I20260709 13:03:33.157632 29262 raft_consensus.cc:359] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.158318 29262 raft_consensus.cc:385] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.158931 29264 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.160573 29264 ts_tablet_manager.cc:1434] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:03:33.158583 29262 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.164260 29264 tablet_bootstrap.cc:492] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:33.168887 29262 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.169628 29516 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.170131 29516 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.172333 29516 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:33.174455 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:33.175161 29140 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:33.175705 29262 ts_tablet_manager.cc:1434] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:03:33.175792 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:33.176786 29262 tablet_bootstrap.cc:492] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:33.177562 29039 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:03:33.179325 29038 leader_election.cc:343] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:33.179447 29264 tablet_bootstrap.cc:654] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.180464 29263 tablet_bootstrap.cc:492] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:33.180447 29516 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:33.181061 29263 ts_tablet_manager.cc:1403] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.058s	user 0.002s	sys 0.017s
I20260709 13:03:33.181061 29516 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.184365 29516 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:33.191112 29516 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.183617 29263 raft_consensus.cc:359] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.192343 29263 raft_consensus.cc:385] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.192605 29263 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.194940 29262 tablet_bootstrap.cc:654] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.193408 29263 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.197875 29264 tablet_bootstrap.cc:492] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:33.198323 29264 ts_tablet_manager.cc:1403] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:03:33.202258 29263 ts_tablet_manager.cc:1434] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:03:33.202463 29516 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:33.203361 29263 tablet_bootstrap.cc:492] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:33.204907 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:33.205490 29140 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:33.213238 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:33.214802 29264 raft_consensus.cc:359] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.215683 29264 raft_consensus.cc:385] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.216041 29264 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
W20260709 13:03:33.221216 29038 leader_election.cc:343] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:33.224045 29264 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.229822 29264 ts_tablet_manager.cc:1434] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.002s
I20260709 13:03:33.229972 29141 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:33.231968 29544 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:33.236075 29264 tablet_bootstrap.cc:492] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:33.246122 29220 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:33.248370 29532 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:33.255455 29263 tablet_bootstrap.cc:654] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.257129 29140 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:33.262408 29039 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:33.264186 29516 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:33.264678 29516 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:33.265484 29516 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.286139 29543 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:33.287792 29543 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.290771 29543 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:33.291426 29262 tablet_bootstrap.cc:492] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:33.291903 29262 ts_tablet_manager.cc:1403] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.115s	user 0.012s	sys 0.018s
I20260709 13:03:33.298117 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:33.299067 29069 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:03:33.299340 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:33.300096 29219 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:33.301692 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:33.301476 29262 raft_consensus.cc:359] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.302230 29262 raft_consensus.cc:385] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.302537 29262 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.304102 29543 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:33.306869 29264 tablet_bootstrap.cc:654] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.303259 29262 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.312968 29543 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:33.313482 29543 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:33.323184 29262 ts_tablet_manager.cc:1434] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:03:33.329084 29262 tablet_bootstrap.cc:492] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:33.334268 29528 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.334860 29528 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.337677 29528 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:33.341326 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:33.342077 29071 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:33.342304 28971 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:33.343225 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:33.344275 29528 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:33.344668 29528 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.345050 29528 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:33.346588 29543 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.351177 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:33.354478 29516 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.355650 29516 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.360781 29516 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:33.361799 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:33.362936 29263 tablet_bootstrap.cc:492] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:33.363827 29263 ts_tablet_manager.cc:1403] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.161s	user 0.013s	sys 0.002s
I20260709 13:03:33.338924 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:33.362578 29140 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:33.367261 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:33.370355 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:33.372320 29263 raft_consensus.cc:359] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.373086 29263 raft_consensus.cc:385] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.373353 29263 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.374183 29516 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:33.374616 29516 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.375017 29516 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:33.375453 29038 leader_election.cc:343] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:33.377408 29263 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.366470 29139 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:33.376570 29071 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:03:33.389398 29543 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:33.394064 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:33.398155 29263 ts_tablet_manager.cc:1434] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.001s
I20260709 13:03:33.399433 29263 tablet_bootstrap.cc:492] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:33.414121 29544 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.414762 29544 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.414626 29516 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.417495 29544 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:33.417853 29516 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:33.420604 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:33.421437 29262 tablet_bootstrap.cc:654] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.421350 29139 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:33.421195 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:33.422441 29140 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:33.424414 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:33.428438 29263 tablet_bootstrap.cc:654] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.367815 29528 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.431737 29516 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:33.397408 29543 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:33.434515 29528 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:33.436970 29516 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:33.439050 29543 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:33.456223 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:33.457168 29139 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:33.472240 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:33.473344 29072 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:33.448372 29516 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:33.534673 29139 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:33.536703 29543 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.542527 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:33.546082 29264 tablet_bootstrap.cc:492] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:33.546543 29264 ts_tablet_manager.cc:1403] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.311s	user 0.020s	sys 0.030s
I20260709 13:03:33.528024 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:33.528717 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:33.547765 29569 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:33.543902 29140 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:33.561013 29569 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:33.561079 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:33.561940 29222 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:33.562021 29569 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.574733 29264 raft_consensus.cc:359] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.575721 29264 raft_consensus.cc:385] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.576062 29264 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.576742 29264 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.578940 29264 ts_tablet_manager.cc:1434] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.000s
I20260709 13:03:33.591775 29264 tablet_bootstrap.cc:492] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:33.592434 29262 tablet_bootstrap.cc:492] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:33.593058 29262 ts_tablet_manager.cc:1403] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.265s	user 0.012s	sys 0.008s
W20260709 13:03:33.606361 29038 leader_election.cc:343] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:33.607209 29038 leader_election.cc:343] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:33.561231 29516 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.657236 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:33.679522 29263 tablet_bootstrap.cc:492] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:33.680292 29263 ts_tablet_manager.cc:1403] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.281s	user 0.032s	sys 0.002s
I20260709 13:03:33.665191 29479 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:03:33.699097 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:33.686780 29543 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.688745 29479 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:33.689446 29072 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:33.705376 29218 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:33.707660 29543 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.769469 29263 raft_consensus.cc:359] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.770388 29263 raft_consensus.cc:385] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.770723 29263 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.772233 29263 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.794606 29262 raft_consensus.cc:359] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.795606 29262 raft_consensus.cc:385] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:33.795974 29262 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:33.805944 29262 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.794982 29516 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:33.776662 29479 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:33.836018 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:33.840842 29262 ts_tablet_manager.cc:1434] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.247s	user 0.010s	sys 0.000s
I20260709 13:03:33.850203 29263 ts_tablet_manager.cc:1434] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.170s	user 0.019s	sys 0.020s
I20260709 13:03:33.850049 29543 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:33.856932 29516 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:03:33.874714 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:33.874958 29264 tablet_bootstrap.cc:654] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.883798 29262 tablet_bootstrap.cc:492] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:33.902988 29532 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:33.903580 29532 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:33.976963 29111 leader_election.cc:343] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:33.933578 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 57274.6us
I20260709 13:03:33.980372 29532 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:33.986513 29262 tablet_bootstrap.cc:654] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:33.989246 29263 tablet_bootstrap.cc:492] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
W20260709 13:03:34.000982 29038 leader_election.cc:343] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.031380 29608 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:34.032171 29608 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.045856 29264 tablet_bootstrap.cc:492] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:34.046799 29264 ts_tablet_manager.cc:1403] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.459s	user 0.014s	sys 0.038s
I20260709 13:03:34.018942 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:34.050464 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:34.054484 29216 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:34.054720 29574 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:34.061436 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:34.066006 29038 consensus_peers.cc:597] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:34.066622 29039 leader_election.cc:343] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.070894 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:34.071559 29146 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.085407 29544 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:03:34.086231 29038 leader_election.cc:343] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.088090 29038 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:34.089977 29532 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:34.093545 29608 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:03:34.085639 29621 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:03:34.095419 29146 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:34.108192 28971 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:34.108654 29113 leader_election.cc:343] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.110174 29113 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:34.120225 29263 tablet_bootstrap.cc:654] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.123355 29264 raft_consensus.cc:359] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.124166 29264 raft_consensus.cc:385] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.124404 29264 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.124979 29264 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.109365 29612 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.133719 29631 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:34.119414 29620 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:34.140847 29264 ts_tablet_manager.cc:1434] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.094s	user 0.007s	sys 0.002s
I20260709 13:03:34.144829 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:34.149679 29600 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.137754 29612 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.150336 29600 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.153574 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:34.166507 29532 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:34.167172 29532 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:34.168150 29532 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.173866 29612 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:34.177147 29262 tablet_bootstrap.cc:492] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:34.177752 29262 ts_tablet_manager.cc:1403] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.294s	user 0.011s	sys 0.016s
I20260709 13:03:34.177886 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:34.178596 29072 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:03:34.184314 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:03:34.186832 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 69654.6us
I20260709 13:03:34.189541 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:34.189553 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:34.211284 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:34.192143 29608 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:34.212401 29610 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:34.212648 29608 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.213022 29608 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:34.208822 29616 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:03:34.226334 28969 catalog_manager.cc:5697] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.228541 29264 tablet_bootstrap.cc:492] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:34.230983 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:34.234613 28968 catalog_manager.cc:5697] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:34.254230 29070 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:34.256146 29262 raft_consensus.cc:359] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.256927 29262 raft_consensus.cc:385] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.257231 29262 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.257906 29262 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.267994 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:34.268882 29146 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:34.258047 29605 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:03:34.270900 29262 ts_tablet_manager.cc:1434] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.000s
I20260709 13:03:34.272019 29262 tablet_bootstrap.cc:492] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:34.226336 29600 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:34.273211 29188 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:34.280427 29609 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.283221 29544 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.278805 29569 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:03:34.283744 29544 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.286002 29544 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:34.288587 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:34.290000 29038 leader_election.cc:343] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.259238 29608 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.300796 29604 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.301306 29220 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:03:34.301258 29604 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.303341 29604 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:34.300269 29264 tablet_bootstrap.cc:654] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.314702 29263 tablet_bootstrap.cc:492] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:34.315142 29263 ts_tablet_manager.cc:1403] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.326s	user 0.014s	sys 0.007s
I20260709 13:03:34.309939 29609 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:34.320184 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:34.322948 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:34.311823 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:03:34.325316 29111 leader_election.cc:343] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.314492 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:34.309365 29604 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:03:34.329275 29604 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:34.329696 29604 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:34.330559 29604 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.326817 29141 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
W20260709 13:03:34.332993 29113 leader_election.cc:343] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.333401 29113 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:34.336858 29039 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:34.339986 29608 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:34.339432 29532 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:34.340557 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:34.340780 29532 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.341121 29532 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.342052 29635 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:34.343345 29604 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:34.345813 29263 raft_consensus.cc:359] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.349341 29263 raft_consensus.cc:385] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.349668 29263 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.350200 29263 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.353039 29263 ts_tablet_manager.cc:1434] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.001s
I20260709 13:03:34.353987 29263 tablet_bootstrap.cc:492] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:34.366667 29264 tablet_bootstrap.cc:492] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:34.367183 29264 ts_tablet_manager.cc:1403] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.139s	user 0.015s	sys 0.002s
I20260709 13:03:34.367588 29532 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.369684 29264 raft_consensus.cc:359] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.370332 29264 raft_consensus.cc:385] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.370600 29264 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.371598 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:34.373066 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:34.372974 29220 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.374248 29147 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.374117 29264 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.377041 29532 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:34.382041 29264 ts_tablet_manager.cc:1434] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 13:03:34.384702 29264 tablet_bootstrap.cc:492] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:34.401576 29262 tablet_bootstrap.cc:654] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.402338 29141 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:03:34.402649 29147 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:34.405445 29574 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:34.412796 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:34.412607 29039 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:34.417619 29263 tablet_bootstrap.cc:654] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.418737 29608 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:34.419278 29574 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:34.419827 29574 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:34.426373 29574 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.437089 29220 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:34.443233 29264 tablet_bootstrap.cc:654] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.465437 29532 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:34.476630 29263 tablet_bootstrap.cc:492] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:34.477125 29263 ts_tablet_manager.cc:1403] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.123s	user 0.010s	sys 0.007s
I20260709 13:03:34.481403 29263 raft_consensus.cc:359] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.482100 29263 raft_consensus.cc:385] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.482379 29263 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.483017 29263 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.485030 29263 ts_tablet_manager.cc:1434] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:03:34.485877 29263 tablet_bootstrap.cc:492] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:34.492660 29262 tablet_bootstrap.cc:492] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:34.493266 29262 ts_tablet_manager.cc:1403] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.222s	user 0.016s	sys 0.024s
I20260709 13:03:34.520953 29263 tablet_bootstrap.cc:654] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.521178 29262 raft_consensus.cc:359] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.521883 29262 raft_consensus.cc:385] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.522158 29262 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.523725 29262 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.525962 29262 ts_tablet_manager.cc:1434] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 13:03:34.525518 28971 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:34.526957 29262 tablet_bootstrap.cc:492] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:34.527144 28971 catalog_manager.cc:5697] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:34.538167 29264 tablet_bootstrap.cc:492] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:34.538601 29264 ts_tablet_manager.cc:1403] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.154s	user 0.018s	sys 0.005s
I20260709 13:03:34.539216 29605 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.539670 29605 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.541080 29264 raft_consensus.cc:359] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.541841 29264 raft_consensus.cc:385] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.543596 29264 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.544348 29264 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.541967 29605 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:34.547537 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:34.548244 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:34.549576 29111 leader_election.cc:343] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.550748 29264 ts_tablet_manager.cc:1434] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:03:34.551832 29264 tablet_bootstrap.cc:492] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
W20260709 13:03:34.553869 29113 leader_election.cc:343] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.554253 29113 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:34.555754 29635 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:34.578193 29262 tablet_bootstrap.cc:654] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.579805 29263 tablet_bootstrap.cc:492] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:34.580090 29532 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:34.580314 29605 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.580350 29263 ts_tablet_manager.cc:1403] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.095s	user 0.016s	sys 0.010s
I20260709 13:03:34.581149 29605 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.583187 29605 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:34.586659 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:34.592844 29113 leader_election.cc:343] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.583863 29263 raft_consensus.cc:359] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.593742 29263 raft_consensus.cc:385] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.594375 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:34.599073 29263 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.599893 29263 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.598673 29622 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.595711 29262 tablet_bootstrap.cc:492] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:34.611845 29262 ts_tablet_manager.cc:1403] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.085s	user 0.008s	sys 0.010s
I20260709 13:03:34.611658 29622 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.613040 29264 tablet_bootstrap.cc:654] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.613862 29622 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:34.615192 29262 raft_consensus.cc:359] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.616007 29262 raft_consensus.cc:385] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.616657 29263 ts_tablet_manager.cc:1434] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.002s
I20260709 13:03:34.616909 29262 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.618458 29263 tablet_bootstrap.cc:492] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
W20260709 13:03:34.619779 29111 leader_election.cc:343] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.620386 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:34.621903 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:34.623307 29038 leader_election.cc:343] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.625881 29605 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:34.632493 29263 tablet_bootstrap.cc:654] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.638656 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:34.642609 29147 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:34.644495 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:34.645875 29622 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:34.646368 29622 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.646797 29622 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.659121 29622 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.659741 29220 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.660566 29262 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.662727 29262 ts_tablet_manager.cc:1434] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.051s	user 0.003s	sys 0.002s
I20260709 13:03:34.664361 29262 tablet_bootstrap.cc:492] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:34.668591 29614 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:34.670105 29262 tablet_bootstrap.cc:654] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.679279 29622 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:34.679836 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:34.677480 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:03:34.682324 29038 leader_election.cc:343] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.681232 29146 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.727870 29264 tablet_bootstrap.cc:492] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:34.731102 29263 tablet_bootstrap.cc:492] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:34.731443 29264 ts_tablet_manager.cc:1403] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.180s	user 0.025s	sys 0.032s
I20260709 13:03:34.731665 29263 ts_tablet_manager.cc:1403] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.113s	user 0.010s	sys 0.037s
I20260709 13:03:34.737332 29264 raft_consensus.cc:359] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.738059 29264 raft_consensus.cc:385] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.738343 29264 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.738950 29264 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.743219 29263 raft_consensus.cc:359] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.743873 29263 raft_consensus.cc:385] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.744160 29263 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.744745 29263 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.745770 29146 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:34.748097 29263 ts_tablet_manager.cc:1434] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:03:34.748184 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:34.749181 29263 tablet_bootstrap.cc:492] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:34.749241 29479 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:34.750067 29479 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:34.750870 29479 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.770128 29264 ts_tablet_manager.cc:1434] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.038s	user 0.001s	sys 0.006s
I20260709 13:03:34.773808 29622 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.774369 29622 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.777849 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:34.778651 29146 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:34.781790 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:34.786396 29479 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:34.786756 29479 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.787086 29479 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.787360 29263 tablet_bootstrap.cc:654] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.788514 29264 tablet_bootstrap.cc:492] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:34.796898 28968 catalog_manager.cc:5697] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:34.800259 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:34.795130 29622 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:34.804382 29038 leader_election.cc:343] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.805199 29262 tablet_bootstrap.cc:492] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:34.805665 29262 ts_tablet_manager.cc:1403] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.141s	user 0.014s	sys 0.010s
I20260709 13:03:34.807164 29479 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.809233 29262 raft_consensus.cc:359] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.809962 29262 raft_consensus.cc:385] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.810231 29262 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.810382 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:34.811033 29146 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.810905 29262 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:34.814111 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:34.815534 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:03:34.821404 29038 leader_election.cc:343] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.826232 29264 tablet_bootstrap.cc:654] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.830579 29543 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.831076 29543 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.836493 29479 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:34.837937 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:34.838609 29067 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:34.839861 29111 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:34.840669 29605 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:34.841032 29605 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.841137 29146 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:34.841396 29605 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.843401 29262 ts_tablet_manager.cc:1434] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.001s
I20260709 13:03:34.844362 29262 tablet_bootstrap.cc:492] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:34.848614 29262 tablet_bootstrap.cc:654] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.848941 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:34.850620 29622 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:34.851226 29622 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:34.851848 29543 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:34.852066 29622 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.856241 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:34.856740 29263 tablet_bootstrap.cc:492] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:34.857167 29263 ts_tablet_manager.cc:1403] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.108s	user 0.020s	sys 0.006s
I20260709 13:03:34.857272 29264 tablet_bootstrap.cc:492] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:34.857748 29264 ts_tablet_manager.cc:1403] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.074s	user 0.011s	sys 0.002s
W20260709 13:03:34.858853 29113 leader_election.cc:343] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.859704 29263 raft_consensus.cc:359] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.860360 29264 raft_consensus.cc:359] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.861037 29264 raft_consensus.cc:385] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.861305 29264 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:34.861789 29605 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.862638 29264 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.865536 29264 ts_tablet_manager.cc:1434] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:03:34.866509 29264 tablet_bootstrap.cc:492] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:34.866612 29605 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:34.868033 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:34.860388 29263 raft_consensus.cc:385] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.870158 29263 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
W20260709 13:03:34.870820 29113 leader_election.cc:343] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:34.871610 29263 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.878985 29220 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.877223 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:34.881784 29067 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.884152 29263 ts_tablet_manager.cc:1434] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:03:34.888381 29622 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:34.889309 29067 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:34.888943 28968 catalog_manager.cc:5697] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:34.873941 29605 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.890764 29111 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:34.892891 29606 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:34.893368 29606 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:34.894124 29606 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.895071 29264 tablet_bootstrap.cc:654] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.896247 29262 tablet_bootstrap.cc:492] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:34.896727 29262 ts_tablet_manager.cc:1403] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.011s
I20260709 13:03:34.899904 29263 tablet_bootstrap.cc:492] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:34.896834 29605 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.902230 29605 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:34.897845 29622 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:03:34.904145 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:34.903646 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:34.909716 29066 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:34.929018 29111 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:34.926391 29609 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.927702 29569 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:34.938145 29609 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.947162 29569 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.949101 29609 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:34.949960 29569 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:34.952414 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:34.953356 29067 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:34.958544 29604 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:34.959004 29604 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.959333 29604 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:34.962301 29217 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:34.945204 29263 tablet_bootstrap.cc:654] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:34.968256 29185 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:34.972149 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:34.975685 29262 raft_consensus.cc:359] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:34.974051 29569 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:34.977042 29262 raft_consensus.cc:385] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:34.980335 29262 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
W20260709 13:03:34.989125 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60185.8us
I20260709 13:03:34.980304 29569 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:34.996541 29569 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.059814 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:35.099992 28969 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:35.099314 29262 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.105245 29141 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:35.115579 29262 ts_tablet_manager.cc:1434] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.219s	user 0.038s	sys 0.070s
I20260709 13:03:35.117705 29262 tablet_bootstrap.cc:492] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:35.121508 29264 tablet_bootstrap.cc:492] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:35.122179 29264 ts_tablet_manager.cc:1403] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.256s	user 0.032s	sys 0.043s
I20260709 13:03:35.111299 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:35.130328 29263 tablet_bootstrap.cc:492] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:35.131597 29263 ts_tablet_manager.cc:1403] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.233s	user 0.020s	sys 0.001s
I20260709 13:03:35.137676 29569 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.148383 29264 raft_consensus.cc:359] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.149389 29264 raft_consensus.cc:385] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.149750 29264 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.150647 29264 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.154258 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:35.155049 29070 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.139743 29569 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:35.158053 29604 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.161046 29147 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:03:35.165500 29262 tablet_bootstrap.cc:654] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.155174 29264 ts_tablet_manager.cc:1434] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.001s
I20260709 13:03:35.170311 29188 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:35.173264 29264 tablet_bootstrap.cc:492] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:35.179495 29263 raft_consensus.cc:359] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.180248 29263 raft_consensus.cc:385] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.180497 29263 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.182334 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:35.183046 29146 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:03:35.185928 29263 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.187602 29672 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.227598 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:35.236884 29070 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
W20260709 13:03:35.241719 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 61286.1us
I20260709 13:03:35.203789 29604 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.249915 29185 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:03:35.257683 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 75284.1us
I20260709 13:03:35.246724 29614 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.275672 29673 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:35.276813 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:03:35.300027 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 45750.5us
I20260709 13:03:35.312728 29263 ts_tablet_manager.cc:1434] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.181s	user 0.045s	sys 0.038s
I20260709 13:03:35.324815 29264 tablet_bootstrap.cc:654] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.279399 29614 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.308179 29673 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:35.331280 29673 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.337334 29614 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.337709 29262 tablet_bootstrap.cc:492] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:35.338157 29262 ts_tablet_manager.cc:1403] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.221s	user 0.016s	sys 0.009s
I20260709 13:03:35.342279 29262 raft_consensus.cc:359] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.342988 29262 raft_consensus.cc:385] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.343343 29262 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.340059 29689 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.344060 29262 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.347131 29263 tablet_bootstrap.cc:492] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:35.344201 29689 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.343199 29672 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.362951 29262 ts_tablet_manager.cc:1434] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 13:03:35.366859 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:35.376268 29544 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:35.377908 29038 leader_election.cc:343] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.380896 29689 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.404960 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:35.407135 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:35.409811 29113 leader_election.cc:343] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.409740 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:35.410741 29140 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:35.420269 29262 tablet_bootstrap.cc:492] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:35.420388 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:35.421098 29216 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.421414 29678 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:35.424731 29264 tablet_bootstrap.cc:492] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:35.425280 29264 ts_tablet_manager.cc:1403] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.252s	user 0.013s	sys 0.007s
W20260709 13:03:35.425657 29111 leader_election.cc:343] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.425531 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:35.426030 29111 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:35.427078 29668 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.426950 29604 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.425460 29678 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.430603 29678 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.430431 29614 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:35.437628 29635 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:35.446244 29113 consensus_peers.cc:597] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:35.463147 29263 tablet_bootstrap.cc:654] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.467049 29544 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:35.456257 29678 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.471864 29262 tablet_bootstrap.cc:654] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.493635 29544 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:35.492583 28969 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:35.496357 29264 raft_consensus.cc:359] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.497084 29264 raft_consensus.cc:385] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.497361 29264 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.499186 29264 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.551450 29604 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.552181 29604 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.557778 29604 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.560917 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:35.567695 29111 leader_election.cc:343] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.569475 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:35.570209 29143 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.579079 29532 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.579775 29532 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.610196 29143 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:35.614400 29678 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.620267 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:35.619822 29264 ts_tablet_manager.cc:1434] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.194s	user 0.019s	sys 0.012s
I20260709 13:03:35.621263 29532 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.646387 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:35.654381 29264 tablet_bootstrap.cc:492] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:35.643694 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:03:35.665004 29039 leader_election.cc:343] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.665673 29532 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:35.666178 29532 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:35.667016 29532 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.667222 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:35.702793 29264 tablet_bootstrap.cc:654] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.684900 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:03:35.713177 29113 leader_election.cc:343] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.713840 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:35.715116 29668 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.721150 29263 tablet_bootstrap.cc:492] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:35.721661 29263 ts_tablet_manager.cc:1403] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.375s	user 0.011s	sys 0.013s
I20260709 13:03:35.724701 29263 raft_consensus.cc:359] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.725450 29263 raft_consensus.cc:385] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.725769 29263 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.731719 29263 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.733914 29263 ts_tablet_manager.cc:1434] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:03:35.734846 29263 tablet_bootstrap.cc:492] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:35.721203 29262 tablet_bootstrap.cc:492] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:35.736994 29262 ts_tablet_manager.cc:1403] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.317s	user 0.044s	sys 0.086s
W20260709 13:03:35.721683 29038 leader_election.cc:343] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.737836 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:35.740707 29532 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.745812 28969 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:35.742667 29038 leader_election.cc:343] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.752766 29263 tablet_bootstrap.cc:654] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.769614 29264 tablet_bootstrap.cc:492] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:35.770131 29264 ts_tablet_manager.cc:1403] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.116s	user 0.010s	sys 0.021s
I20260709 13:03:35.771183 29544 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.771740 29544 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.776394 29678 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:35.776979 29678 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.777467 29264 raft_consensus.cc:359] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.778149 29264 raft_consensus.cc:385] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.778399 29264 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.779016 29678 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.775691 29683 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:35.779026 29264 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.781615 29262 raft_consensus.cc:359] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.782315 29262 raft_consensus.cc:385] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.782585 29262 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.783239 29262 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.785032 29264 ts_tablet_manager.cc:1434] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.005s
I20260709 13:03:35.785420 29262 ts_tablet_manager.cc:1434] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.001s
I20260709 13:03:35.786058 29264 tablet_bootstrap.cc:492] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:35.786442 29262 tablet_bootstrap.cc:492] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:35.793131 29263 tablet_bootstrap.cc:492] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:35.793654 29263 ts_tablet_manager.cc:1403] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.059s	user 0.010s	sys 0.000s
I20260709 13:03:35.797766 29264 tablet_bootstrap.cc:654] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.796394 29263 raft_consensus.cc:359] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.799201 29263 raft_consensus.cc:385] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.799476 29263 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.800271 29263 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.802835 29544 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.804576 29262 tablet_bootstrap.cc:654] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.802544 29263 ts_tablet_manager.cc:1434] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:03:35.813714 29263 tablet_bootstrap.cc:492] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:35.817029 29262 tablet_bootstrap.cc:492] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:35.827431 29263 tablet_bootstrap.cc:654] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.828020 29262 ts_tablet_manager.cc:1403] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.000s
I20260709 13:03:35.833590 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:35.834056 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:35.834409 29213 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:35.834703 29143 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:03:35.838256 29683 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.838754 29683 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.831137 29262 raft_consensus.cc:359] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.839819 29216 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.840051 29262 raft_consensus.cc:385] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.840735 29262 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.840574 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:35.841382 29143 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:35.846668 29264 tablet_bootstrap.cc:492] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:35.847164 29264 ts_tablet_manager.cc:1403] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.061s	user 0.008s	sys 0.024s
I20260709 13:03:35.841426 29262 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.842767 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:35.844380 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:35.849522 29262 ts_tablet_manager.cc:1434] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:03:35.849828 29479 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:35.850226 29479 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:35.850500 29262 tablet_bootstrap.cc:492] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:35.850636 29479 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:35.858467 29264 raft_consensus.cc:359] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.859215 29264 raft_consensus.cc:385] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.859510 29264 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.859303 29683 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:35.863785 29479 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.866533 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:35.868604 29038 leader_election.cc:343] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.870846 29264 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.873975 29263 tablet_bootstrap.cc:492] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:35.880409 29263 ts_tablet_manager.cc:1403] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.007s
I20260709 13:03:35.880760 29262 tablet_bootstrap.cc:654] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.877059 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:35.878232 29039 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:35.883634 29264 ts_tablet_manager.cc:1434] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:03:35.884274 29532 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:35.886376 29532 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.886739 29532 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.887142 29479 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.890205 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:35.894212 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:35.894610 29264 tablet_bootstrap.cc:492] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:35.897423 29263 raft_consensus.cc:359] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.898129 29263 raft_consensus.cc:385] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.894235 29685 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:35.898393 29263 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.895275 29532 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.900918 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:35.901798 29213 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:03:35.901983 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:35.903399 29683 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:35.905139 29263 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.907704 29263 ts_tablet_manager.cc:1434] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
I20260709 13:03:35.908237 29264 tablet_bootstrap.cc:654] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.908776 29263 tablet_bootstrap.cc:492] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:35.909384 29532 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.912288 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:35.912900 29148 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.916124 29262 tablet_bootstrap.cc:492] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:35.916628 29262 ts_tablet_manager.cc:1403] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.066s	user 0.011s	sys 0.001s
I20260709 13:03:35.917827 29263 tablet_bootstrap.cc:654] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.928679 29262 raft_consensus.cc:359] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.929550 29263 tablet_bootstrap.cc:492] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:35.929692 29262 raft_consensus.cc:385] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.930224 29262 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.930342 29263 ts_tablet_manager.cc:1403] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 13:03:35.933205 29263 raft_consensus.cc:359] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.933974 29263 raft_consensus.cc:385] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.934284 29263 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.931131 29262 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.935274 29606 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.935014 29263 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.934372 29264 tablet_bootstrap.cc:492] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:35.935804 29606 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.936578 29264 ts_tablet_manager.cc:1403] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.015s
I20260709 13:03:35.942533 29263 ts_tablet_manager.cc:1434] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:03:35.944000 29263 tablet_bootstrap.cc:492] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:35.944913 29264 raft_consensus.cc:359] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.945704 29264 raft_consensus.cc:385] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:35.946007 29264 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:35.942587 29262 ts_tablet_manager.cc:1434] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 13:03:35.946745 29264 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.948594 29262 tablet_bootstrap.cc:492] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:35.949215 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:35.949540 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:35.949889 29066 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
W20260709 13:03:35.954838 29113 leader_election.cc:343] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:35.963979 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:35.955098 29606 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.967468 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:35.969341 29263 tablet_bootstrap.cc:654] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:35.962307 29148 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:35.972155 29668 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:35.972715 29668 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.973131 29668 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.973308 29039 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:35.976678 29264 ts_tablet_manager.cc:1434] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.040s	user 0.003s	sys 0.006s
I20260709 13:03:35.972087 29142 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:03:35.976147 29614 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.982632 29614 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.978386 29678 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:35.985427 29678 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:35.986117 29614 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:35.986972 29678 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.988086 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:35.988773 29148 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:35.990679 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:35.991297 29685 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:35.991894 29532 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:35.992300 29532 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:35.992630 29532 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:35.995205 29264 tablet_bootstrap.cc:492] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:35.993060 29685 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:35.995810 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:03:36.005631 29038 leader_election.cc:343] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.003731 29479 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:36.022408 29672 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.024287 29672 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.026443 29672 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:36.027619 29479 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:36.028746 29479 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.058561 29668 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.067487 29668 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:36.070649 29532 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:36.076900 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 81171.7us
I20260709 13:03:36.078462 29262 tablet_bootstrap.cc:654] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.078646 29532 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:36.082391 29263 tablet_bootstrap.cc:492] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:36.082562 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:36.082875 29263 ts_tablet_manager.cc:1403] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.139s	user 0.018s	sys 0.005s
I20260709 13:03:36.083297 29073 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:36.084190 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 78610.7us
I20260709 13:03:36.098961 29073 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:36.158022 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:36.171103 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:36.171828 29140 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:36.099668 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:36.147951 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:36.177546 29143 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:36.179741 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:36.082988 29685 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:36.156821 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:36.109714 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:36.165612 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:36.165689 29714 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.186779 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:36.199791 29263 raft_consensus.cc:359] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.183001 29606 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:36.201597 29263 raft_consensus.cc:385] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.194332 29142 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.182722 29214 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:36.211484 29263 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.226941 29262 tablet_bootstrap.cc:492] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:36.229399 29262 ts_tablet_manager.cc:1403] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.281s	user 0.022s	sys 0.017s
I20260709 13:03:36.225965 29264 tablet_bootstrap.cc:654] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.225637 29263 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.229385 29067 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:36.248230 29262 raft_consensus.cc:359] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.248971 29262 raft_consensus.cc:385] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.249326 29262 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.256214 29262 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.228677 29606 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:03:36.228536 29681 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:36.201572 29714 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:36.264132 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 65625.9us
I20260709 13:03:36.265144 29263 ts_tablet_manager.cc:1434] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.182s	user 0.010s	sys 0.005s
W20260709 13:03:36.269078 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 79634.8us
I20260709 13:03:36.266881 29678 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.292318 29606 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.282384 29678 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.300578 29262 ts_tablet_manager.cc:1434] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.071s	user 0.019s	sys 0.006s
I20260709 13:03:36.308269 29262 tablet_bootstrap.cc:492] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:36.300598 29678 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:36.327133 29714 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:36.335326 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:36.336995 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:36.337636 29219 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:36.338843 29262 tablet_bootstrap.cc:654] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.335976 29146 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:36.320794 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:36.356307 29038 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:36.329051 29142 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:36.360464 29219 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.361018 29713 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:36.361256 29039 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:36.363130 29713 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:36.363485 29713 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.363153 29719 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:36.363886 29713 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.364480 29681 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.364888 29681 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:36.378934 29113 leader_election.cc:343] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.379022 29185 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:36.381258 29672 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:36.375428 29718 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:36.385509 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:36.386160 29140 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:36.387771 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:36.365509 29719 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:36.389009 29719 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:36.390545 29038 leader_election.cc:343] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.383425 29713 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:36.462128 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 100934us
I20260709 13:03:36.492995 29713 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:36.524770 29667 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.525313 29262 tablet_bootstrap.cc:492] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:36.525763 29262 ts_tablet_manager.cc:1403] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.224s	user 0.024s	sys 0.028s
W20260709 13:03:36.527549 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 109585us
I20260709 13:03:36.530714 29719 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.534915 29264 tablet_bootstrap.cc:492] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:36.535483 29264 ts_tablet_manager.cc:1403] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.551s	user 0.026s	sys 0.012s
I20260709 13:03:36.525350 29667 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.539811 29667 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:36.543591 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:36.554287 29724 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.559269 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:36.554906 29724 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:36.564857 29113 leader_election.cc:343] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.557099 29264 raft_consensus.cc:359] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.566434 29264 raft_consensus.cc:385] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.567085 29264 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
W20260709 13:03:36.567783 29038 leader_election.cc:343] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:36.568161 29264 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.582293 29264 ts_tablet_manager.cc:1434] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.046s	user 0.004s	sys 0.004s
I20260709 13:03:36.581851 29262 raft_consensus.cc:359] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.583547 29262 raft_consensus.cc:385] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.583870 29262 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.585067 29262 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:36.583066 29038 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:36.588074 29263 tablet_bootstrap.cc:492] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:36.580672 29719 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:36.595597 28971 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:36.597034 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:36.597918 29146 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.600708 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:36.600805 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:36.601402 29216 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.601534 29142 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:36.605053 28971 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 2 to 3, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 3 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:36.607163 29264 tablet_bootstrap.cc:492] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:36.612661 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:36.620385 29724 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:36.618837 29262 ts_tablet_manager.cc:1434] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 13:03:36.644927 29262 tablet_bootstrap.cc:492] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:36.633968 29681 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.657752 29263 tablet_bootstrap.cc:654] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.659852 29146 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:36.666290 29142 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:36.668761 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:36.669258 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:36.669631 29146 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:03:36.670476 29067 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:36.672204 29264 tablet_bootstrap.cc:654] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.665829 29217 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:36.669691 29681 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:36.677397 29216 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:36.679816 29262 tablet_bootstrap.cc:654] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:36.681391 29038 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:36.685173 29479 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:36.685847 29479 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:36.686781 29479 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.687297 29719 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:36.690392 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:36.691263 29143 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:36.722290 29667 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.722914 29667 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.730142 29722 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:36.730885 29263 tablet_bootstrap.cc:492] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:36.731638 29263 ts_tablet_manager.cc:1403] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.144s	user 0.012s	sys 0.018s
I20260709 13:03:36.730885 29722 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.772922 29722 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:36.796859 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:36.798591 29713 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:36.808657 29263 raft_consensus.cc:359] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.810417 29263 raft_consensus.cc:385] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:36.810891 29263 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:36.811947 29263 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:36.865278 29667 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:36.991829 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:36.999334 29264 tablet_bootstrap.cc:492] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:37.090781 29264 ts_tablet_manager.cc:1403] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.491s	user 0.019s	sys 0.020s
I20260709 13:03:37.090873 29070 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:03:37.005244 29262 tablet_bootstrap.cc:492] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:37.026292 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:37.094569 29073 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:37.094521 29262 ts_tablet_manager.cc:1403] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.450s	user 0.028s	sys 0.031s
I20260709 13:03:37.095399 28968 catalog_manager.cc:5697] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.098168 29262 raft_consensus.cc:359] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.098826 29262 raft_consensus.cc:385] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.099269 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:37.119283 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:37.122109 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:37.122645 29264 raft_consensus.cc:359] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.123481 29264 raft_consensus.cc:385] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.123804 29264 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.124481 29264 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.099086 29262 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.031273 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:03:37.126574 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 95393.8us
I20260709 13:03:37.034497 29142 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:37.040719 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:37.083426 29713 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:37.127743 29069 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:37.128433 29264 ts_tablet_manager.cc:1434] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:03:37.128346 29720 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:37.128661 29713 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.130501 29264 tablet_bootstrap.cc:492] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:37.133095 29262 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.135115 29217 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.132938 29667 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:37.136270 29667 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.136629 29667 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.141595 29606 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:37.143661 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:37.144439 29543 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:37.144801 29543 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:37.145138 29543 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:37.103379 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 277551us
I20260709 13:03:37.150986 29039 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:37.149582 29714 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:37.152104 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:37.161026 29724 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:03:37.162379 29113 leader_election.cc:343] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:37.164893 29262 ts_tablet_manager.cc:1434] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.000s
I20260709 13:03:37.166143 29683 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.166579 29683 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.167798 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:37.176196 29668 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:37.178612 29730 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:37.181878 29683 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:37.184370 29263 ts_tablet_manager.cc:1434] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.452s	user 0.008s	sys 0.003s
I20260709 13:03:37.185513 29263 tablet_bootstrap.cc:492] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:37.187345 29262 tablet_bootstrap.cc:492] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:37.192196 29614 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:37.192807 29614 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:37.193630 29614 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.202625 29543 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.204149 29264 tablet_bootstrap.cc:654] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.214274 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:03:37.217892 29113 leader_election.cc:343] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:37.182174 29667 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:37.229395 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 53006.6us
I20260709 13:03:37.231496 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:37.232234 29146 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:37.211687 29543 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:37.237211 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60586.6us
I20260709 13:03:37.221908 29733 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:03:37.261163 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:37.261827 29070 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.273067 29667 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:37.335605 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:37.345644 29262 tablet_bootstrap.cc:654] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.356457 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:37.361361 29263 tablet_bootstrap.cc:654] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:37.348529 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:37.368659 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:37.376771 29065 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.378690 29729 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:37.382200 29678 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:03:37.377679 29070 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:37.393740 29264 tablet_bootstrap.cc:492] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
W20260709 13:03:37.393923 29113 leader_election.cc:343] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:37.394258 29264 ts_tablet_manager.cc:1403] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.264s	user 0.008s	sys 0.011s
I20260709 13:03:37.417255 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:37.421876 29667 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:03:37.432694 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52479.8us
W20260709 13:03:37.445636 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:37.431249 29667 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:03:37.451819 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 72212.6us
I20260709 13:03:37.473062 29667 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.480590 29065 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:37.483074 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:37.497488 29264 raft_consensus.cc:359] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.498364 29264 raft_consensus.cc:385] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.498621 29264 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.499568 29264 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.510524 29728 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:37.516752 29264 ts_tablet_manager.cc:1434] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.122s	user 0.006s	sys 0.001s
I20260709 13:03:37.495899 29689 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:37.519613 29689 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:03:37.530241 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 93672.4us
W20260709 13:03:37.531330 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 78481.8us
I20260709 13:03:37.517403 29728 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.538105 29689 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.537936 29763 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:03:37.609242 29263 tablet_bootstrap.cc:492] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:37.609747 29263 ts_tablet_manager.cc:1403] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.424s	user 0.019s	sys 0.010s
I20260709 13:03:37.625275 29262 tablet_bootstrap.cc:492] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:37.625779 29262 ts_tablet_manager.cc:1403] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.439s	user 0.021s	sys 0.002s
I20260709 13:03:37.617383 29729 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:37.673393 29262 raft_consensus.cc:359] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.675557 29262 raft_consensus.cc:385] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.675832 29262 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.676497 29262 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:37.678938 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48054us
I20260709 13:03:37.682798 29262 ts_tablet_manager.cc:1434] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 13:03:37.711989 28969 catalog_manager.cc:5697] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.714246 29728 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:37.714066 28969 catalog_manager.cc:5697] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:37.703251 29263 raft_consensus.cc:359] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.716238 29263 raft_consensus.cc:385] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:37.716763 29263 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:37.717729 29263 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.719882 29263 ts_tablet_manager.cc:1434] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.110s	user 0.004s	sys 0.001s
W20260709 13:03:37.714972 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 78191.5us
I20260709 13:03:37.753226 29733 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:37.768350 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 46513.4us
I20260709 13:03:37.753786 29733 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:37.773146 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:37.783519 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:37.958701 29113 leader_election.cc:343] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:37.825223 29169 debug-util.cc:398] Leaking SignalData structure 0x7b08001f7420 after lost signal to thread 27937
W20260709 13:03:38.008251 29169 debug-util.cc:398] Leaking SignalData structure 0x7b080013c6e0 after lost signal to thread 29005
I20260709 13:03:37.956426 29724 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.021338 29263 tablet_bootstrap.cc:492] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:37.979377 29067 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 0.
I20260709 13:03:38.030901 29262 tablet_bootstrap.cc:492] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:38.016423 29724 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.039127 29724 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:38.042132 29479 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:38.044551 29733 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:38.051482 29783 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:38.052309 29783 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.016948 29668 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:38.079929 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 330737us
W20260709 13:03:38.079234 29169 debug-util.cc:398] Leaking SignalData structure 0x7b08002905c0 after lost signal to thread 29097
I20260709 13:03:38.074899 29668 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.106446 29713 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:03:38.115525 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:38.137756 29668 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:38.140713 29169 debug-util.cc:398] Leaking SignalData structure 0x7b0800295780 after lost signal to thread 29172
I20260709 13:03:38.147617 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:38.148327 29213 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:03:38.154448 29753 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:38.154994 29753 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.157598 29262 tablet_bootstrap.cc:654] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.158918 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:03:38.170300 29169 debug-util.cc:398] Leaking SignalData structure 0x7b080022e880 after lost signal to thread 29246
I20260709 13:03:38.203284 29783 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:38.227779 29218 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.210556 29673 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:38.225350 29737 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.273525 29263 tablet_bootstrap.cc:654] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:38.240842 29737 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.235004 29673 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.352710 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:38.421113 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:38.422049 29147 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:38.429797 29753 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:38.511513 29737 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:38.499645 29673 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:38.537694 29264 tablet_bootstrap.cc:492] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
W20260709 13:03:38.543900 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 363246us
W20260709 13:03:38.546422 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 381257us
I20260709 13:03:38.576253 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:38.593084 29262 tablet_bootstrap.cc:492] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:38.593784 29262 ts_tablet_manager.cc:1403] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.563s	user 0.014s	sys 0.028s
I20260709 13:03:38.607358 29264 tablet_bootstrap.cc:654] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:38.610122 29038 leader_election.cc:343] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:38.618366 29113 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:38.620540 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:38.620628 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:03:38.623162 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:38.628237 29776 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:38.628691 29733 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:38.630663 29753 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:38.643334 29774 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:03:38.628692 29776 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:38.635432 29733 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:38.659705 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:38.666157 29113 consensus_peers.cc:597] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:38.646065 29776 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:38.649463 29753 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.643898 29774 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.678759 29774 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:38.681401 29113 leader_election.cc:343] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.683473 29263 tablet_bootstrap.cc:492] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:38.684059 29263 ts_tablet_manager.cc:1403] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.663s	user 0.023s	sys 0.052s
I20260709 13:03:38.647737 29733 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.686913 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:38.673321 29753 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.714794 29776 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:38.734813 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 103732us
I20260709 13:03:38.739324 29777 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:38.741688 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:38.742368 29777 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:03:38.742441 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:38.742755 29777 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:38.743124 29777 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.746097 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:38.746573 29753 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.748420 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:38.748549 29753 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:38.750133 29777 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.750471 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:38.759291 29262 raft_consensus.cc:359] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.760066 29262 raft_consensus.cc:385] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.760324 29262 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.761859 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:38.761869 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:38.764348 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:38.765059 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:38.765988 29072 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:38.753023 29753 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:38.768301 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:38.771202 29779 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:03:38.777206 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:38.790318 29262 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.811789 29807 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:38.770678 29728 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:38.813750 29753 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.836890 29733 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:03:38.849143 29111 leader_election.cc:343] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.849764 29111 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:38.846930 29812 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:38.845309 29753 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:38.875375 29264 tablet_bootstrap.cc:492] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:38.876065 29264 ts_tablet_manager.cc:1403] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.349s	user 0.030s	sys 0.026s
I20260709 13:03:38.885506 29262 ts_tablet_manager.cc:1434] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.291s	user 0.009s	sys 0.000s
I20260709 13:03:38.889300 29728 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:38.891738 29776 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:38.893440 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:38.893508 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:38.850944 29812 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.897054 29263 raft_consensus.cc:359] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.897804 29263 raft_consensus.cc:385] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.898108 29263 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.898840 29039 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:38.895324 29807 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:38.899533 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:38.898768 29263 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.899737 29807 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.901878 29733 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:38.866225 29635 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:38.904850 29113 leader_election.cc:343] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:38.907363 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:38.909480 29776 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:38.912564 29807 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:38.913365 29812 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:38.914040 29264 raft_consensus.cc:359] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.914912 29264 raft_consensus.cc:385] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:38.918951 29264 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:38.917778 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:38.916347 29263 ts_tablet_manager.cc:1434] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.232s	user 0.008s	sys 0.000s
I20260709 13:03:38.920197 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:38.922256 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:38.922636 29143 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 1: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:03:38.923110 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:38.924865 29263 tablet_bootstrap.cc:492] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:38.925318 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:38.925339 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:38.927747 29144 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:38.928472 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:38.929286 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:38.932453 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:38.933279 29264 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:38.937438 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:38.940274 29753 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:38.949107 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:38.949762 29222 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:38.950695 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:38.954049 29264 ts_tablet_manager.cc:1434] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.078s	user 0.005s	sys 0.003s
I20260709 13:03:38.954475 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:38.953241 29777 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:38.963728 29111 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:38.970067 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:38.975988 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:38.973284 29812 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:38.973809 29667 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:38.981732 29678 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:38.982800 29678 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:38.982959 29798 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:38.983208 29678 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:38.984309 29222 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:38.993202 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:03:38.996948 29733 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:38.998623 29678 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.002308 29262 tablet_bootstrap.cc:492] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Bootstrap starting.
I20260709 13:03:39.002321 29188 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:39.005661 29681 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:39.006984 29678 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:39.010244 28969 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 93) took 1650 ms. Trace:
I20260709 13:03:39.014391 28971 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 98) took 1620 ms. Trace:
I20260709 13:03:39.016415 29188 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:39.017867 29264 tablet_bootstrap.cc:492] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:39.030901 29726 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:39.034673 28968 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 101) took 1775 ms. Trace:
I20260709 13:03:39.020985 29685 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.015662 28971 rpcz_store.cc:276] 0709 13:03:37.394131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.394568 (+   437us) service_pool.cc:224] Handling call
0709 13:03:39.014331 (+1619763us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6846,"reported_tablets":4}
I20260709 13:03:39.035231 28968 rpcz_store.cc:276] 0709 13:03:37.259568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.259855 (+   287us) service_pool.cc:224] Handling call
0709 13:03:37.501982 (+242127us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b5400018954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a77090 00007f7836b15f3a 00007f783c8cc1cc 00007f783c9ca37e 00007f783ae8b2a4 00007f783ae8b232 00007f783ae8b161 00007f783ae8b0dc 00007f783ae8a392 00007f7834e4848c 00007f7834e478c6
0709 13:03:39.034583 (+1532601us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":13923072}
I20260709 13:03:39.042205 29726 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:39.049039 29263 tablet_bootstrap.cc:654] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.044976 29726 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.068527 29723 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.069104 29723 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.074265 29723 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:39.075053 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:39.010644 28969 rpcz_store.cc:276] 0709 13:03:37.359905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:37.360093 (+   188us) service_pool.cc:224] Handling call
0709 13:03:39.010153 (+1650060us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:38.139220 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:38.139849 (+   629us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:38.139878 (+    29us) write_op.cc:435] Acquired schema lock
0709 13:03:38.139900 (+    22us) tablet.cc:662] Decoding operations
0709 13:03:38.140326 (+   426us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:38.140377 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:38.140404 (+    27us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:03:38.170428 (+ 30024us) tablet.cc:701] Row locks acquired
0709 13:03:38.170456 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:03:38.178647 (+  8191us) write_op.cc:270] Start()
0709 13:03:38.203160 (+ 24513us) write_op.cc:276] Timestamp: P: 1783602218176646 usec, L: 0
0709 13:03:38.203201 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:03:38.249039 (+ 45838us) log.cc:844] Serialized 928 byte log entry
0709 13:03:38.581034 (+331995us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b18000eeae0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:03:38.829090 (+248056us) op_driver.cc:464] REPLICATION: finished
0709 13:03:38.945983 (+116893us) write_op.cc:301] APPLY: starting
0709 13:03:38.946061 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:38.946177 (+   116us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:38.946198 (+    21us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:38.950349 (+  4151us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:38.950510 (+   161us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:03:38.950569 (+    59us) write_op.cc:312] APPLY: finished
0709 13:03:38.950751 (+   182us) log.cc:844] Serialized 28 byte log entry
0709 13:03:38.950906 (+   155us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:38.950983 (+    77us) write_op.cc:454] Released schema lock
0709 13:03:38.951023 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:38.951637 (+   614us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:38.951665 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1087,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":114809,"apply.run_cpu_time_us":1683,"apply.run_wall_time_us":5641,"num_ops":2,"prepare.queue_time_us":360881,"prepare.run_cpu_time_us":32326,"prepare.run_wall_time_us":417463,"raft.queue_time_us":106886,"raft.run_cpu_time_us":22794,"raft.run_wall_time_us":177703,"replication_time_us":574911,"spinlock_wait_cycles":12084992,"thread_start_us":755753,"threads_started":4,"wal-append.queue_time_us":303349}]]}
W20260709 13:03:39.086567 29111 leader_election.cc:343] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.062963 28968 catalog_manager.cc:5697] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:03:39.127182 29262 tablet_bootstrap.cc:654] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:39.128364 29819 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:03:39.085667 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:03:39.154093 29820 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 13:03:39.325493 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 159391us
W20260709 13:03:39.326421 29821 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.218s	user 0.001s	sys 0.010s
I20260709 13:03:39.327725 29673 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:39.328281 29673 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.330291 29673 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:39.325695 29264 tablet_bootstrap.cc:654] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:39.341518 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 176031us
W20260709 13:03:39.342118 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 175820us
W20260709 13:03:39.343806 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 115270us
W20260709 13:03:39.357143 29113 leader_election.cc:343] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.357617 29113 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:39.369027 29606 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.422652 29263 tablet_bootstrap.cc:492] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:39.423154 29263 ts_tablet_manager.cc:1403] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.499s	user 0.020s	sys 0.005s
I20260709 13:03:39.424899 29262 tablet_bootstrap.cc:492] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: No bootstrap required, opened a new log
I20260709 13:03:39.425578 29262 ts_tablet_manager.cc:1403] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Time spent bootstrapping tablet: real 0.424s	user 0.022s	sys 0.004s
I20260709 13:03:39.444425 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:03:39.452947 29823 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:03:39.453429 29262 raft_consensus.cc:359] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.454125 29262 raft_consensus.cc:385] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.454385 29262 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.455145 29262 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.456578 29263 raft_consensus.cc:359] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.457311 29263 raft_consensus.cc:385] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.457487 29262 ts_tablet_manager.cc:1434] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 13:03:39.457747 29263 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.458520 29263 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:03:39.469234 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 99854.9us
I20260709 13:03:39.470520 29263 ts_tablet_manager.cc:1434] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 13:03:39.488754 29263 tablet_bootstrap.cc:492] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:39.514847 29770 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.515403 29770 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.517720 29770 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:39.519438 29263 tablet_bootstrap.cc:654] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.520701 29264 tablet_bootstrap.cc:492] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:39.521055 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:39.521168 29264 ts_tablet_manager.cc:1403] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.553s	user 0.029s	sys 0.013s
I20260709 13:03:39.530210 29264 raft_consensus.cc:359] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.536263 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:39.537329 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:39.537850 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:39.540839 29827 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:39.530875 29264 raft_consensus.cc:385] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.543632 29264 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.544269 29264 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.546754 29264 ts_tablet_manager.cc:1434] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:03:39.547742 29264 tablet_bootstrap.cc:492] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1: Bootstrap starting.
I20260709 13:03:39.552990 29139 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:39.554112 29263 tablet_bootstrap.cc:492] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:39.556162 29263 ts_tablet_manager.cc:1403] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.084s	user 0.007s	sys 0.005s
W20260709 13:03:39.556341 29038 leader_election.cc:343] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.558578 29830 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:39.564009 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:39.564726 29830 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:39.565080 29830 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.565411 29830 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.568311 29263 raft_consensus.cc:359] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.569017 29263 raft_consensus.cc:385] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.569293 29263 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.572129 29263 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.572723 29830 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.575089 29830 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:39.577101 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:39.577347 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:39.577983 29139 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.578861 29264 tablet_bootstrap.cc:654] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.580021 29263 ts_tablet_manager.cc:1434] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
W20260709 13:03:39.580389 29038 leader_election.cc:343] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:39.584789 29263 tablet_bootstrap.cc:492] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:39.600955 29263 tablet_bootstrap.cc:654] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.611127 29139 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:03:39.612977 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:39.614254 29782 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:39.614759 29782 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:39.615557 29782 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.626111 29264 tablet_bootstrap.cc:492] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1: No bootstrap required, opened a new log
I20260709 13:03:39.627022 29264 ts_tablet_manager.cc:1403] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1: Time spent bootstrapping tablet: real 0.080s	user 0.019s	sys 0.017s
I20260709 13:03:39.626111 29770 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:03:39.648530 29264 raft_consensus.cc:359] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.649277 29264 raft_consensus.cc:385] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.649554 29264 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.652637 29263 tablet_bootstrap.cc:492] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:39.653131 29263 ts_tablet_manager.cc:1403] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.006s
I20260709 13:03:39.654630 28971 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:39.650190 29264 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.655752 29263 raft_consensus.cc:359] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.660709 29263 raft_consensus.cc:385] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.661024 29263 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.661716 29263 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.657194 28971 catalog_manager.cc:5697] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:39.666992 29263 ts_tablet_manager.cc:1434] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:03:39.668017 29263 tablet_bootstrap.cc:492] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:39.680580 29264 ts_tablet_manager.cc:1434] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1: Time spent starting tablet: real 0.053s	user 0.004s	sys 0.003s
I20260709 13:03:39.680331 28969 catalog_manager.cc:5697] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:39.688097 29263 tablet_bootstrap.cc:654] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.712215 29214 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.760063 29827 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.760666 29827 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.793037 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:39.793926 29065 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:39.792601 29827 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:39.795742 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:39.796278 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:39.796923 29141 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:39.796933 29827 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:39.798846 29827 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:39.799208 29827 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.803957 28971 catalog_manager.cc:5697] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:39.822309 29263 tablet_bootstrap.cc:492] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:39.822831 29263 ts_tablet_manager.cc:1403] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.155s	user 0.034s	sys 0.069s
I20260709 13:03:39.826694 29263 raft_consensus.cc:359] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.827418 29263 raft_consensus.cc:385] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.827674 29263 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.828326 29263 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.830858 29263 ts_tablet_manager.cc:1434] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:03:39.831828 29263 tablet_bootstrap.cc:492] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Bootstrap starting.
I20260709 13:03:39.838233 29263 tablet_bootstrap.cc:654] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:39.840225 29827 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.846702 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:39.847321 29071 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.872040 29071 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:39.873139 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:39.873874 29729 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:39.874099 29720 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:39.874590 29720 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.874366 29729 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:39.875758 29729 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.876775 29720 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:39.878309 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:39.879069 29141 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 1: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:03:39.880287 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:39.882483 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:39.883376 29681 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.893466 29681 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:39.897835 29827 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:39.899395 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:39.900039 29141 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:39.905486 29263 tablet_bootstrap.cc:492] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: No bootstrap required, opened a new log
I20260709 13:03:39.906025 29263 ts_tablet_manager.cc:1403] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent bootstrapping tablet: real 0.074s	user 0.025s	sys 0.029s
I20260709 13:03:39.910717 29263 raft_consensus.cc:359] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.911424 29263 raft_consensus.cc:385] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:39.911696 29263 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Initialized, Role: FOLLOWER
I20260709 13:03:39.918246 29263 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:39.921393 29263 ts_tablet_manager.cc:1434] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.003s
W20260709 13:03:39.926707 29019 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:40.040719 28968 catalog_manager.cc:5697] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.112563 29214 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.118594 29714 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.119632 29714 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:40.145776 29141 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:40.159861 29685 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.161481 29685 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:40.169845 29714 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:40.176280 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:40.177343 29065 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:40.179628 29185 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:40.182355 29681 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:40.178356 29685 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:40.183147 29681 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.183543 29681 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.196345 29720 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:03:40.197355 29720 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:40.201328 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:40.209383 29720 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:40.210945 29681 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:40.210021 29714 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:03:40.213002 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:40.214267 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:40.233593 29148 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 0.
I20260709 13:03:40.216446 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:40.234607 29681 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:40.235443 29733 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:40.236668 29216 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 1: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:03:40.240666 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:03:40.241446 29065 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.252630 29065 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:40.259043 29185 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:03:40.260023 29681 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:40.260713 29681 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:03:40.261871 29681 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:40.278791 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:40.279623 29141 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 0.
I20260709 13:03:40.293525 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:03:40.297506 29039 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:40.298970 29782 raft_consensus.cc:2764] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:03:40.307221 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:40.307934 29142 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:40.303551 29681 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:40.317425 29142 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:03:40.335210 28968 catalog_manager.cc:5697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.345956 29782 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:40.346664 29782 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:40.349254 29782 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:40.354382 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:03:40.362285 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:40.363399 29141 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:03:40.377825 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:40.379503 29719 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:40.379958 29719 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:40.380371 29719 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:40.408887 29719 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:40.412045 29719 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:40.415306 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:03:40.426975 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:03:40.427803 29142 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:40.435213 29681 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.438563 29142 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:03:40.440263 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:03:40.446475 29782 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:40.447019 29782 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:03:40.447935 29827 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:03:40.447896 29782 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:40.448539 29827 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:03:40.451191 29827 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:40.459717 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:03:40.500324 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:03:40.501286 29141 raft_consensus.cc:2368] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3.
I20260709 13:03:40.505759 29188 leader_election.cc:400] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3.
I20260709 13:03:40.506091 29188 leader_election.cc:403] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:40.506717 29681 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:03:40.506609 29827 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:40.517213 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56332.7us
W20260709 13:03:40.519541 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 58108.5us
I20260709 13:03:40.521044 29681 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3.
I20260709 13:03:40.544100 28968 catalog_manager.cc:5697] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:40.701588 29681 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:40.719050 29729 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:40.866032 29140 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:03:40.868633 29719 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:40.891455 29222 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:03:40.893141 29719 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:41.697793 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:41.717329 29862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:41.723178 29863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:41.731777 27934 server_base.cc:1061] running on GCE node
W20260709 13:03:41.732586 29865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:41.738873 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:41.741182 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:41.741533 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602221741509 us; error 0 us; skew 500 ppm
I20260709 13:03:41.742630 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:41.760967 27934 webserver.cc:533] Webserver started at http://127.27.71.132:36793/ using document root <none> and password file <none>
I20260709 13:03:41.761787 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:41.762112 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:41.762571 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:41.764662 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-3-root/instance:
uuid: "7e78ffbcc98242068a874a5d9e341477"
format_stamp: "Formatted at 2026-07-09 13:03:41 on dist-test-slave-mlt7"
I20260709 13:03:41.785542 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.020s	user 0.010s	sys 0.007s
I20260709 13:03:41.804641 29870 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:41.805889 27934 fs_manager.cc:730] Time spent opening block manager: real 0.017s	user 0.008s	sys 0.007s
I20260709 13:03:41.806293 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-3-root
uuid: "7e78ffbcc98242068a874a5d9e341477"
format_stamp: "Formatted at 2026-07-09 13:03:41 on dist-test-slave-mlt7"
I20260709 13:03:41.806677 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:41.958300 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:41.960569 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:41.987243 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:42.001295 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:42.001642 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:42.001981 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:42.002280 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
W20260709 13:03:42.101779 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60229.4us
I20260709 13:03:42.498689 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.132:36041
I20260709 13:03:42.498903 29935 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.132:36041 every 8 connection(s)
I20260709 13:03:42.562369 27934 auto_leader_rebalancer-test.cc:384] add a tserver: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:42.614024 29936 heartbeater.cc:344] Connected to a master server at 127.27.71.190:37537
I20260709 13:03:42.614557 29936 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:42.615552 29936 heartbeater.cc:507] Master 127.27.71.190:37537 requested a full tablet report, sending...
I20260709 13:03:42.621538 28971 ts_manager.cc:194] Registered new tserver with Master: 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:03:42.623517 28971 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52826
I20260709 13:03:43.451982 29214 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:43.470381 29140 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:43.472787 29881 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:43.471290 29073 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:43.476773 29729 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:43.502682 29881 raft_consensus.cc:2955] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:43.513365 29141 raft_consensus.cc:2955] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:43.534240 28971 catalog_manager.cc:5697] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:43.529877 29073 raft_consensus.cc:2955] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:43.546784 29214 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:03:43.550783 29186 consensus_peers.cc:597] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c54509bf16d642138312814fefa3f6dc. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:43.570475 29186 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5d7d7e7e55b46f89b835614737ee91b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:43.571352 29140 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:43.573477 29729 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:43.589147 29073 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:43.592811 29881 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:43.594779 29729 raft_consensus.cc:2955] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:43.607746 29140 raft_consensus.cc:2955] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:43.629274 29936 heartbeater.cc:499] Master 127.27.71.190:37537 was elected leader, sending a full tablet report...
I20260709 13:03:43.648797 29068 raft_consensus.cc:2955] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:43.720229 28969 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index -1 to 3, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 3 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:43.734951 29140 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:43.782524 29214 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:43.783244 29068 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:43.807775 29847 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:43.821931 29812 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:03:43.826721 29112 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c8a2336e86c4846bf4e69ec61ba3fe0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:43.845247 29959 raft_consensus.cc:2955] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:43.874423 28968 catalog_manager.cc:5697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.935631 29068 raft_consensus.cc:2955] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:43.940428 29140 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:43.959736 29214 raft_consensus.cc:2955] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:03:43.965406 29112 consensus_peers.cc:597] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be73fc18460b4609ad4da2946e3acb87. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:43.989845 29218 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:43.992079 29959 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:44.022224 29847 raft_consensus.cc:2955] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:44.030505 29216 raft_consensus.cc:2955] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:03:44.056938 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 90721.8us
I20260709 13:03:44.061254 29071 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:44.088330 29966 ts_tablet_manager.cc:933] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:44.103971 28971 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:44.123888 29965 ts_tablet_manager.cc:933] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:44.156097 29959 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:44.158217 29965 tablet_copy_client.cc:323] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.131:45851
I20260709 13:03:44.178011 29966 tablet_copy_client.cc:323] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.131:45851
I20260709 13:03:44.230011 29232 tablet_copy_service.cc:140] P 6ad9faa1c14444b29c9db8e71f46e2d6: Received BeginTabletCopySession request for tablet c54509bf16d642138312814fefa3f6dc from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:51820)
I20260709 13:03:44.230378 29231 tablet_copy_service.cc:140] P 6ad9faa1c14444b29c9db8e71f46e2d6: Received BeginTabletCopySession request for tablet d5d7d7e7e55b46f89b835614737ee91b from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:51820)
I20260709 13:03:44.230778 29232 tablet_copy_service.cc:161] P 6ad9faa1c14444b29c9db8e71f46e2d6: Beginning new tablet copy session on tablet c54509bf16d642138312814fefa3f6dc from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:51820: session id = 7e78ffbcc98242068a874a5d9e341477-c54509bf16d642138312814fefa3f6dc
I20260709 13:03:44.231004 29972 ts_tablet_manager.cc:933] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:44.236174 29231 tablet_copy_service.cc:161] P 6ad9faa1c14444b29c9db8e71f46e2d6: Beginning new tablet copy session on tablet d5d7d7e7e55b46f89b835614737ee91b from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:51820: session id = 7e78ffbcc98242068a874a5d9e341477-d5d7d7e7e55b46f89b835614737ee91b
I20260709 13:03:44.242832 29231 tablet_copy_source_session.cc:215] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:44.246129 29232 tablet_copy_source_session.cc:215] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:44.245402 29071 raft_consensus.cc:2955] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:44.247041 29966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5d7d7e7e55b46f89b835614737ee91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:44.252136 29965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c54509bf16d642138312814fefa3f6dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:44.258211 29972 tablet_copy_client.cc:323] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.130:35337
I20260709 13:03:44.300535 29965 tablet_copy_client.cc:806] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:44.301368 29965 tablet_copy_client.cc:670] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:44.302412 29966 tablet_copy_client.cc:806] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:44.302963 29966 tablet_copy_client.cc:670] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:44.308492 29158 tablet_copy_service.cc:140] P 730186a8c70f43f2a57d80163bbc37d2: Received BeginTabletCopySession request for tablet 8c8a2336e86c4846bf4e69ec61ba3fe0 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:60830)
I20260709 13:03:44.309068 29158 tablet_copy_service.cc:161] P 730186a8c70f43f2a57d80163bbc37d2: Beginning new tablet copy session on tablet 8c8a2336e86c4846bf4e69ec61ba3fe0 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:60830: session id = 7e78ffbcc98242068a874a5d9e341477-8c8a2336e86c4846bf4e69ec61ba3fe0
I20260709 13:03:44.310117 29966 tablet_copy_client.cc:538] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:44.310757 29965 tablet_copy_client.cc:538] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:44.319571 29158 tablet_copy_source_session.cc:215] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:44.323434 29972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c8a2336e86c4846bf4e69ec61ba3fe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:44.331727 29966 tablet_bootstrap.cc:492] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:44.334623 29965 tablet_bootstrap.cc:492] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:44.402974 29972 tablet_copy_client.cc:806] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:44.403625 29972 tablet_copy_client.cc:670] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:44.409766 29972 tablet_copy_client.cc:538] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:44.437568 29966 tablet_bootstrap.cc:492] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:44.438555 29966 tablet_bootstrap.cc:492] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:44.439242 29966 ts_tablet_manager.cc:1403] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.108s	user 0.031s	sys 0.021s
I20260709 13:03:44.442390 29966 raft_consensus.cc:359] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:44.443223 29966 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:44.443830 29966 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:44.481163 29972 tablet_bootstrap.cc:492] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:44.527945 29966 ts_tablet_manager.cc:1434] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.088s	user 0.010s	sys 0.030s
I20260709 13:03:44.535667 29232 tablet_copy_service.cc:342] P 6ad9faa1c14444b29c9db8e71f46e2d6: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-d5d7d7e7e55b46f89b835614737ee91b received from {username='slave'} at 127.0.0.1:51820
I20260709 13:03:44.536222 29232 tablet_copy_service.cc:434] P 6ad9faa1c14444b29c9db8e71f46e2d6: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-d5d7d7e7e55b46f89b835614737ee91b on tablet d5d7d7e7e55b46f89b835614737ee91b with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:44.538115 29978 ts_tablet_manager.cc:933] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:44.551864 29972 tablet_bootstrap.cc:492] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:44.561069 29972 tablet_bootstrap.cc:492] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:44.561844 29972 ts_tablet_manager.cc:1403] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.081s	user 0.016s	sys 0.025s
I20260709 13:03:44.564883 29972 raft_consensus.cc:359] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:44.565757 29972 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:44.568454 29972 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:44.584233 29972 ts_tablet_manager.cc:1434] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:03:44.586493 29158 tablet_copy_service.cc:342] P 730186a8c70f43f2a57d80163bbc37d2: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-8c8a2336e86c4846bf4e69ec61ba3fe0 received from {username='slave'} at 127.0.0.1:60830
I20260709 13:03:44.587031 29158 tablet_copy_service.cc:434] P 730186a8c70f43f2a57d80163bbc37d2: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-8c8a2336e86c4846bf4e69ec61ba3fe0 on tablet 8c8a2336e86c4846bf4e69ec61ba3fe0 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:44.704957 29978 tablet_copy_client.cc:323] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.130:35337
I20260709 13:03:44.706897 29158 tablet_copy_service.cc:140] P 730186a8c70f43f2a57d80163bbc37d2: Received BeginTabletCopySession request for tablet be73fc18460b4609ad4da2946e3acb87 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:60830)
I20260709 13:03:44.707430 29158 tablet_copy_service.cc:161] P 730186a8c70f43f2a57d80163bbc37d2: Beginning new tablet copy session on tablet be73fc18460b4609ad4da2946e3acb87 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:60830: session id = 7e78ffbcc98242068a874a5d9e341477-be73fc18460b4609ad4da2946e3acb87
I20260709 13:03:44.727507 29158 tablet_copy_source_session.cc:215] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:44.731070 29978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be73fc18460b4609ad4da2946e3acb87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:44.792603 29978 tablet_copy_client.cc:806] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:44.793292 29978 tablet_copy_client.cc:670] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:44.798017 29978 tablet_copy_client.cc:538] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:03:44.792652 29965 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.057s	user 0.004s	sys 0.000s
I20260709 13:03:44.820797 29978 tablet_bootstrap.cc:492] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:44.821993 29965 tablet_bootstrap.cc:492] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:44.833698 29911 raft_consensus.cc:1217] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:44.981987 29965 tablet_bootstrap.cc:492] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:44.982776 29965 ts_tablet_manager.cc:1403] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.648s	user 0.042s	sys 0.042s
I20260709 13:03:44.979532 29910 raft_consensus.cc:1217] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 13:03:44.997296 29965 raft_consensus.cc:359] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:44.998219 29965 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:44.998889 29965 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:45.054234 29965 ts_tablet_manager.cc:1434] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.071s	user 0.000s	sys 0.005s
I20260709 13:03:45.060210 29232 tablet_copy_service.cc:342] P 6ad9faa1c14444b29c9db8e71f46e2d6: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-c54509bf16d642138312814fefa3f6dc received from {username='slave'} at 127.0.0.1:51820
I20260709 13:03:45.060675 29232 tablet_copy_service.cc:434] P 6ad9faa1c14444b29c9db8e71f46e2d6: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-c54509bf16d642138312814fefa3f6dc on tablet c54509bf16d642138312814fefa3f6dc with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:45.161980 29910 raft_consensus.cc:1217] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:45.181836 29978 tablet_bootstrap.cc:492] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:45.182904 29978 tablet_bootstrap.cc:492] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:45.183655 29978 ts_tablet_manager.cc:1403] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.363s	user 0.026s	sys 0.005s
I20260709 13:03:45.192651 29978 raft_consensus.cc:359] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:45.193805 29978 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:45.194633 29978 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:45.197047 29978 ts_tablet_manager.cc:1434] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:03:45.210584 29158 tablet_copy_service.cc:342] P 730186a8c70f43f2a57d80163bbc37d2: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-be73fc18460b4609ad4da2946e3acb87 received from {username='slave'} at 127.0.0.1:60830
I20260709 13:03:45.211031 29158 tablet_copy_service.cc:434] P 730186a8c70f43f2a57d80163bbc37d2: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-be73fc18460b4609ad4da2946e3acb87 on tablet be73fc18460b4609ad4da2946e3acb87 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:45.275571 29960 raft_consensus.cc:1064] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:45.278915 29960 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:45.309281 29910 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:45.311296 29986 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:45.396925 29068 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:45.398656 29969 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:45.406184 29218 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:45.409379 29960 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:45.425951 29949 raft_consensus.cc:1064] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:45.428920 29949 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:45.448894 29847 raft_consensus.cc:2955] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:45.466418 29910 raft_consensus.cc:2955] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:45.498816 28968 catalog_manager.cc:5697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:45.532017 29911 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 3 LEARNER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:45.534178 29985 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:45.581437 29214 raft_consensus.cc:2955] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:45.600706 29909 raft_consensus.cc:1217] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:45.681344 29073 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:45.714634 29881 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:45.760012 29949 raft_consensus.cc:1064] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:45.763319 29949 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:45.808316 29073 raft_consensus.cc:2955] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:45.850190 29910 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:45.853564 29949 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:45.862198 29140 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:45.871260 29729 raft_consensus.cc:2955] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:45.875314 29073 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:45.879832 29068 raft_consensus.cc:2955] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:45.900612 29881 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:45.905709 29143 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:45.938028 29910 raft_consensus.cc:2955] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:45.978824 29996 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:45.982936 28969 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index 3 to 4, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.006904 29142 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:46.044440 29910 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:03:46.051666 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55920.9us
W20260709 13:03:46.053136 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68497.9us
I20260709 13:03:46.056438 29960 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:46.059248 29881 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:46.065385 29993 raft_consensus.cc:2955] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.069803 29910 raft_consensus.cc:2955] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.067802 29217 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:46.084728 29985 raft_consensus.cc:2955] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.089784 29073 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:46.089901 29064 raft_consensus.cc:2955] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.092245 29911 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:46.093902 29949 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:46.099059 29949 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:46.102658 29729 raft_consensus.cc:2955] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.105588 29073 raft_consensus.cc:2955] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.136080 29073 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:46.141680 28957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d5d7d7e7e55b46f89b835614737ee91b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:46.146032 28969 catalog_manager.cc:5697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.152801 28958 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8c8a2336e86c4846bf4e69ec61ba3fe0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:46.157377 28971 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 4 to 5, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New cstate: current_term: 3 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.165396 30008 raft_consensus.cc:1064] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:46.168488 30008 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:46.174510 29986 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:46.176807 29134 tablet_service.cc:1558] Processing DeleteTablet for tablet d5d7d7e7e55b46f89b835614737ee91b with delete_type TABLET_DATA_TOMBSTONED (TS 730186a8c70f43f2a57d80163bbc37d2 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42040
I20260709 13:03:46.198972 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c8a2336e86c4846bf4e69ec61ba3fe0 with delete_type TABLET_DATA_TOMBSTONED (TS 6ad9faa1c14444b29c9db8e71f46e2d6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42954
I20260709 13:03:46.218338 28971 catalog_manager.cc:5697] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.210116 29148 raft_consensus.cc:2955] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:03:46.254248 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50863.9us
W20260709 13:03:46.280391 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 74102.7us
I20260709 13:03:46.306401 30016 tablet_replica.cc:333] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:03:46.382658 29216 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:46.341656 29217 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:46.346333 29073 raft_consensus.cc:2955] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.362639 29064 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:46.367424 29910 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:46.368614 30013 tablet_replica.cc:333] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:03:46.369800 29909 raft_consensus.cc:2955] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.394598 29994 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:03:46.373133 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 161500us
I20260709 13:03:46.294796 29911 raft_consensus.cc:2955] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.407070 29847 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:46.408843 29847 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:46.426759 29148 raft_consensus.cc:2955] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.440874 30016 raft_consensus.cc:2243] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:46.441802 30016 raft_consensus.cc:2272] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:46.491451 29143 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:46.521710 30016 ts_tablet_manager.cc:1916] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:46.568672 29910 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:46.629133 30023 raft_consensus.cc:2955] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.631458 29911 raft_consensus.cc:2955] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.632731 30013 raft_consensus.cc:2243] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:46.636432 30013 raft_consensus.cc:2272] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:46.638669 30014 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:46.640308 30013 ts_tablet_manager.cc:1916] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:46.645397 29217 raft_consensus.cc:2955] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.658133 30016 ts_tablet_manager.cc:1929] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 13:03:46.658725 30016 log.cc:1199] T d5d7d7e7e55b46f89b835614737ee91b P 730186a8c70f43f2a57d80163bbc37d2: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/wals/d5d7d7e7e55b46f89b835614737ee91b
I20260709 13:03:46.670125 29729 raft_consensus.cc:2955] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.680516 28957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c54509bf16d642138312814fefa3f6dc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:46.693358 29909 raft_consensus.cc:2955] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.695447 28969 catalog_manager.cc:5697] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.697471 28969 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.702991 30017 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:46.763710 29145 raft_consensus.cc:2955] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.774047 29071 raft_consensus.cc:2955] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:46.788022 30013 ts_tablet_manager.cc:1929] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:46.801287 30013 log.cc:1199] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/wals/8c8a2336e86c4846bf4e69ec61ba3fe0
I20260709 13:03:46.804633 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet 8c8a2336e86c4846bf4e69ec61ba3fe0 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:46.817605 28958 catalog_manager.cc:5028] TS 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): tablet d5d7d7e7e55b46f89b835614737ee91b (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:46.836779 29059 tablet_service.cc:1558] Processing DeleteTablet for tablet c54509bf16d642138312814fefa3f6dc with delete_type TABLET_DATA_TOMBSTONED (TS a936878f59c840a6bd176a1d2d7347b1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47946
I20260709 13:03:46.848141 29140 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:03:46.929649 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 76282.5us
I20260709 13:03:46.953090 30027 tablet_replica.cc:333] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:03:46.954419 30027 raft_consensus.cc:2243] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:46.955415 30027 raft_consensus.cc:2272] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:46.987195 30011 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:47.033582 30027 ts_tablet_manager.cc:1916] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:03:47.060802 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 206716us
I20260709 13:03:47.067543 29909 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:47.070416 30032 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:03:47.071264 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 121004us
I20260709 13:03:47.076978 29213 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:47.078567 29993 raft_consensus.cc:2955] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:47.081097 30023 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:47.088855 29222 raft_consensus.cc:2955] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:47.091554 29909 raft_consensus.cc:2955] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:47.112885 28958 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet be73fc18460b4609ad4da2946e3acb87 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:47.113530 30027 ts_tablet_manager.cc:1929] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:47.114358 30027 log.cc:1199] T c54509bf16d642138312814fefa3f6dc P a936878f59c840a6bd176a1d2d7347b1: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/wals/c54509bf16d642138312814fefa3f6dc
I20260709 13:03:47.116920 28955 catalog_manager.cc:5028] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): tablet c54509bf16d642138312814fefa3f6dc (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:47.127779 28969 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:47.144917 29059 tablet_service.cc:1558] Processing DeleteTablet for tablet be73fc18460b4609ad4da2946e3acb87 with delete_type TABLET_DATA_TOMBSTONED (TS a936878f59c840a6bd176a1d2d7347b1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47946
I20260709 13:03:47.146077 30027 tablet_replica.cc:333] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:03:47.146943 30027 raft_consensus.cc:2243] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:47.147750 30027 raft_consensus.cc:2272] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:47.176115 30027 ts_tablet_manager.cc:1916] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:47.193431 30027 ts_tablet_manager.cc:1929] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:47.193730 30027 log.cc:1199] T be73fc18460b4609ad4da2946e3acb87 P a936878f59c840a6bd176a1d2d7347b1: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/wals/be73fc18460b4609ad4da2946e3acb87
I20260709 13:03:47.195679 28955 catalog_manager.cc:5028] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): tablet be73fc18460b4609ad4da2946e3acb87 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:48.267261 29071 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:48.290692 29213 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.293186 30018 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:48.300493 29144 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.306298 30019 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.311935 30018 raft_consensus.cc:2955] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.321363 29222 raft_consensus.cc:2955] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.330228 29145 raft_consensus.cc:2955] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.345119 29068 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:48.346909 28970 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:03:48.372386 29037 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3c777774468e477ca1ab0352dff56c9c. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:48.383695 29037 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b27f1b9ed9964227ac908c2897d935e9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:48.384200 29222 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.385402 29145 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.387543 29987 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.388315 30018 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.407282 29987 raft_consensus.cc:2955] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.418635 29222 raft_consensus.cc:2955] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.426441 29145 raft_consensus.cc:2955] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.448860 28969 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.480566 29145 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:03:48.506138 29112 consensus_peers.cc:597] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e96bceabcbf747e0acec327bc1abc729. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:48.510139 29222 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.511539 30031 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.511999 29068 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.513928 30031 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.522807 30030 raft_consensus.cc:2955] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.526489 29222 raft_consensus.cc:2955] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.527076 29071 raft_consensus.cc:2955] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.547093 28968 catalog_manager.cc:5697] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.575719 29068 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:48.591363 29145 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:48.591225 29037 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d2ee8b4c1bf649b9b89ad6243f3890a8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:48.584367 29222 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.600416 30019 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.601353 30018 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.613178 30018 raft_consensus.cc:2955] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.618700 29222 raft_consensus.cc:2955] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.621039 29144 raft_consensus.cc:2955] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.640199 28968 catalog_manager.cc:5697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:48.868330 30054 ts_tablet_manager.cc:933] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:03:48.917268 30056 ts_tablet_manager.cc:933] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:03:48.918301 30054 tablet_copy_client.cc:323] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.129:41631
I20260709 13:03:48.946961 30056 tablet_copy_client.cc:323] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.129:41631
I20260709 13:03:48.962116 29083 tablet_copy_service.cc:140] P a936878f59c840a6bd176a1d2d7347b1: Received BeginTabletCopySession request for tablet 3c777774468e477ca1ab0352dff56c9c from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:40862)
I20260709 13:03:48.962498 29082 tablet_copy_service.cc:140] P a936878f59c840a6bd176a1d2d7347b1: Received BeginTabletCopySession request for tablet b27f1b9ed9964227ac908c2897d935e9 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:40862)
I20260709 13:03:48.962751 29083 tablet_copy_service.cc:161] P a936878f59c840a6bd176a1d2d7347b1: Beginning new tablet copy session on tablet 3c777774468e477ca1ab0352dff56c9c from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:40862: session id = 7e78ffbcc98242068a874a5d9e341477-3c777774468e477ca1ab0352dff56c9c
I20260709 13:03:48.963487 29082 tablet_copy_service.cc:161] P a936878f59c840a6bd176a1d2d7347b1: Beginning new tablet copy session on tablet b27f1b9ed9964227ac908c2897d935e9 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:40862: session id = 7e78ffbcc98242068a874a5d9e341477-b27f1b9ed9964227ac908c2897d935e9
I20260709 13:03:48.976889 30060 ts_tablet_manager.cc:933] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:03:48.976172 29082 tablet_copy_source_session.cc:215] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:48.981346 30056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b27f1b9ed9964227ac908c2897d935e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:48.992981 29083 tablet_copy_source_session.cc:215] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:49.007784 30056 tablet_copy_client.cc:806] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:49.008941 30060 tablet_copy_client.cc:323] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.129:41631
I20260709 13:03:49.017616 30054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c777774468e477ca1ab0352dff56c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:49.023370 29083 tablet_copy_service.cc:140] P a936878f59c840a6bd176a1d2d7347b1: Received BeginTabletCopySession request for tablet d2ee8b4c1bf649b9b89ad6243f3890a8 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:40862)
I20260709 13:03:49.023967 29083 tablet_copy_service.cc:161] P a936878f59c840a6bd176a1d2d7347b1: Beginning new tablet copy session on tablet d2ee8b4c1bf649b9b89ad6243f3890a8 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:40862: session id = 7e78ffbcc98242068a874a5d9e341477-d2ee8b4c1bf649b9b89ad6243f3890a8
I20260709 13:03:49.032045 29083 tablet_copy_source_session.cc:215] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:49.019766 30056 tablet_copy_client.cc:670] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:49.036866 30054 tablet_copy_client.cc:806] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:49.037472 30054 tablet_copy_client.cc:670] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:49.042620 30060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ee8b4c1bf649b9b89ad6243f3890a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:49.056278 30056 tablet_copy_client.cc:538] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:49.077720 30054 tablet_copy_client.cc:538] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:49.083586 30056 tablet_bootstrap.cc:492] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:49.089813 30062 ts_tablet_manager.cc:933] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:49.106120 30062 tablet_copy_client.cc:323] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.130:35337
I20260709 13:03:49.107837 29158 tablet_copy_service.cc:140] P 730186a8c70f43f2a57d80163bbc37d2: Received BeginTabletCopySession request for tablet e96bceabcbf747e0acec327bc1abc729 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:60830)
I20260709 13:03:49.108435 29158 tablet_copy_service.cc:161] P 730186a8c70f43f2a57d80163bbc37d2: Beginning new tablet copy session on tablet e96bceabcbf747e0acec327bc1abc729 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:60830: session id = 7e78ffbcc98242068a874a5d9e341477-e96bceabcbf747e0acec327bc1abc729
I20260709 13:03:49.116043 29158 tablet_copy_source_session.cc:215] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:49.119261 30062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e96bceabcbf747e0acec327bc1abc729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:49.122424 30054 tablet_bootstrap.cc:492] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:49.131873 30060 tablet_copy_client.cc:806] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:49.132555 30060 tablet_copy_client.cc:670] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:49.139112 30060 tablet_copy_client.cc:538] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:49.158077 30060 tablet_bootstrap.cc:492] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:49.161559 30062 tablet_copy_client.cc:806] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:49.162171 30062 tablet_copy_client.cc:670] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:49.169142 30062 tablet_copy_client.cc:538] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:49.183547 30062 tablet_bootstrap.cc:492] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:49.184161 30056 tablet_bootstrap.cc:492] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:49.185132 30056 tablet_bootstrap.cc:492] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:49.185891 30056 ts_tablet_manager.cc:1403] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.103s	user 0.024s	sys 0.000s
I20260709 13:03:49.189232 30056 raft_consensus.cc:359] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:49.190066 30056 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:49.190706 30056 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:49.212095 30054 tablet_bootstrap.cc:492] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:49.213027 30054 tablet_bootstrap.cc:492] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:49.213726 30054 ts_tablet_manager.cc:1403] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.092s	user 0.023s	sys 0.006s
I20260709 13:03:49.217057 30056 ts_tablet_manager.cc:1434] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.031s	user 0.011s	sys 0.000s
I20260709 13:03:49.216950 30054 raft_consensus.cc:359] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:49.217859 30054 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:49.218747 29083 tablet_copy_service.cc:342] P a936878f59c840a6bd176a1d2d7347b1: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-b27f1b9ed9964227ac908c2897d935e9 received from {username='slave'} at 127.0.0.1:40862
I20260709 13:03:49.219342 29083 tablet_copy_service.cc:434] P a936878f59c840a6bd176a1d2d7347b1: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-b27f1b9ed9964227ac908c2897d935e9 on tablet b27f1b9ed9964227ac908c2897d935e9 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:49.218757 30054 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:49.220237 30060 tablet_bootstrap.cc:492] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:49.221271 30060 tablet_bootstrap.cc:492] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:49.221563 30054 ts_tablet_manager.cc:1434] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:03:49.222090 30060 ts_tablet_manager.cc:1403] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.064s	user 0.027s	sys 0.000s
I20260709 13:03:49.223237 29083 tablet_copy_service.cc:342] P a936878f59c840a6bd176a1d2d7347b1: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-3c777774468e477ca1ab0352dff56c9c received from {username='slave'} at 127.0.0.1:40862
I20260709 13:03:49.226863 29083 tablet_copy_service.cc:434] P a936878f59c840a6bd176a1d2d7347b1: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-3c777774468e477ca1ab0352dff56c9c on tablet 3c777774468e477ca1ab0352dff56c9c with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:49.231639 30060 raft_consensus.cc:359] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:49.232544 30060 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:49.233213 30060 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:49.236366 30062 tablet_bootstrap.cc:492] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:49.237323 30062 tablet_bootstrap.cc:492] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:49.238003 30062 ts_tablet_manager.cc:1403] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.055s	user 0.021s	sys 0.010s
I20260709 13:03:49.240756 30060 ts_tablet_manager.cc:1434] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:03:49.242699 29083 tablet_copy_service.cc:342] P a936878f59c840a6bd176a1d2d7347b1: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-d2ee8b4c1bf649b9b89ad6243f3890a8 received from {username='slave'} at 127.0.0.1:40862
I20260709 13:03:49.243156 29083 tablet_copy_service.cc:434] P a936878f59c840a6bd176a1d2d7347b1: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-d2ee8b4c1bf649b9b89ad6243f3890a8 on tablet d2ee8b4c1bf649b9b89ad6243f3890a8 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:49.247961 30062 raft_consensus.cc:359] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:49.248870 30062 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:49.249507 30062 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:49.251760 30062 ts_tablet_manager.cc:1434] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.001s
I20260709 13:03:49.255035 29158 tablet_copy_service.cc:342] P 730186a8c70f43f2a57d80163bbc37d2: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-e96bceabcbf747e0acec327bc1abc729 received from {username='slave'} at 127.0.0.1:60830
I20260709 13:03:49.255443 29158 tablet_copy_service.cc:434] P 730186a8c70f43f2a57d80163bbc37d2: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-e96bceabcbf747e0acec327bc1abc729 on tablet e96bceabcbf747e0acec327bc1abc729 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:49.267213 29909 raft_consensus.cc:1217] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:49.450877 29909 raft_consensus.cc:1217] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:49.528764 29909 raft_consensus.cc:1217] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:49.686132 29909 raft_consensus.cc:1217] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:49.850988 29987 raft_consensus.cc:1064] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:49.859248 29987 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:49.864714 30019 raft_consensus.cc:1064] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:49.867264 30019 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:49.887192 29909 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.895274 30047 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:49.919565 29141 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.922595 29222 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:49.935788 30072 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:49.957715 30075 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:50.003809 30078 raft_consensus.cc:2955] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.006271 29909 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.017920 29987 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:50.023496 29214 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:03:50.026357 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 58282.6us
I20260709 13:03:50.032943 29141 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.050581 30047 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:50.061807 29909 raft_consensus.cc:2955] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.064677 29143 raft_consensus.cc:2955] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.073402 30071 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:50.081845 30031 raft_consensus.cc:1064] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:50.084445 30031 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:50.097116 28968 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.097841 29909 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.100095 30031 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:50.104858 29217 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.106657 30031 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:50.169309 29068 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:03:50.171357 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51072.7us
I20260709 13:03:50.189043 30086 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:50.199342 30030 raft_consensus.cc:2955] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.225780 29217 raft_consensus.cc:2955] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.259270 30047 raft_consensus.cc:1064] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:50.269161 29214 raft_consensus.cc:2955] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.261216 30082 raft_consensus.cc:2955] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.281651 29144 raft_consensus.cc:2955] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.263124 30047 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:03:50.334707 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 113903us
I20260709 13:03:50.358564 29216 raft_consensus.cc:2955] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.476718 29064 raft_consensus.cc:2955] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.497430 29909 raft_consensus.cc:2955] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.524770 29910 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.530642 29139 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.532984 29911 raft_consensus.cc:2955] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.536578 30072 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:03:50.559652 29216 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.602919 30081 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:50.621935 30047 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:03:50.644027 28971 catalog_manager.cc:5697] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:03:50.756502 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 71479.8us
I20260709 13:03:50.798723 28971 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.803009 30072 raft_consensus.cc:2955] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.849408 29216 raft_consensus.cc:2955] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:50.875109 29064 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:50.899693 29143 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:50.946600 30079 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:50.979251 29144 raft_consensus.cc:2955] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.013612 29909 raft_consensus.cc:2955] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.041029 29910 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:51.046172 28971 catalog_manager.cc:5697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.061232 30082 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:51.060719 30084 raft_consensus.cc:2955] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.067762 29910 raft_consensus.cc:2955] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.102584 29147 raft_consensus.cc:2955] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.113502 28968 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.120388 28955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3c777774468e477ca1ab0352dff56c9c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:51.128402 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet 3c777774468e477ca1ab0352dff56c9c with delete_type TABLET_DATA_TOMBSTONED (TS 6ad9faa1c14444b29c9db8e71f46e2d6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42954
I20260709 13:03:51.130543 29073 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:51.137471 29073 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:51.141878 30120 tablet_replica.cc:333] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:03:51.142472 29214 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:51.148813 30081 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:51.154235 29909 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:51.155733 30081 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:51.189263 29217 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:51.194571 30116 raft_consensus.cc:2955] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.196645 29910 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:51.211460 30072 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:51.215495 30120 raft_consensus.cc:2243] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:51.216249 30120 raft_consensus.cc:2272] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:51.220775 30120 ts_tablet_manager.cc:1916] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:51.221748 30116 raft_consensus.cc:2955] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.224278 29910 raft_consensus.cc:2955] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.224612 29911 raft_consensus.cc:2955] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.244397 30120 ts_tablet_manager.cc:1929] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:51.245097 30120 log.cc:1199] T 3c777774468e477ca1ab0352dff56c9c P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/wals/3c777774468e477ca1ab0352dff56c9c
I20260709 13:03:51.246690 30072 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:51.247903 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet 3c777774468e477ca1ab0352dff56c9c (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:51.250638 28971 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.252954 28955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d2ee8b4c1bf649b9b89ad6243f3890a8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:51.260601 28955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b27f1b9ed9964227ac908c2897d935e9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:51.268241 29216 raft_consensus.cc:2955] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.269204 29217 raft_consensus.cc:2955] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.263111 28968 catalog_manager.cc:5697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.301301 29144 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:51.306939 29134 tablet_service.cc:1558] Processing DeleteTablet for tablet b27f1b9ed9964227ac908c2897d935e9 with delete_type TABLET_DATA_TOMBSTONED (TS 730186a8c70f43f2a57d80163bbc37d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42040
I20260709 13:03:51.325918 29910 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:51.330878 30086 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:51.335121 30121 tablet_replica.cc:333] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:03:51.337059 29064 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:51.345391 30070 raft_consensus.cc:2955] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.349427 30086 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:51.352407 29910 raft_consensus.cc:2955] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.354101 29134 tablet_service.cc:1558] Processing DeleteTablet for tablet d2ee8b4c1bf649b9b89ad6243f3890a8 with delete_type TABLET_DATA_TOMBSTONED (TS 730186a8c70f43f2a57d80163bbc37d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42040
I20260709 13:03:51.360404 29064 raft_consensus.cc:2955] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:51.367251 28958 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e96bceabcbf747e0acec327bc1abc729 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:51.383700 30121 raft_consensus.cc:2243] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:51.384701 30121 raft_consensus.cc:2272] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:51.390479 30121 ts_tablet_manager.cc:1916] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:51.390793 28968 catalog_manager.cc:5697] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:51.420437 30121 ts_tablet_manager.cc:1929] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:51.420995 30121 log.cc:1199] T b27f1b9ed9964227ac908c2897d935e9 P 730186a8c70f43f2a57d80163bbc37d2: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/wals/b27f1b9ed9964227ac908c2897d935e9
I20260709 13:03:51.422995 30121 tablet_replica.cc:333] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:03:51.423110 28958 catalog_manager.cc:5028] TS 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): tablet b27f1b9ed9964227ac908c2897d935e9 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:51.436682 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet e96bceabcbf747e0acec327bc1abc729 with delete_type TABLET_DATA_TOMBSTONED (TS 6ad9faa1c14444b29c9db8e71f46e2d6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42954
I20260709 13:03:51.437853 30120 tablet_replica.cc:333] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:03:51.438901 30120 raft_consensus.cc:2243] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:51.439670 30120 raft_consensus.cc:2272] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:51.447566 30120 ts_tablet_manager.cc:1916] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:51.437860 30121 raft_consensus.cc:2243] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:51.450912 30121 raft_consensus.cc:2272] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:51.457365 30121 ts_tablet_manager.cc:1916] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:51.479641 30120 ts_tablet_manager.cc:1929] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:51.480154 30120 log.cc:1199] T e96bceabcbf747e0acec327bc1abc729 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/wals/e96bceabcbf747e0acec327bc1abc729
I20260709 13:03:51.482465 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet e96bceabcbf747e0acec327bc1abc729 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:51.484901 30121 ts_tablet_manager.cc:1929] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:51.485361 30121 log.cc:1199] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 730186a8c70f43f2a57d80163bbc37d2: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/wals/d2ee8b4c1bf649b9b89ad6243f3890a8
I20260709 13:03:51.486728 28958 catalog_manager.cc:5028] TS 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): tablet d2ee8b4c1bf649b9b89ad6243f3890a8 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
W20260709 13:03:51.846702 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08000ae2c0 after lost signal to thread 27937
W20260709 13:03:51.847783 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08000c20e0 after lost signal to thread 29005
W20260709 13:03:51.848951 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08001a77c0 after lost signal to thread 29097
W20260709 13:03:51.849774 29243 debug-util.cc:398] Leaking SignalData structure 0x7b080031b040 after lost signal to thread 29172
W20260709 13:03:51.850544 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08002cdf40 after lost signal to thread 29246
W20260709 13:03:51.851333 29243 debug-util.cc:398] Leaking SignalData structure 0x7b0800300140 after lost signal to thread 29935
I20260709 13:03:52.551397 29144 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:03:52.564917 29112 consensus_peers.cc:597] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9e4dbd865b648aaa1bce128c9de27af. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:52.566154 29213 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:52.567956 29068 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:52.569064 30070 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:52.571389 30122 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:03:52.602831 29064 raft_consensus.cc:2955] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:52.603050 30126 raft_consensus.cc:2955] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:52.607831 29213 raft_consensus.cc:2955] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:52.634564 29144 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:52.639576 28969 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:03:52.645951 29112 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a385068bbb144e6fa0e8e9974398b1b9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:52.648989 29213 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.648999 29064 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.651443 30125 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:52.653676 30125 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:52.718072 30070 raft_consensus.cc:2955] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:52.727111 29213 raft_consensus.cc:2955] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:52.743304 29064 raft_consensus.cc:2955] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:52.757946 29216 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:52.767596 28970 catalog_manager.cc:5697] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:03:52.787791 29186 consensus_peers.cc:597] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 128a7406f55d4400bfd5fadcbc14d065. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:52.788924 29144 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:52.792357 29064 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:52.793197 30123 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:52.794431 30106 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:52.802230 30106 raft_consensus.cc:2955] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:52.826068 29216 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:52.827334 28970 catalog_manager.cc:5697] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:52.829059 29064 raft_consensus.cc:2955] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:52.831677 29139 raft_consensus.cc:2955] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:03:52.867658 29186 consensus_peers.cc:597] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc31aa85851e41609957a765b2d40774. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:52.868459 29139 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:52.869882 30098 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:52.873345 29064 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:52.874863 30106 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:52.889959 30123 raft_consensus.cc:2955] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:52.909842 29073 raft_consensus.cc:2955] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:52.911332 29144 raft_consensus.cc:2955] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:52.941835 28970 catalog_manager.cc:5697] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:53.111167 30146 ts_tablet_manager.cc:933] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:53.122473 30147 ts_tablet_manager.cc:933] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:03:53.132715 30146 tablet_copy_client.cc:323] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.130:35337
I20260709 13:03:53.136022 29158 tablet_copy_service.cc:140] P 730186a8c70f43f2a57d80163bbc37d2: Received BeginTabletCopySession request for tablet a9e4dbd865b648aaa1bce128c9de27af from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:60830)
I20260709 13:03:53.136976 29158 tablet_copy_service.cc:161] P 730186a8c70f43f2a57d80163bbc37d2: Beginning new tablet copy session on tablet a9e4dbd865b648aaa1bce128c9de27af from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:60830: session id = 7e78ffbcc98242068a874a5d9e341477-a9e4dbd865b648aaa1bce128c9de27af
I20260709 13:03:53.152525 29158 tablet_copy_source_session.cc:215] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:53.163000 30147 tablet_copy_client.cc:323] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.130:35337
I20260709 13:03:53.167906 29157 tablet_copy_service.cc:140] P 730186a8c70f43f2a57d80163bbc37d2: Received BeginTabletCopySession request for tablet a385068bbb144e6fa0e8e9974398b1b9 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:60830)
I20260709 13:03:53.168535 29157 tablet_copy_service.cc:161] P 730186a8c70f43f2a57d80163bbc37d2: Beginning new tablet copy session on tablet a385068bbb144e6fa0e8e9974398b1b9 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:60830: session id = 7e78ffbcc98242068a874a5d9e341477-a385068bbb144e6fa0e8e9974398b1b9
I20260709 13:03:53.175771 30146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e4dbd865b648aaa1bce128c9de27af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.208081 29157 tablet_copy_source_session.cc:215] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:53.213706 30147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a385068bbb144e6fa0e8e9974398b1b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.224943 30150 ts_tablet_manager.cc:933] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:53.235316 30146 tablet_copy_client.cc:806] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:53.236172 30146 tablet_copy_client.cc:670] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:53.241201 30146 tablet_copy_client.cc:538] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:53.248405 30150 tablet_copy_client.cc:323] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.131:45851
I20260709 13:03:53.250686 29232 tablet_copy_service.cc:140] P 6ad9faa1c14444b29c9db8e71f46e2d6: Received BeginTabletCopySession request for tablet 128a7406f55d4400bfd5fadcbc14d065 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:51820)
I20260709 13:03:53.251368 29232 tablet_copy_service.cc:161] P 6ad9faa1c14444b29c9db8e71f46e2d6: Beginning new tablet copy session on tablet 128a7406f55d4400bfd5fadcbc14d065 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:51820: session id = 7e78ffbcc98242068a874a5d9e341477-128a7406f55d4400bfd5fadcbc14d065
I20260709 13:03:53.263957 30147 tablet_copy_client.cc:806] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:53.264746 30147 tablet_copy_client.cc:670] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:53.265425 29232 tablet_copy_source_session.cc:215] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:53.272404 30150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 128a7406f55d4400bfd5fadcbc14d065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.278415 30147 tablet_copy_client.cc:538] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:53.290879 30146 tablet_bootstrap.cc:492] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:53.302460 30147 tablet_bootstrap.cc:492] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:53.316028 30150 tablet_copy_client.cc:806] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:53.316741 30150 tablet_copy_client.cc:670] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:53.329404 30150 tablet_copy_client.cc:538] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:53.338991 30150 tablet_bootstrap.cc:492] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:53.362701 30147 tablet_bootstrap.cc:492] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.363823 30147 tablet_bootstrap.cc:492] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:53.364590 30147 ts_tablet_manager.cc:1403] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.062s	user 0.023s	sys 0.005s
I20260709 13:03:53.368160 30147 raft_consensus.cc:359] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:53.373522 30147 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:53.374372 30147 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:53.417310 30155 ts_tablet_manager.cc:933] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:53.440663 30147 ts_tablet_manager.cc:1434] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.076s	user 0.011s	sys 0.018s
I20260709 13:03:53.458617 30155 tablet_copy_client.cc:323] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.131:45851
I20260709 13:03:53.466414 29157 tablet_copy_service.cc:342] P 730186a8c70f43f2a57d80163bbc37d2: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-a385068bbb144e6fa0e8e9974398b1b9 received from {username='slave'} at 127.0.0.1:60830
I20260709 13:03:53.472211 29157 tablet_copy_service.cc:434] P 730186a8c70f43f2a57d80163bbc37d2: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-a385068bbb144e6fa0e8e9974398b1b9 on tablet a385068bbb144e6fa0e8e9974398b1b9 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:53.481417 29232 tablet_copy_service.cc:140] P 6ad9faa1c14444b29c9db8e71f46e2d6: Received BeginTabletCopySession request for tablet cc31aa85851e41609957a765b2d40774 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:51820)
I20260709 13:03:53.482012 29232 tablet_copy_service.cc:161] P 6ad9faa1c14444b29c9db8e71f46e2d6: Beginning new tablet copy session on tablet cc31aa85851e41609957a765b2d40774 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:51820: session id = 7e78ffbcc98242068a874a5d9e341477-cc31aa85851e41609957a765b2d40774
I20260709 13:03:53.485200 30146 tablet_bootstrap.cc:492] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.486272 30146 tablet_bootstrap.cc:492] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:53.493216 30150 tablet_bootstrap.cc:492] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.495352 30150 tablet_bootstrap.cc:492] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:53.500612 30150 ts_tablet_manager.cc:1403] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.162s	user 0.030s	sys 0.003s
I20260709 13:03:53.512778 30146 ts_tablet_manager.cc:1403] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.196s	user 0.033s	sys 0.053s
I20260709 13:03:53.524223 30150 raft_consensus.cc:359] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:53.525230 30150 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:53.525913 30150 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:53.540581 30150 ts_tablet_manager.cc:1434] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:03:53.545675 29232 tablet_copy_source_session.cc:215] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:53.552322 30155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc31aa85851e41609957a765b2d40774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.546561 30146 raft_consensus.cc:359] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:53.553869 30146 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:53.554582 30146 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:53.556901 29232 tablet_copy_service.cc:342] P 6ad9faa1c14444b29c9db8e71f46e2d6: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-128a7406f55d4400bfd5fadcbc14d065 received from {username='slave'} at 127.0.0.1:51820
I20260709 13:03:53.557284 30146 ts_tablet_manager.cc:1434] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:03:53.559307 29157 tablet_copy_service.cc:342] P 730186a8c70f43f2a57d80163bbc37d2: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-a9e4dbd865b648aaa1bce128c9de27af received from {username='slave'} at 127.0.0.1:60830
I20260709 13:03:53.559738 29157 tablet_copy_service.cc:434] P 730186a8c70f43f2a57d80163bbc37d2: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-a9e4dbd865b648aaa1bce128c9de27af on tablet a9e4dbd865b648aaa1bce128c9de27af with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:53.557401 29232 tablet_copy_service.cc:434] P 6ad9faa1c14444b29c9db8e71f46e2d6: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-128a7406f55d4400bfd5fadcbc14d065 on tablet 128a7406f55d4400bfd5fadcbc14d065 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:53.591670 29911 raft_consensus.cc:1217] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:53.601820 29911 raft_consensus.cc:1217] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:53.603819 29910 raft_consensus.cc:1217] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:03:53.652143 30155 tablet_copy_client.cc:806] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:53.652792 30155 tablet_copy_client.cc:670] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:53.658406 30155 tablet_copy_client.cc:538] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:53.668885 30155 tablet_bootstrap.cc:492] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:53.709532 30155 tablet_bootstrap.cc:492] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.710549 30155 tablet_bootstrap.cc:492] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:53.711364 30155 ts_tablet_manager.cc:1403] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.043s	user 0.028s	sys 0.000s
I20260709 13:03:53.714808 30155 raft_consensus.cc:359] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:53.715643 30155 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:53.716349 30155 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:53.726588 30155 ts_tablet_manager.cc:1434] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.003s
I20260709 13:03:53.728609 29232 tablet_copy_service.cc:342] P 6ad9faa1c14444b29c9db8e71f46e2d6: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-cc31aa85851e41609957a765b2d40774 received from {username='slave'} at 127.0.0.1:51820
I20260709 13:03:53.731424 29232 tablet_copy_service.cc:434] P 6ad9faa1c14444b29c9db8e71f46e2d6: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-cc31aa85851e41609957a765b2d40774 on tablet cc31aa85851e41609957a765b2d40774 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:53.797763 29910 raft_consensus.cc:1217] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:53.990428 30159 raft_consensus.cc:1064] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:53.994472 30159 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:54.014045 30162 raft_consensus.cc:1064] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:54.016950 30162 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:54.066702 29910 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:54.068933 30167 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:54.070999 29068 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:54.072832 30162 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:54.082796 29064 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.083256 29911 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.084538 30163 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:54.085609 30167 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:54.097270 29213 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:54.099215 29213 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.104925 30159 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:54.105763 30167 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:03:54.132135 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 67818.3us
I20260709 13:03:54.170496 30167 raft_consensus.cc:2955] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.179735 29909 raft_consensus.cc:2955] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.183490 29222 raft_consensus.cc:2955] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.188539 30142 raft_consensus.cc:1064] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:54.191224 30142 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:54.200632 28970 catalog_manager.cc:5697] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.223153 29068 raft_consensus.cc:2955] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.225765 29909 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.227834 29147 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.229987 30106 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:54.230315 29909 raft_consensus.cc:2955] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.223971 30159 raft_consensus.cc:2955] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.279876 29073 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:03:54.281805 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47066.5us
I20260709 13:03:54.281468 29213 raft_consensus.cc:2955] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.282852 30106 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:54.314366 30106 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:54.364880 29068 raft_consensus.cc:2955] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:03:54.381024 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 66136.2us
W20260709 13:03:54.386260 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 73198.8us
I20260709 13:03:54.389326 30145 raft_consensus.cc:2955] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.393451 30142 raft_consensus.cc:1064] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:54.396173 30142 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:54.433923 29144 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:54.436421 29147 raft_consensus.cc:2955] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.446763 29910 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:54.448493 30166 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:54.449846 28970 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.457228 30159 raft_consensus.cc:2955] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.459625 29909 raft_consensus.cc:2955] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.465095 29064 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:54.477849 29911 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.481588 29910 raft_consensus.cc:2955] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.481666 30142 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:03:54.496920 30166 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:54.503983 29140 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.510268 29072 raft_consensus.cc:2955] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.512575 29068 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:54.515233 30145 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:54.520090 29064 raft_consensus.cc:2955] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.535193 28967 catalog_manager.cc:5697] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.535759 28958 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a385068bbb144e6fa0e8e9974398b1b9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:54.536724 30184 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:03:54.571803 28966 catalog_manager.cc:5697] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:54.576699 29144 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:54.594175 29222 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.596741 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet a385068bbb144e6fa0e8e9974398b1b9 with delete_type TABLET_DATA_TOMBSTONED (TS 6ad9faa1c14444b29c9db8e71f46e2d6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42954
I20260709 13:03:54.600414 30159 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:54.615181 30159 raft_consensus.cc:2955] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.608507 29909 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.624559 30183 raft_consensus.cc:2955] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.628854 30070 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:54.620328 29213 raft_consensus.cc:2955] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.637516 29909 raft_consensus.cc:2955] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.647605 29214 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:54.666124 29910 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.669879 30177 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:54.679033 28966 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:54.681856 30106 raft_consensus.cc:2955] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.683328 29910 raft_consensus.cc:2955] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.704784 28968 catalog_manager.cc:5697] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.716162 28958 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a9e4dbd865b648aaa1bce128c9de27af with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:54.719283 29147 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.720964 29139 raft_consensus.cc:2955] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.621384 30192 tablet_replica.cc:333] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:03:54.729710 30192 raft_consensus.cc:2243] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:54.733891 28957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 128a7406f55d4400bfd5fadcbc14d065 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:54.740540 30192 raft_consensus.cc:2272] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:54.740772 30182 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:54.751142 30192 ts_tablet_manager.cc:1916] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:54.757157 29909 raft_consensus.cc:2955] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.764067 29059 tablet_service.cc:1558] Processing DeleteTablet for tablet a9e4dbd865b648aaa1bce128c9de27af with delete_type TABLET_DATA_TOMBSTONED (TS a936878f59c840a6bd176a1d2d7347b1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47946
I20260709 13:03:54.764315 29056 tablet_service.cc:1558] Processing DeleteTablet for tablet 128a7406f55d4400bfd5fadcbc14d065 with delete_type TABLET_DATA_TOMBSTONED (TS a936878f59c840a6bd176a1d2d7347b1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47946
I20260709 13:03:54.774587 29146 raft_consensus.cc:2955] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.775724 30194 tablet_replica.cc:333] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:03:54.777222 30194 raft_consensus.cc:2243] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:54.777954 30194 raft_consensus.cc:2272] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:54.781415 30194 ts_tablet_manager.cc:1916] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:54.801509 29064 raft_consensus.cc:2955] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.811796 30194 ts_tablet_manager.cc:1929] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:54.813609 30194 log.cc:1199] T a9e4dbd865b648aaa1bce128c9de27af P a936878f59c840a6bd176a1d2d7347b1: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/wals/a9e4dbd865b648aaa1bce128c9de27af
I20260709 13:03:54.815446 30194 tablet_replica.cc:333] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:03:54.815955 28955 catalog_manager.cc:5028] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): tablet a9e4dbd865b648aaa1bce128c9de27af (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:54.822914 28969 catalog_manager.cc:5697] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.850822 30194 raft_consensus.cc:2243] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:54.851723 30194 raft_consensus.cc:2272] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:54.857208 30194 ts_tablet_manager.cc:1916] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:54.872049 30192 ts_tablet_manager.cc:1929] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:03:54.872623 30192 log.cc:1199] T a385068bbb144e6fa0e8e9974398b1b9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/wals/a385068bbb144e6fa0e8e9974398b1b9
I20260709 13:03:54.874637 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet a385068bbb144e6fa0e8e9974398b1b9 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:54.917495 29214 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:54.959786 29909 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.963227 29064 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:54.964442 30182 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:54.967212 30194 ts_tablet_manager.cc:1929] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:54.969362 30194 log.cc:1199] T 128a7406f55d4400bfd5fadcbc14d065 P a936878f59c840a6bd176a1d2d7347b1: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/wals/128a7406f55d4400bfd5fadcbc14d065
I20260709 13:03:54.971343 28955 catalog_manager.cc:5028] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): tablet 128a7406f55d4400bfd5fadcbc14d065 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:54.971372 30183 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:54.977231 30106 raft_consensus.cc:2955] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.986073 29909 raft_consensus.cc:2955] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:54.988386 29068 raft_consensus.cc:2955] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:55.016716 28957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cc31aa85851e41609957a765b2d40774 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:55.021083 28969 catalog_manager.cc:5697] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:55.039949 29134 tablet_service.cc:1558] Processing DeleteTablet for tablet cc31aa85851e41609957a765b2d40774 with delete_type TABLET_DATA_TOMBSTONED (TS 730186a8c70f43f2a57d80163bbc37d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42040
I20260709 13:03:55.041682 30195 tablet_replica.cc:333] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:03:55.043844 30195 raft_consensus.cc:2243] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:55.046558 30195 raft_consensus.cc:2272] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:55.055536 30195 ts_tablet_manager.cc:1916] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:55.071573 30195 ts_tablet_manager.cc:1929] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:55.071990 30195 log.cc:1199] T cc31aa85851e41609957a765b2d40774 P 730186a8c70f43f2a57d80163bbc37d2: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/wals/cc31aa85851e41609957a765b2d40774
I20260709 13:03:55.073333 28958 catalog_manager.cc:5028] TS 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): tablet cc31aa85851e41609957a765b2d40774 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:56.126093 29068 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:03:56.135172 29037 consensus_peers.cc:597] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2fae9bf99a434b83a73451cee1c56e6a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:56.137878 29222 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:56.139531 30185 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:56.140113 29146 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:56.141851 30185 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:56.152371 30158 raft_consensus.cc:2955] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:56.155815 29222 raft_consensus.cc:2955] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:56.173709 29064 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:03:56.188545 29037 consensus_peers.cc:597] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 88cabbc6d2424a53a76857f415831c36. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:56.211578 29222 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:56.220228 29140 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:56.219386 28966 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:56.222716 30185 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:56.227028 30202 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:56.245987 30158 raft_consensus.cc:2955] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:56.249164 29222 raft_consensus.cc:2955] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:56.263280 29213 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:56.265331 29146 raft_consensus.cc:2955] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:56.268164 29140 raft_consensus.cc:2955] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:03:56.288887 29186 consensus_peers.cc:597] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 01d4b885cd86484fbaa6c528fc2ed1bc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:56.293953 29146 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:56.295387 29064 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:56.315546 28971 catalog_manager.cc:5697] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:56.295459 30182 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:03:56.327086 30183 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:56.401883 30214 raft_consensus.cc:2955] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:56.410704 29146 raft_consensus.cc:2955] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:56.411816 29064 raft_consensus.cc:2955] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:56.421475 29068 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:03:56.434872 29037 consensus_peers.cc:597] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3842b36ffd134cd28c657475df48e741. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:56.438062 28971 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:56.440627 29213 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:56.445003 29146 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:56.446504 30203 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:56.448568 30203 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:56.482004 30185 raft_consensus.cc:2955] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:56.483803 29213 raft_consensus.cc:2955] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:56.484654 29144 raft_consensus.cc:2955] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:56.501753 28971 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:56.576766 30221 ts_tablet_manager.cc:933] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:03:56.578941 30221 tablet_copy_client.cc:323] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.129:41631
I20260709 13:03:56.581041 29083 tablet_copy_service.cc:140] P a936878f59c840a6bd176a1d2d7347b1: Received BeginTabletCopySession request for tablet 2fae9bf99a434b83a73451cee1c56e6a from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:40862)
I20260709 13:03:56.581655 29083 tablet_copy_service.cc:161] P a936878f59c840a6bd176a1d2d7347b1: Beginning new tablet copy session on tablet 2fae9bf99a434b83a73451cee1c56e6a from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:40862: session id = 7e78ffbcc98242068a874a5d9e341477-2fae9bf99a434b83a73451cee1c56e6a
I20260709 13:03:56.616200 29083 tablet_copy_source_session.cc:215] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:56.620381 30221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fae9bf99a434b83a73451cee1c56e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:56.659850 30221 tablet_copy_client.cc:806] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:56.660738 30221 tablet_copy_client.cc:670] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:56.665060 30221 tablet_copy_client.cc:538] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:56.672461 30221 tablet_bootstrap.cc:492] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:56.696768 30221 tablet_bootstrap.cc:492] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:56.697867 30221 tablet_bootstrap.cc:492] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:56.698689 30221 ts_tablet_manager.cc:1403] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.009s
I20260709 13:03:56.702207 30221 raft_consensus.cc:359] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:56.703030 30221 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:56.703791 30221 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:56.735082 30225 ts_tablet_manager.cc:933] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:03:56.741214 30221 ts_tablet_manager.cc:1434] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.042s	user 0.021s	sys 0.008s
I20260709 13:03:56.756623 29083 tablet_copy_service.cc:342] P a936878f59c840a6bd176a1d2d7347b1: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-2fae9bf99a434b83a73451cee1c56e6a received from {username='slave'} at 127.0.0.1:40862
I20260709 13:03:56.757337 29083 tablet_copy_service.cc:434] P a936878f59c840a6bd176a1d2d7347b1: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-2fae9bf99a434b83a73451cee1c56e6a on tablet 2fae9bf99a434b83a73451cee1c56e6a with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:56.763986 30221 ts_tablet_manager.cc:933] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:03:56.844714 30225 tablet_copy_client.cc:323] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.131:45851
I20260709 13:03:56.847363 29232 tablet_copy_service.cc:140] P 6ad9faa1c14444b29c9db8e71f46e2d6: Received BeginTabletCopySession request for tablet 01d4b885cd86484fbaa6c528fc2ed1bc from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:51820)
I20260709 13:03:56.847950 29232 tablet_copy_service.cc:161] P 6ad9faa1c14444b29c9db8e71f46e2d6: Beginning new tablet copy session on tablet 01d4b885cd86484fbaa6c528fc2ed1bc from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:51820: session id = 7e78ffbcc98242068a874a5d9e341477-01d4b885cd86484fbaa6c528fc2ed1bc
I20260709 13:03:56.851660 30221 tablet_copy_client.cc:323] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.129:41631
I20260709 13:03:56.856359 29083 tablet_copy_service.cc:140] P a936878f59c840a6bd176a1d2d7347b1: Received BeginTabletCopySession request for tablet 88cabbc6d2424a53a76857f415831c36 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:40862)
I20260709 13:03:56.857021 29083 tablet_copy_service.cc:161] P a936878f59c840a6bd176a1d2d7347b1: Beginning new tablet copy session on tablet 88cabbc6d2424a53a76857f415831c36 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:40862: session id = 7e78ffbcc98242068a874a5d9e341477-88cabbc6d2424a53a76857f415831c36
I20260709 13:03:56.861384 29232 tablet_copy_source_session.cc:215] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:56.864538 29083 tablet_copy_source_session.cc:215] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:56.869407 30225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d4b885cd86484fbaa6c528fc2ed1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:56.875192 30221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88cabbc6d2424a53a76857f415831c36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:56.892781 30227 ts_tablet_manager.cc:933] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:03:56.919266 30225 tablet_copy_client.cc:806] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:56.920893 30225 tablet_copy_client.cc:670] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:56.929320 30225 tablet_copy_client.cc:538] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:56.939818 30225 tablet_bootstrap.cc:492] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:56.954648 30221 tablet_copy_client.cc:806] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:56.955294 30221 tablet_copy_client.cc:670] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:56.959386 30221 tablet_copy_client.cc:538] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:56.972978 30221 tablet_bootstrap.cc:492] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:56.971969 30227 tablet_copy_client.cc:323] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.129:41631
I20260709 13:03:56.976989 29083 tablet_copy_service.cc:140] P a936878f59c840a6bd176a1d2d7347b1: Received BeginTabletCopySession request for tablet 3842b36ffd134cd28c657475df48e741 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:40862)
I20260709 13:03:56.977560 29083 tablet_copy_service.cc:161] P a936878f59c840a6bd176a1d2d7347b1: Beginning new tablet copy session on tablet 3842b36ffd134cd28c657475df48e741 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:40862: session id = 7e78ffbcc98242068a874a5d9e341477-3842b36ffd134cd28c657475df48e741
I20260709 13:03:56.996634 30225 tablet_bootstrap.cc:492] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:56.997752 30225 tablet_bootstrap.cc:492] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:56.998610 30225 ts_tablet_manager.cc:1403] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.059s	user 0.026s	sys 0.001s
I20260709 13:03:57.010951 30225 raft_consensus.cc:359] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:57.011821 30225 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:57.015741 30225 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:57.019622 30225 ts_tablet_manager.cc:1434] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 13:03:57.021711 29232 tablet_copy_service.cc:342] P 6ad9faa1c14444b29c9db8e71f46e2d6: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-01d4b885cd86484fbaa6c528fc2ed1bc received from {username='slave'} at 127.0.0.1:51820
I20260709 13:03:57.022153 29232 tablet_copy_service.cc:434] P 6ad9faa1c14444b29c9db8e71f46e2d6: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-01d4b885cd86484fbaa6c528fc2ed1bc on tablet 01d4b885cd86484fbaa6c528fc2ed1bc with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:57.031352 29083 tablet_copy_source_session.cc:215] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:57.041179 30227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3842b36ffd134cd28c657475df48e741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:57.044679 30221 tablet_bootstrap.cc:492] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:57.045740 30221 tablet_bootstrap.cc:492] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:57.046510 30221 ts_tablet_manager.cc:1403] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.074s	user 0.026s	sys 0.014s
I20260709 13:03:57.050843 30221 raft_consensus.cc:359] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:57.051860 30221 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:57.052646 30221 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:57.056968 30221 ts_tablet_manager.cc:1434] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:03:57.059244 29083 tablet_copy_service.cc:342] P a936878f59c840a6bd176a1d2d7347b1: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-88cabbc6d2424a53a76857f415831c36 received from {username='slave'} at 127.0.0.1:40862
I20260709 13:03:57.059825 29083 tablet_copy_service.cc:434] P a936878f59c840a6bd176a1d2d7347b1: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-88cabbc6d2424a53a76857f415831c36 on tablet 88cabbc6d2424a53a76857f415831c36 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:57.096626 30227 tablet_copy_client.cc:806] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:57.098877 30227 tablet_copy_client.cc:670] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:57.110037 29910 raft_consensus.cc:1217] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:57.111538 30227 tablet_copy_client.cc:538] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:57.115100 29909 raft_consensus.cc:1217] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:57.148223 30227 tablet_bootstrap.cc:492] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:03:57.200500 30227 tablet_bootstrap.cc:492] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:57.202032 30227 tablet_bootstrap.cc:492] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:03:57.211143 30227 ts_tablet_manager.cc:1403] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.063s	user 0.018s	sys 0.008s
I20260709 13:03:57.214818 30227 raft_consensus.cc:359] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:57.215778 30227 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:03:57.216534 30227 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:57.230063 30227 ts_tablet_manager.cc:1434] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.004s
I20260709 13:03:57.231429 29909 raft_consensus.cc:1217] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:57.237073 29083 tablet_copy_service.cc:342] P a936878f59c840a6bd176a1d2d7347b1: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-3842b36ffd134cd28c657475df48e741 received from {username='slave'} at 127.0.0.1:40862
I20260709 13:03:57.237526 29083 tablet_copy_service.cc:434] P a936878f59c840a6bd176a1d2d7347b1: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-3842b36ffd134cd28c657475df48e741 on tablet 3842b36ffd134cd28c657475df48e741 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:03:57.412114 29909 raft_consensus.cc:1217] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:57.763339 30235 raft_consensus.cc:1064] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:57.766021 30235 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:57.800421 29909 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:57.802074 30213 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:57.810725 30203 raft_consensus.cc:1064] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:57.820401 30203 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:57.834638 30239 raft_consensus.cc:1064] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:57.837270 30239 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:57.856177 29909 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:57.858261 30185 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:57.871861 29213 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:57.873956 29068 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:57.880548 29910 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:57.883931 30240 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:57.884847 30185 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:57.887677 30235 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:57.894004 29213 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:57.903730 29140 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:57.905822 29144 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:57.906503 30185 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:57.935521 30203 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:57.938827 30182 raft_consensus.cc:2955] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:57.941349 29071 raft_consensus.cc:2955] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:57.970000 29140 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:57.975441 28971 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:57.982945 30185 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:57.999327 30185 raft_consensus.cc:2955] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:57.989682 30214 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:03:58.016299 30240 raft_consensus.cc:2955] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.017947 29910 raft_consensus.cc:2955] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.021591 30237 raft_consensus.cc:1064] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:03:58.022022 29217 raft_consensus.cc:2955] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.031884 30237 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:03:58.056124 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 66165.3us
I20260709 13:03:58.060459 28969 catalog_manager.cc:5697] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.069541 29213 raft_consensus.cc:2955] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.086370 29910 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:58.088243 30239 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:58.094393 29140 raft_consensus.cc:2955] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.097074 29217 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:58.098315 30239 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:58.098441 29144 raft_consensus.cc:2955] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.101687 29909 raft_consensus.cc:2955] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.138567 28966 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.149858 29146 raft_consensus.cc:2955] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.151531 29139 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:58.156086 30240 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:58.158613 29909 raft_consensus.cc:2955] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.196486 30238 raft_consensus.cc:2955] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.200891 29139 raft_consensus.cc:2955] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.216749 29910 raft_consensus.cc:2955] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.229130 29217 raft_consensus.cc:2955] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.255263 28971 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.265753 29072 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:58.271140 29910 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:58.271908 29217 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:58.276377 30237 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:58.278644 30239 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:58.267568 29068 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:58.325171 29910 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:58.325088 30240 raft_consensus.cc:2955] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.328598 29139 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:58.332111 29214 raft_consensus.cc:2955] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.337808 29072 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:58.341415 29910 raft_consensus.cc:2955] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.347338 30204 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:58.353470 28955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2fae9bf99a434b83a73451cee1c56e6a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:58.357236 28966 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.361423 29908 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:58.364298 29214 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:58.366344 30204 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:58.377159 29134 tablet_service.cc:1558] Processing DeleteTablet for tablet 2fae9bf99a434b83a73451cee1c56e6a with delete_type TABLET_DATA_TOMBSTONED (TS 730186a8c70f43f2a57d80163bbc37d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42040
I20260709 13:03:58.376410 29139 raft_consensus.cc:2955] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.384263 30266 tablet_replica.cc:333] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:03:58.384878 30250 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:58.394923 30266 raft_consensus.cc:2243] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:58.395630 30266 raft_consensus.cc:2272] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:58.396664 30239 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:58.398969 30266 ts_tablet_manager.cc:1916] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:58.374881 30204 raft_consensus.cc:2955] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.408193 28966 catalog_manager.cc:5697] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.419885 30185 raft_consensus.cc:2955] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.422801 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet 88cabbc6d2424a53a76857f415831c36 with delete_type TABLET_DATA_TOMBSTONED (TS 6ad9faa1c14444b29c9db8e71f46e2d6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42954
I20260709 13:03:58.419970 29910 raft_consensus.cc:2955] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.431074 30267 tablet_replica.cc:333] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:03:58.432246 30267 raft_consensus.cc:2243] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:58.433068 30267 raft_consensus.cc:2272] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:58.434648 30266 ts_tablet_manager.cc:1929] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:58.435859 29910 raft_consensus.cc:2955] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.440227 28955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3842b36ffd134cd28c657475df48e741 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:58.440148 30266 log.cc:1199] T 2fae9bf99a434b83a73451cee1c56e6a P 730186a8c70f43f2a57d80163bbc37d2: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/wals/2fae9bf99a434b83a73451cee1c56e6a
I20260709 13:03:58.438357 30267 ts_tablet_manager.cc:1916] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:58.440112 29214 raft_consensus.cc:2955] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.452523 28958 catalog_manager.cc:5028] TS 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): tablet 2fae9bf99a434b83a73451cee1c56e6a (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:58.456543 28971 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.466873 28955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 88cabbc6d2424a53a76857f415831c36 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:58.471637 29134 tablet_service.cc:1558] Processing DeleteTablet for tablet 3842b36ffd134cd28c657475df48e741 with delete_type TABLET_DATA_TOMBSTONED (TS 730186a8c70f43f2a57d80163bbc37d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42040
I20260709 13:03:58.472607 30266 tablet_replica.cc:333] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:03:58.473642 30266 raft_consensus.cc:2243] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:58.474282 30266 raft_consensus.cc:2272] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:58.477653 30267 ts_tablet_manager.cc:1929] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:58.478219 30267 log.cc:1199] T 88cabbc6d2424a53a76857f415831c36 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/wals/88cabbc6d2424a53a76857f415831c36
I20260709 13:03:58.479502 30266 ts_tablet_manager.cc:1916] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:58.480111 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet 88cabbc6d2424a53a76857f415831c36 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:58.505558 30266 ts_tablet_manager.cc:1929] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:58.506587 30266 log.cc:1199] T 3842b36ffd134cd28c657475df48e741 P 730186a8c70f43f2a57d80163bbc37d2: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/wals/3842b36ffd134cd28c657475df48e741
I20260709 13:03:58.508879 28958 catalog_manager.cc:5028] TS 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): tablet 3842b36ffd134cd28c657475df48e741 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:58.528617 29213 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:03:58.534160 29910 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:58.536140 30251 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:58.540710 29139 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:58.542181 30234 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:58.543552 30251 raft_consensus.cc:2955] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.547079 29910 raft_consensus.cc:2955] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.566396 29139 raft_consensus.cc:2955] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.574334 28957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 01d4b885cd86484fbaa6c528fc2ed1bc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:58.566377 28966 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:03:58.609634 29059 tablet_service.cc:1558] Processing DeleteTablet for tablet 01d4b885cd86484fbaa6c528fc2ed1bc with delete_type TABLET_DATA_TOMBSTONED (TS a936878f59c840a6bd176a1d2d7347b1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47946
I20260709 13:03:58.619818 30268 tablet_replica.cc:333] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:03:58.621034 30268 raft_consensus.cc:2243] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:58.621796 30268 raft_consensus.cc:2272] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:58.625746 30268 ts_tablet_manager.cc:1916] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:58.639050 30268 ts_tablet_manager.cc:1929] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:58.640200 30268 log.cc:1199] T 01d4b885cd86484fbaa6c528fc2ed1bc P a936878f59c840a6bd176a1d2d7347b1: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/wals/01d4b885cd86484fbaa6c528fc2ed1bc
I20260709 13:03:58.642441 28955 catalog_manager.cc:5028] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): tablet 01d4b885cd86484fbaa6c528fc2ed1bc (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:03:59.680691 29072 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:03:59.692451 29037 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e250871b4ad74e079a538cdb04065b00. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:59.696547 29139 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:59.695637 29213 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:59.699146 30271 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:59.700249 30272 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:59.709607 30204 raft_consensus.cc:2955] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.711145 29213 raft_consensus.cc:2955] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.726930 29140 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:59.730232 29146 raft_consensus.cc:2955] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.729743 28966 catalog_manager.cc:5697] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 2 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:03:59.747532 29112 consensus_peers.cc:597] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5acfd220cbb54cd181c08050de276f65. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:59.748906 29071 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:59.750754 29213 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:59.751511 30233 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:59.753001 30163 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:59.776191 30233 raft_consensus.cc:2955] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.777940 29071 raft_consensus.cc:2955] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.784221 29213 raft_consensus.cc:2955] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.799890 29071 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:59.800318 28967 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:03:59.811344 29037 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 984bea8952114e3f9c0a045e7f557e37. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:59.811383 29146 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:59.814324 30272 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:59.814667 29213 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:59.816119 30272 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:59.844399 30185 raft_consensus.cc:2955] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.846050 29146 raft_consensus.cc:2955] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.853721 29213 raft_consensus.cc:2955] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.866426 28966 catalog_manager.cc:5697] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.869812 29071 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:03:59.887210 29213 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:59.889621 29037 consensus_peers.cc:597] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 023c90d48e094012bc60b1e600fffa14. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:59.890164 29146 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:59.890440 30185 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:59.892292 30204 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:59.904089 30271 raft_consensus.cc:2955] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.905833 29213 raft_consensus.cc:2955] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.906138 29146 raft_consensus.cc:2955] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:03:59.922986 28971 catalog_manager.cc:5697] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:00.168212 30289 ts_tablet_manager.cc:933] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:00.171051 30289 tablet_copy_client.cc:323] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.129:41631
I20260709 13:04:00.172827 29083 tablet_copy_service.cc:140] P a936878f59c840a6bd176a1d2d7347b1: Received BeginTabletCopySession request for tablet e250871b4ad74e079a538cdb04065b00 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:40862)
I20260709 13:04:00.173324 29083 tablet_copy_service.cc:161] P a936878f59c840a6bd176a1d2d7347b1: Beginning new tablet copy session on tablet e250871b4ad74e079a538cdb04065b00 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:40862: session id = 7e78ffbcc98242068a874a5d9e341477-e250871b4ad74e079a538cdb04065b00
I20260709 13:04:00.188603 29083 tablet_copy_source_session.cc:215] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:00.192480 30289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e250871b4ad74e079a538cdb04065b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:00.214733 30289 tablet_copy_client.cc:806] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:00.215396 30289 tablet_copy_client.cc:670] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:00.219681 30289 tablet_copy_client.cc:538] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:00.230784 30289 tablet_bootstrap.cc:492] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:00.258755 30289 tablet_bootstrap.cc:492] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:00.259963 30289 tablet_bootstrap.cc:492] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:00.260694 30289 ts_tablet_manager.cc:1403] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.012s
I20260709 13:04:00.263676 30289 raft_consensus.cc:359] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:00.264523 30289 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:00.265132 30289 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:00.268340 30289 ts_tablet_manager.cc:1434] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:00.321380 29083 tablet_copy_service.cc:342] P a936878f59c840a6bd176a1d2d7347b1: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-e250871b4ad74e079a538cdb04065b00 received from {username='slave'} at 127.0.0.1:40862
I20260709 13:04:00.322880 29083 tablet_copy_service.cc:434] P a936878f59c840a6bd176a1d2d7347b1: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-e250871b4ad74e079a538cdb04065b00 on tablet e250871b4ad74e079a538cdb04065b00 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:00.372346 30300 ts_tablet_manager.cc:933] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:00.388675 30289 ts_tablet_manager.cc:933] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:00.396078 30300 tablet_copy_client.cc:323] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.129:41631
I20260709 13:04:00.447031 29083 tablet_copy_service.cc:140] P a936878f59c840a6bd176a1d2d7347b1: Received BeginTabletCopySession request for tablet 984bea8952114e3f9c0a045e7f557e37 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:40862)
I20260709 13:04:00.446597 30289 tablet_copy_client.cc:323] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.130:35337
I20260709 13:04:00.449007 29083 tablet_copy_service.cc:161] P a936878f59c840a6bd176a1d2d7347b1: Beginning new tablet copy session on tablet 984bea8952114e3f9c0a045e7f557e37 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:40862: session id = 7e78ffbcc98242068a874a5d9e341477-984bea8952114e3f9c0a045e7f557e37
I20260709 13:04:00.457176 30305 ts_tablet_manager.cc:933] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:00.457428 29157 tablet_copy_service.cc:140] P 730186a8c70f43f2a57d80163bbc37d2: Received BeginTabletCopySession request for tablet 5acfd220cbb54cd181c08050de276f65 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:60830)
I20260709 13:04:00.458122 29157 tablet_copy_service.cc:161] P 730186a8c70f43f2a57d80163bbc37d2: Beginning new tablet copy session on tablet 5acfd220cbb54cd181c08050de276f65 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:60830: session id = 7e78ffbcc98242068a874a5d9e341477-5acfd220cbb54cd181c08050de276f65
I20260709 13:04:00.563992 30305 tablet_copy_client.cc:323] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.129:41631
I20260709 13:04:00.573619 29083 tablet_copy_source_session.cc:215] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:00.584167 29157 tablet_copy_source_session.cc:215] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:00.595075 30300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 984bea8952114e3f9c0a045e7f557e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:00.605881 30289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5acfd220cbb54cd181c08050de276f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:00.625869 29083 tablet_copy_service.cc:140] P a936878f59c840a6bd176a1d2d7347b1: Received BeginTabletCopySession request for tablet 023c90d48e094012bc60b1e600fffa14 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:40862)
I20260709 13:04:00.626533 29083 tablet_copy_service.cc:161] P a936878f59c840a6bd176a1d2d7347b1: Beginning new tablet copy session on tablet 023c90d48e094012bc60b1e600fffa14 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:40862: session id = 7e78ffbcc98242068a874a5d9e341477-023c90d48e094012bc60b1e600fffa14
I20260709 13:04:00.637574 29083 tablet_copy_source_session.cc:215] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:00.654887 30305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 023c90d48e094012bc60b1e600fffa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:00.660637 29911 raft_consensus.cc:1217] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:04:00.681090 30289 tablet_copy_client.cc:806] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:00.682509 30289 tablet_copy_client.cc:670] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:00.684664 30300 tablet_copy_client.cc:806] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:00.689700 30289 tablet_copy_client.cc:538] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:00.700553 30300 tablet_copy_client.cc:670] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:00.702668 30305 tablet_copy_client.cc:806] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:00.703343 30305 tablet_copy_client.cc:670] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:00.709399 30300 tablet_copy_client.cc:538] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:00.716054 30305 tablet_copy_client.cc:538] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:00.719408 30300 tablet_bootstrap.cc:492] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:00.722416 30289 tablet_bootstrap.cc:492] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:00.747174 30305 tablet_bootstrap.cc:492] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:00.756659 30300 tablet_bootstrap.cc:492] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:00.757774 30300 tablet_bootstrap.cc:492] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:00.758574 30300 ts_tablet_manager.cc:1403] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.015s
I20260709 13:04:00.762477 30300 raft_consensus.cc:359] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:00.763733 30300 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:00.764506 30300 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:00.767719 30300 ts_tablet_manager.cc:1434] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:04:00.784864 30289 tablet_bootstrap.cc:492] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:00.786069 30289 tablet_bootstrap.cc:492] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:00.786942 30289 ts_tablet_manager.cc:1403] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.065s	user 0.021s	sys 0.007s
I20260709 13:04:00.790246 30289 raft_consensus.cc:359] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:00.791332 30289 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:00.783849 30305 tablet_bootstrap.cc:492] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:00.799532 30305 tablet_bootstrap.cc:492] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:00.800450 30305 ts_tablet_manager.cc:1403] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.054s	user 0.022s	sys 0.006s
I20260709 13:04:00.792133 30289 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:00.802039 29082 tablet_copy_service.cc:342] P a936878f59c840a6bd176a1d2d7347b1: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-984bea8952114e3f9c0a045e7f557e37 received from {username='slave'} at 127.0.0.1:40862
I20260709 13:04:00.812309 29082 tablet_copy_service.cc:434] P a936878f59c840a6bd176a1d2d7347b1: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-984bea8952114e3f9c0a045e7f557e37 on tablet 984bea8952114e3f9c0a045e7f557e37 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:00.810585 30305 raft_consensus.cc:359] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:00.812844 30305 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:00.813583 30305 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:00.814431 30289 ts_tablet_manager.cc:1434] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:04:00.816112 30305 ts_tablet_manager.cc:1434] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:04:00.818039 29157 tablet_copy_service.cc:342] P 730186a8c70f43f2a57d80163bbc37d2: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-5acfd220cbb54cd181c08050de276f65 received from {username='slave'} at 127.0.0.1:60830
I20260709 13:04:00.818459 29157 tablet_copy_service.cc:434] P 730186a8c70f43f2a57d80163bbc37d2: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-5acfd220cbb54cd181c08050de276f65 on tablet 5acfd220cbb54cd181c08050de276f65 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:00.826607 29082 tablet_copy_service.cc:342] P a936878f59c840a6bd176a1d2d7347b1: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-023c90d48e094012bc60b1e600fffa14 received from {username='slave'} at 127.0.0.1:40862
I20260709 13:04:00.827059 29082 tablet_copy_service.cc:434] P a936878f59c840a6bd176a1d2d7347b1: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-023c90d48e094012bc60b1e600fffa14 on tablet 023c90d48e094012bc60b1e600fffa14 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:00.893229 29911 raft_consensus.cc:1217] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:00.997030 29911 raft_consensus.cc:1217] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:01.219074 30185 raft_consensus.cc:1064] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:01.220969 30185 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:01.231557 29911 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:01.240497 30270 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:01.247076 29215 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:01.249833 30272 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:01.250617 29140 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:01.253643 30271 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:01.266331 30271 raft_consensus.cc:2955] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.268574 29911 raft_consensus.cc:2955] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.271018 29146 raft_consensus.cc:2955] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.286187 29910 raft_consensus.cc:1217] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:01.293805 28967 catalog_manager.cc:5697] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.296864 29215 raft_consensus.cc:2955] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.368171 29068 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:01.384048 29911 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:04:01.384807 29146 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:04:01.385752 30271 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:01.386785 30185 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:01.393841 30185 raft_consensus.cc:2955] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.395895 29146 raft_consensus.cc:2955] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.396714 29911 raft_consensus.cc:2955] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.412966 28969 catalog_manager.cc:5697] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New cstate: current_term: 2 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.422708 28955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e250871b4ad74e079a538cdb04065b00 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:01.443446 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet e250871b4ad74e079a538cdb04065b00 with delete_type TABLET_DATA_TOMBSTONED (TS 6ad9faa1c14444b29c9db8e71f46e2d6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42954
I20260709 13:04:01.489737 30320 tablet_replica.cc:333] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:04:01.490979 30320 raft_consensus.cc:2243] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.491878 30320 raft_consensus.cc:2272] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.513309 30320 ts_tablet_manager.cc:1916] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:01.514252 30270 raft_consensus.cc:1064] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:01.517267 30270 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:01.541796 29911 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.544049 29140 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.546355 30185 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.548255 30185 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.557649 30312 raft_consensus.cc:1064] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:01.564656 30312 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:01.582901 30270 raft_consensus.cc:2955] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.592934 29911 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.585155 29140 raft_consensus.cc:2955] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.594419 30271 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.585899 30320 ts_tablet_manager.cc:1929] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:04:01.595602 30320 log.cc:1199] T e250871b4ad74e079a538cdb04065b00 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/wals/e250871b4ad74e079a538cdb04065b00
I20260709 13:04:01.597669 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet e250871b4ad74e079a538cdb04065b00 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:01.608546 29139 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:04:01.610159 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51893us
I20260709 13:04:01.612826 29214 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.613003 29215 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.617751 30185 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.622725 30312 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.624722 30312 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.625914 28966 catalog_manager.cc:5697] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.636296 29908 raft_consensus.cc:2955] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.656240 29215 raft_consensus.cc:2955] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.658596 30328 raft_consensus.cc:2955] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.684211 29214 raft_consensus.cc:2955] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.687463 29911 raft_consensus.cc:2955] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.704873 29140 raft_consensus.cc:2955] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.706540 28966 catalog_manager.cc:5697] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.714161 29071 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:01.721935 29144 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:01.723330 30327 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:01.728642 29911 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:01.732075 30328 raft_consensus.cc:2955] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.736079 29072 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:01.739517 29144 raft_consensus.cc:2955] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.743875 30271 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:01.756549 29908 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:01.767019 30185 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:04:01.762533 29911 raft_consensus.cc:2955] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.777268 29215 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:01.782503 30185 raft_consensus.cc:2955] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.785197 28955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 984bea8952114e3f9c0a045e7f557e37 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:01.785053 29909 raft_consensus.cc:2955] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.796217 28955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 023c90d48e094012bc60b1e600fffa14 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:01.795768 28971 catalog_manager.cc:5697] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.798810 30185 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:01.806245 30163 raft_consensus.cc:1064] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:01.807216 29215 raft_consensus.cc:2955] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.808687 30163 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:01.822069 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet 984bea8952114e3f9c0a045e7f557e37 with delete_type TABLET_DATA_TOMBSTONED (TS 6ad9faa1c14444b29c9db8e71f46e2d6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42954
I20260709 13:04:01.823030 30320 tablet_replica.cc:333] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:04:01.830556 28966 catalog_manager.cc:5697] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.852494 30320 raft_consensus.cc:2243] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.853288 30320 raft_consensus.cc:2272] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.857211 30320 ts_tablet_manager.cc:1916] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:01.859191 29134 tablet_service.cc:1558] Processing DeleteTablet for tablet 023c90d48e094012bc60b1e600fffa14 with delete_type TABLET_DATA_TOMBSTONED (TS 730186a8c70f43f2a57d80163bbc37d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42040
I20260709 13:04:01.864549 29911 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.864549 29214 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.867869 30233 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.869249 29071 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:01.872951 30233 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.874084 30334 tablet_replica.cc:333] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:04:01.875197 30163 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:01.885741 30334 raft_consensus.cc:2243] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:01.886600 30334 raft_consensus.cc:2272] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:01.886729 30320 ts_tablet_manager.cc:1929] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:01.887152 30320 log.cc:1199] T 984bea8952114e3f9c0a045e7f557e37 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/wals/984bea8952114e3f9c0a045e7f557e37
I20260709 13:04:01.896665 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet 984bea8952114e3f9c0a045e7f557e37 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:01.897420 30301 raft_consensus.cc:2955] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.899817 29071 raft_consensus.cc:2955] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.906973 30334 ts_tablet_manager.cc:1916] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:01.902681 29214 raft_consensus.cc:2955] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.917541 29911 raft_consensus.cc:2955] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.934882 30334 ts_tablet_manager.cc:1929] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:01.935437 30334 log.cc:1199] T 023c90d48e094012bc60b1e600fffa14 P 730186a8c70f43f2a57d80163bbc37d2: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/wals/023c90d48e094012bc60b1e600fffa14
I20260709 13:04:01.937290 28958 catalog_manager.cc:5028] TS 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): tablet 023c90d48e094012bc60b1e600fffa14 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:01.951995 28971 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:01.970060 29140 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:01.979046 29909 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:01.980803 30338 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:01.981923 29214 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:01.985009 30301 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:01.993814 29215 raft_consensus.cc:2955] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:02.002703 30301 raft_consensus.cc:2955] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:02.002741 29911 raft_consensus.cc:2955] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:02.018889 28969 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:02.053050 29059 tablet_service.cc:1558] Processing DeleteTablet for tablet 5acfd220cbb54cd181c08050de276f65 with delete_type TABLET_DATA_TOMBSTONED (TS a936878f59c840a6bd176a1d2d7347b1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47946
I20260709 13:04:02.054636 28958 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5acfd220cbb54cd181c08050de276f65 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:02.057159 30340 tablet_replica.cc:333] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:04:02.058210 30340 raft_consensus.cc:2243] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:02.058806 30340 raft_consensus.cc:2272] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:02.062146 30340 ts_tablet_manager.cc:1916] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:02.077483 30340 ts_tablet_manager.cc:1929] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:02.077867 30340 log.cc:1199] T 5acfd220cbb54cd181c08050de276f65 P a936878f59c840a6bd176a1d2d7347b1: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/wals/5acfd220cbb54cd181c08050de276f65
I20260709 13:04:02.079844 28955 catalog_manager.cc:5028] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): tablet 5acfd220cbb54cd181c08050de276f65 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:02.696022 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:02.747845 29214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:02.748433 29214 raft_consensus.cc:606] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Received request to transfer leadership to TS 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:02.760141 29140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:04:02.760792 29140 raft_consensus.cc:606] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Received request to transfer leadership to TS 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:02.763192 29214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:02.763618 29214 raft_consensus.cc:606] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Received request to transfer leadership to TS 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:02.766068 29144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:04:02.766628 29144 raft_consensus.cc:606] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:02.769207 29144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:04:02.769708 29144 raft_consensus.cc:606] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:02.787213 29072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e68315e143764938be60ec716705c15f"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:02.787659 29072 raft_consensus.cc:606] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:02.788745 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:02.789999 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:02.858104 30234 raft_consensus.cc:993] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:02.858935 30322 raft_consensus.cc:1081] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Signalling peer 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:02.860487 29911 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:54878
I20260709 13:04:02.861047 29911 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:02.861384 29911 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.866106 29911 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:02.868199 29911 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:02.869133 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:02.869899 29140 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.869627 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:02.871524 29215 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:02.871886 29215 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:02.872676 29215 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:02.874075 29215 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.877791 29140 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:02.878959 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:02.880527 30346 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:02.880604 29215 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:02.881664 30346 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:02.882911 30346 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:02.890364 28969 catalog_manager.cc:5697] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:02.969205 30301 raft_consensus.cc:993] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:02.969676 30163 raft_consensus.cc:1081] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Signalling peer 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:02.971050 29911 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:54892
I20260709 13:04:02.971465 29911 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:02.971745 29911 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.977278 29911 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:02.979153 29911 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:02.980104 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:02.980515 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:02.980648 29215 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.981045 29140 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:02.981343 29140 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:02.981895 29140 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:02.982713 29140 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.985808 29215 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:02.986964 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:02.987810 30346 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:02.988323 30346 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:02.988353 29140 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:02.989286 30346 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:02.997260 28969 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:03.008934 30163 raft_consensus.cc:993] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:03.009402 30301 raft_consensus.cc:1081] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:03.010762 29215 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42970
I20260709 13:04:03.011219 29215 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:03.011498 29215 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.015846 29215 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:03.017553 29215 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:03.018332 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:03.018507 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:03.018914 29072 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.018982 29140 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:03.020092 29140 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:03.020877 29140 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:03.021916 29140 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.025470 29072 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:03.026669 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:03.027426 30234 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:03.027859 30234 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:03.028200 29140 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:03.028715 30234 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:03.030417 30352 raft_consensus.cc:993] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:03.030901 30322 raft_consensus.cc:1081] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Signalling peer 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:03.032426 29909 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:54878
I20260709 13:04:03.032960 29909 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:03.033351 29909 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.040174 29909 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:03.042480 29909 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:03.042887 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:03.043494 29140 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.049544 29140 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:03.051182 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:03.052023 30346 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:03.052582 30346 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:03.053449 30346 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:03.051769 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:03.054734 29215 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:03.055065 29215 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:03.055639 29215 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:03.057262 29215 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.064267 29215 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:03.057644 28969 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:03.077178 28967 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:03.134655 30270 raft_consensus.cc:993] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: : Instructing follower 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:03.135807 30185 raft_consensus.cc:1081] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Signalling peer 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:03.137578 29144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e68315e143764938be60ec716705c15f"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:42044
I20260709 13:04:03.138044 29144 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:03.137673 29140 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:03.138375 29144 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.144382 29072 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:03.144531 29112 consensus_peers.cc:597] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9f522a55d4b4051a75e0bf87cd2619e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:03.145372 29215 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:03.145668 30301 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:03.145658 29144 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:03.147306 30163 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.147619 29144 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:03.150096 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:03.150586 29213 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.150909 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:03.151490 29072 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:03.151785 29072 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:03.152555 29072 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:03.153760 29072 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.156281 29213 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:03.157191 30301 raft_consensus.cc:993] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:03.157989 29113 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:03.160145 30357 raft_consensus.cc:1081] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:03.160255 30163 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:03.160817 30163 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:03.162482 29072 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:03.161648 30163 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:03.163022 30357 raft_consensus.cc:2955] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.164943 29072 raft_consensus.cc:2955] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.166756 29215 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42970
I20260709 13:04:03.167276 29215 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:03.167613 29215 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.184031 29215 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:03.185714 28969 catalog_manager.cc:5697] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.186642 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:03.187182 29072 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.189467 29215 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:03.214128 28967 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:03.221421 29215 raft_consensus.cc:2955] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:04:03.227900 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48742.6us
I20260709 13:04:03.232189 29072 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:03.233242 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:03.233970 30234 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:03.234331 30234 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:03.234964 30234 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:03.235338 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:03.235961 29144 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:03.236267 29144 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:03.236986 29144 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:03.239338 29144 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:03.241175 29213 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:03.246398 29144 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
W20260709 13:04:03.261987 29186 consensus_peers.cc:597] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c02e27bf25948329604215b11034f44. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:03.265753 29144 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:03.266359 29068 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:03.267827 30352 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.268770 30234 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.279430 28967 catalog_manager.cc:5697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:03.283262 30352 raft_consensus.cc:2955] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.285144 29144 raft_consensus.cc:2955] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.286821 29068 raft_consensus.cc:2955] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.308108 28967 catalog_manager.cc:5697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.312866 29144 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:03.321537 29213 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:03.321637 29112 consensus_peers.cc:597] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3aaa941bea174ae08dc8aad56e0425f2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:03.324204 29068 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:03.324498 30357 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:03.326313 30163 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.339097 30163 raft_consensus.cc:2955] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.344002 29068 raft_consensus.cc:2955] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.346935 29213 raft_consensus.cc:2955] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.353935 29140 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:04:03.363234 29112 consensus_peers.cc:597] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a57ab99b58a448de88023e7166f78ce2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:03.363901 29214 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:03.366406 30163 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.360884 28971 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:03.375422 29072 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:03.378400 30357 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.395990 30163 raft_consensus.cc:2955] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.400619 29064 raft_consensus.cc:2955] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.410238 29214 raft_consensus.cc:2955] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.421245 28971 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:03.474880 29213 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:03.476091 30346 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:03.500603 29144 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:03.502225 30347 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:03.512642 29140 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:03.515563 30347 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:03.518407 29214 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:03.523689 29139 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:03.525300 30234 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.556643 29064 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:03.564474 30347 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:03.565958 30352 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.573324 29072 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:03.581735 29214 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:03.583380 30163 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.589289 30357 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.600868 30385 ts_tablet_manager.cc:933] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:03.634024 29146 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:03.645602 29213 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:03.652024 30377 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:03.654528 30377 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:03.659932 29072 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:03.662772 30345 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.677459 29139 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:03.686252 30352 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:03.687244 30385 tablet_copy_client.cc:323] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.130:35337
I20260709 13:04:03.689183 29157 tablet_copy_service.cc:140] P 730186a8c70f43f2a57d80163bbc37d2: Received BeginTabletCopySession request for tablet a9f522a55d4b4051a75e0bf87cd2619e from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:60830)
I20260709 13:04:03.689742 29157 tablet_copy_service.cc:161] P 730186a8c70f43f2a57d80163bbc37d2: Beginning new tablet copy session on tablet a9f522a55d4b4051a75e0bf87cd2619e from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:60830: session id = 7e78ffbcc98242068a874a5d9e341477-a9f522a55d4b4051a75e0bf87cd2619e
I20260709 13:04:03.698410 29157 tablet_copy_source_session.cc:215] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:03.702028 30385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9f522a55d4b4051a75e0bf87cd2619e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:03.756484 30385 tablet_copy_client.cc:806] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:03.757093 30385 tablet_copy_client.cc:670] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:03.774706 30385 tablet_copy_client.cc:538] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:03.790946 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:03.803817 30385 tablet_bootstrap.cc:492] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:03.855619 30395 ts_tablet_manager.cc:933] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:03.869304 30396 ts_tablet_manager.cc:933] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:03.879824 30395 tablet_copy_client.cc:323] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.130:35337
I20260709 13:04:03.878123 30397 ts_tablet_manager.cc:933] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:03.892521 30385 tablet_bootstrap.cc:492] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:03.895489 30385 tablet_bootstrap.cc:492] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:03.896802 29157 tablet_copy_service.cc:140] P 730186a8c70f43f2a57d80163bbc37d2: Received BeginTabletCopySession request for tablet 3aaa941bea174ae08dc8aad56e0425f2 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:60830)
I20260709 13:04:03.897758 29157 tablet_copy_service.cc:161] P 730186a8c70f43f2a57d80163bbc37d2: Beginning new tablet copy session on tablet 3aaa941bea174ae08dc8aad56e0425f2 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:60830: session id = 7e78ffbcc98242068a874a5d9e341477-3aaa941bea174ae08dc8aad56e0425f2
I20260709 13:04:03.898967 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:03.907181 29157 tablet_copy_source_session.cc:215] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:03.909241 30385 ts_tablet_manager.cc:1403] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.106s	user 0.018s	sys 0.006s
I20260709 13:04:03.910300 30395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aaa941bea174ae08dc8aad56e0425f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:03.909997 30397 tablet_copy_client.cc:323] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.130:35337
I20260709 13:04:03.914177 30385 raft_consensus.cc:359] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:03.921427 29157 tablet_copy_service.cc:140] P 730186a8c70f43f2a57d80163bbc37d2: Received BeginTabletCopySession request for tablet a57ab99b58a448de88023e7166f78ce2 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:60830)
I20260709 13:04:03.922020 29157 tablet_copy_service.cc:161] P 730186a8c70f43f2a57d80163bbc37d2: Beginning new tablet copy session on tablet a57ab99b58a448de88023e7166f78ce2 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:60830: session id = 7e78ffbcc98242068a874a5d9e341477-a57ab99b58a448de88023e7166f78ce2
I20260709 13:04:03.924775 30396 tablet_copy_client.cc:323] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.131:45851
I20260709 13:04:03.921694 30385 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:03.928916 29232 tablet_copy_service.cc:140] P 6ad9faa1c14444b29c9db8e71f46e2d6: Received BeginTabletCopySession request for tablet 2c02e27bf25948329604215b11034f44 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:51820)
I20260709 13:04:03.929507 29232 tablet_copy_service.cc:161] P 6ad9faa1c14444b29c9db8e71f46e2d6: Beginning new tablet copy session on tablet 2c02e27bf25948329604215b11034f44 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:51820: session id = 7e78ffbcc98242068a874a5d9e341477-2c02e27bf25948329604215b11034f44
I20260709 13:04:03.929375 30385 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:03.936014 30395 tablet_copy_client.cc:806] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:03.936671 30395 tablet_copy_client.cc:670] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:03.938841 29232 tablet_copy_source_session.cc:215] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:03.941851 30385 ts_tablet_manager.cc:1434] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 13:04:03.948267 29158 tablet_copy_service.cc:342] P 730186a8c70f43f2a57d80163bbc37d2: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-a9f522a55d4b4051a75e0bf87cd2619e received from {username='slave'} at 127.0.0.1:60830
I20260709 13:04:03.948693 29158 tablet_copy_service.cc:434] P 730186a8c70f43f2a57d80163bbc37d2: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-a9f522a55d4b4051a75e0bf87cd2619e on tablet a9f522a55d4b4051a75e0bf87cd2619e with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:03.951079 30395 tablet_copy_client.cc:538] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:03.953023 29157 tablet_copy_source_session.cc:215] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:03.956269 30397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a57ab99b58a448de88023e7166f78ce2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:03.962599 30395 tablet_bootstrap.cc:492] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:03.978760 30397 tablet_copy_client.cc:806] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:03.979346 30397 tablet_copy_client.cc:670] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:03.981240 30396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c02e27bf25948329604215b11034f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:03.983083 30397 tablet_copy_client.cc:538] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:03.992532 30397 tablet_bootstrap.cc:492] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:03.995561 30395 tablet_bootstrap.cc:492] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:03.996414 29213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:03.997081 29213 raft_consensus.cc:606] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:03.997207 30395 tablet_bootstrap.cc:492] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:04.000591 30395 ts_tablet_manager.cc:1403] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.038s	user 0.025s	sys 0.000s
I20260709 13:04:04.001365 29072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "813e480732fd4061837e784d4d768b1f"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:04.001956 29072 raft_consensus.cc:606] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:04.006951 30395 raft_consensus.cc:359] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:04.008790 30395 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:04.009690 30395 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:04.012120 29139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:04:04.012727 29139 raft_consensus.cc:606] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Received request to transfer leadership to TS 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:04.013487 30395 ts_tablet_manager.cc:1434] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:04:04.017812 29157 tablet_copy_service.cc:342] P 730186a8c70f43f2a57d80163bbc37d2: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-3aaa941bea174ae08dc8aad56e0425f2 received from {username='slave'} at 127.0.0.1:60830
I20260709 13:04:04.018363 29157 tablet_copy_service.cc:434] P 730186a8c70f43f2a57d80163bbc37d2: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-3aaa941bea174ae08dc8aad56e0425f2 on tablet 3aaa941bea174ae08dc8aad56e0425f2 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:04.020810 29213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc31aa85851e41609957a765b2d40774"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:04.021618 29213 raft_consensus.cc:606] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Received request to transfer leadership to TS 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:04.035890 29072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:04.036562 29072 raft_consensus.cc:606] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Received request to transfer leadership to TS 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:04.038841 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:04.039990 30397 tablet_bootstrap.cc:492] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:04.040486 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:04.040920 30397 tablet_bootstrap.cc:492] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:04.041673 30397 ts_tablet_manager.cc:1403] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.049s	user 0.024s	sys 0.016s
I20260709 13:04:04.045434 30397 raft_consensus.cc:359] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:04.046311 30397 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:04.046980 30397 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:04.049005 30396 tablet_copy_client.cc:806] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:04.049624 30396 tablet_copy_client.cc:670] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:04.049928 30397 ts_tablet_manager.cc:1434] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:04.051906 29157 tablet_copy_service.cc:342] P 730186a8c70f43f2a57d80163bbc37d2: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-a57ab99b58a448de88023e7166f78ce2 received from {username='slave'} at 127.0.0.1:60830
I20260709 13:04:04.052356 29157 tablet_copy_service.cc:434] P 730186a8c70f43f2a57d80163bbc37d2: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-a57ab99b58a448de88023e7166f78ce2 on tablet a57ab99b58a448de88023e7166f78ce2 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:04.059609 30396 tablet_copy_client.cc:538] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:04.073849 30401 raft_consensus.cc:993] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:04.074357 30345 raft_consensus.cc:1081] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Signalling peer 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:04.076135 29910 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc31aa85851e41609957a765b2d40774"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:54878
I20260709 13:04:04.076696 29910 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:04.077044 29910 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.084111 29910 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.086767 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:04.087064 30396 tablet_bootstrap.cc:492] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:04.087332 29072 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.089622 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:04.090201 29213 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:04.090363 29910 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:04.090518 29213 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:04.093953 29072 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:04.091682 29213 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.095254 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:04.095758 29213 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.096699 30346 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:04.097193 30346 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:04.098043 30346 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.101606 29910 raft_consensus.cc:1217] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:04.102439 29213 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:04.116864 28971 catalog_manager.cc:5697] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.123817 30396 tablet_bootstrap.cc:492] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:04.124805 30396 tablet_bootstrap.cc:492] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:04.125451 30396 ts_tablet_manager.cc:1403] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.039s	user 0.023s	sys 0.000s
I20260709 13:04:04.128567 30396 raft_consensus.cc:359] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:04.129812 30396 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:04.130504 30396 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:04.133502 30396 ts_tablet_manager.cc:1434] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:04:04.138636 29232 tablet_copy_service.cc:342] P 6ad9faa1c14444b29c9db8e71f46e2d6: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-2c02e27bf25948329604215b11034f44 received from {username='slave'} at 127.0.0.1:51820
I20260709 13:04:04.139067 29232 tablet_copy_service.cc:434] P 6ad9faa1c14444b29c9db8e71f46e2d6: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-2c02e27bf25948329604215b11034f44 on tablet 2c02e27bf25948329604215b11034f44 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:04.206933 29910 raft_consensus.cc:1217] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:04.244374 30401 raft_consensus.cc:993] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:04.244874 30234 raft_consensus.cc:1081] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Signalling peer 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:04.246833 29139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:42054
I20260709 13:04:04.247351 29139 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:04.247680 29139 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.254366 29139 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:04.256214 29139 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:04.256932 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:04.257517 29068 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.257580 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:04.258189 29213 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:04.258486 29213 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:04.259217 29213 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:04.260249 29213 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.263401 29068 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:04.265321 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:04.267295 30357 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:04.267362 29213 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:04.268905 30357 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:04.269658 30357 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:04.281672 28967 catalog_manager.cc:5697] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 2, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:04.317616 30362 raft_consensus.cc:993] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:04.318137 30362 raft_consensus.cc:1081] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Signalling peer 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:04.319613 29910 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:54892
I20260709 13:04:04.320061 29910 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:04.320335 29910 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.325368 29910 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.327955 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:04.328336 29910 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:04.328501 29139 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:04.328877 29139 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:04.330382 29139 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.331573 29139 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.350203 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:04.350821 29213 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.355124 30390 raft_consensus.cc:993] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:04.355662 30390 raft_consensus.cc:1081] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:04.360610 29214 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "813e480732fd4061837e784d4d768b1f"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:04:04.361120 29214 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:04.361429 29214 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.364460 29139 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:04.365628 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:04.366396 30346 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:04.366854 30346 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:04.367668 30346 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.369494 29213 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:04.370433 29214 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:04.372399 29214 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:04.373283 30270 raft_consensus.cc:993] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: : Instructing follower 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:04.373667 29910 raft_consensus.cc:1217] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:04.377434 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:04.377957 29139 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.378109 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:04.378793 29068 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:04.379148 29068 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:04.380155 29068 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:04.381465 29068 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.384446 29139 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:04.373883 30270 raft_consensus.cc:1081] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Signalling peer 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:04.390368 28967 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.393672 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:04.394542 30234 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:04.395614 30234 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:04.396481 30234 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:04.399221 29910 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:54902
I20260709 13:04:04.399847 29910 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:04.400276 29910 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.400760 29068 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:04.406536 29910 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.408571 29910 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:04.410337 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:04.410917 29214 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.417636 28967 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.422292 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:04.422983 29068 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:04.423332 29068 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:04.424094 29068 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.425237 29068 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:04.425953 29214 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:04.426930 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:04.427781 30346 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:04.428300 30346 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:04.429692 30346 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.433787 29910 raft_consensus.cc:1217] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:04.437120 29068 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:04.447328 28967 catalog_manager.cc:5697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.506275 30362 raft_consensus.cc:1064] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:04.509030 30362 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.515908 29910 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:04.516167 29214 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:04.517439 29068 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:04.517891 30357 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:04.519724 30357 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:04.521462 30362 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:04.537407 30357 raft_consensus.cc:2955] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.539057 29910 raft_consensus.cc:2955] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.550030 29068 raft_consensus.cc:2955] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.579264 28967 catalog_manager.cc:5697] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.596125 29214 raft_consensus.cc:2955] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.630613 29144 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.637936 29909 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:04.639393 29214 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:04.639499 30362 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:04.640743 30420 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:04.649833 30419 raft_consensus.cc:2955] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.651647 29909 raft_consensus.cc:2955] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.660995 28958 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a9f522a55d4b4051a75e0bf87cd2619e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:04.669739 28971 catalog_manager.cc:5697] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.670261 29214 raft_consensus.cc:2955] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.685099 29068 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:04.686007 29059 tablet_service.cc:1558] Processing DeleteTablet for tablet a9f522a55d4b4051a75e0bf87cd2619e with delete_type TABLET_DATA_TOMBSTONED (TS a936878f59c840a6bd176a1d2d7347b1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47946
I20260709 13:04:04.687425 30357 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:04.703143 30421 tablet_replica.cc:333] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:04:04.706281 30421 raft_consensus.cc:2243] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:04.707072 30421 raft_consensus.cc:2272] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:04.714339 29072 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:04.715615 30346 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:04.719166 30420 raft_consensus.cc:1064] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:04.723206 30420 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.725660 29214 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:04.727115 30346 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:04.711773 30421 ts_tablet_manager.cc:1916] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:04.767148 29909 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:04.770490 29213 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:04.773322 30357 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:04.775609 30419 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:04.780406 29072 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:04.781980 30420 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:04.785352 29217 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:04.797574 30362 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:04.800359 29144 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:04.801863 30377 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:04.809639 30421 ts_tablet_manager.cc:1929] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:04.810105 30421 log.cc:1199] T a9f522a55d4b4051a75e0bf87cd2619e P a936878f59c840a6bd176a1d2d7347b1: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/wals/a9f522a55d4b4051a75e0bf87cd2619e
I20260709 13:04:04.815801 28955 catalog_manager.cc:5028] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): tablet a9f522a55d4b4051a75e0bf87cd2619e (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:04.832182 29215 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:04.837459 30346 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:04.862229 30163 raft_consensus.cc:1064] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:04.864627 30163 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.908871 30362 raft_consensus.cc:2955] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.913416 29909 raft_consensus.cc:2955] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.917603 29215 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:04.922163 30419 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:04.923692 29072 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:04.925585 29072 raft_consensus.cc:2955] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.929440 30420 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:04.945803 28971 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.947821 30420 raft_consensus.cc:2955] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.950855 29214 raft_consensus.cc:2955] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.952049 29064 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:04.962925 29216 raft_consensus.cc:2955] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.969357 29144 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:04.971745 30401 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:04.974325 30345 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:04.974529 29064 raft_consensus.cc:2955] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.946666 29910 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:04.983832 30401 raft_consensus.cc:1064] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:04.986835 29139 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.987114 30401 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:04.991773 29068 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:04.997380 30346 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:05.012418 29910 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:05.015345 30163 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:05.016853 29216 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:05.017907 29222 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:05.019625 30346 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:05.022475 30362 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:05.027546 30357 raft_consensus.cc:2955] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.029315 29216 raft_consensus.cc:2955] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:04.985770 30420 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:05.031981 29908 raft_consensus.cc:2955] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.041450 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:05.055238 29911 raft_consensus.cc:2955] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:04:05.082862 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 96178.8us
I20260709 13:04:05.083611 28969 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.086333 28958 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3aaa941bea174ae08dc8aad56e0425f2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:05.089133 29908 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:05.096238 28966 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:05.100481 30443 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:05.120747 29059 tablet_service.cc:1558] Processing DeleteTablet for tablet 3aaa941bea174ae08dc8aad56e0425f2 with delete_type TABLET_DATA_TOMBSTONED (TS a936878f59c840a6bd176a1d2d7347b1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47946
I20260709 13:04:05.120977 29144 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:05.121672 30421 tablet_replica.cc:333] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:04:05.122656 30421 raft_consensus.cc:2243] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:05.123400 30421 raft_consensus.cc:2272] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:05.127161 30421 ts_tablet_manager.cc:1916] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:05.139604 29064 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:05.141219 30401 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:05.168619 30443 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:05.234403 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:05.229206 30421 ts_tablet_manager.cc:1929] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:05.237104 30421 log.cc:1199] T 3aaa941bea174ae08dc8aad56e0425f2 P a936878f59c840a6bd176a1d2d7347b1: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/wals/3aaa941bea174ae08dc8aad56e0425f2
I20260709 13:04:05.239998 30345 raft_consensus.cc:2955] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.242268 29911 raft_consensus.cc:2955] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.264962 28955 catalog_manager.cc:5028] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): tablet 3aaa941bea174ae08dc8aad56e0425f2 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:05.271850 29144 raft_consensus.cc:2955] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.273777 28967 catalog_manager.cc:5697] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.288966 29064 raft_consensus.cc:2955] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.295969 29222 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:05.302716 29911 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:05.304230 30449 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:05.316381 30443 raft_consensus.cc:2955] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.319649 29908 raft_consensus.cc:2955] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.326193 29140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:04:05.326812 29140 raft_consensus.cc:606] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:05.329954 28957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2c02e27bf25948329604215b11034f44 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:05.334896 29216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:05.335494 29216 raft_consensus.cc:606] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:05.339100 29072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79cdfc6a725145348b70034422ef477b"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:05.339643 29072 raft_consensus.cc:606] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:05.342958 29072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:05.343514 29072 raft_consensus.cc:606] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:05.346262 29068 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:05.346881 29072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:05.347460 29072 raft_consensus.cc:606] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:05.348552 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:05.349323 28971 catalog_manager.cc:5697] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.350086 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:05.351099 30449 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:05.356736 29068 raft_consensus.cc:2955] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.361845 29134 tablet_service.cc:1558] Processing DeleteTablet for tablet 2c02e27bf25948329604215b11034f44 with delete_type TABLET_DATA_TOMBSTONED (TS 730186a8c70f43f2a57d80163bbc37d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42040
I20260709 13:04:05.366572 30453 tablet_replica.cc:333] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:04:05.367756 30453 raft_consensus.cc:2243] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:05.368541 30453 raft_consensus.cc:2272] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:05.373059 30453 ts_tablet_manager.cc:1916] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:05.391168 30453 ts_tablet_manager.cc:1929] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:05.391649 30453 log.cc:1199] T 2c02e27bf25948329604215b11034f44 P 730186a8c70f43f2a57d80163bbc37d2: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/wals/2c02e27bf25948329604215b11034f44
I20260709 13:04:05.397773 28958 catalog_manager.cc:5028] TS 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): tablet 2c02e27bf25948329604215b11034f44 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:05.458869 30185 raft_consensus.cc:993] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:05.459441 30442 raft_consensus.cc:1081] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:05.465659 29216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:04:05.466207 29216 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:05.466578 29216 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:05.475281 29216 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.479063 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:05.479651 29068 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:05.479962 29068 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:05.480291 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:05.480825 29144 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:05.480686 29068 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.481752 29068 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:05.487180 29144 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:05.488026 29068 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:05.488055 29216 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:05.488340 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:05.489281 30443 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:05.489768 30443 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:05.490587 30443 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.501214 28967 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 3 to 4, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 4 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:05.502383 30449 raft_consensus.cc:993] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:05.502866 30443 raft_consensus.cc:1081] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Signalling peer 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:05.504411 29144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:42054
I20260709 13:04:05.504885 29144 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:05.505178 29144 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.511863 29144 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.514418 29144 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:05.515535 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:05.516085 29216 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:05.516294 29216 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:05.517001 29216 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.517871 29216 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.523006 29216 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:05.514206 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:05.524320 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:05.524466 29068 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.525163 30163 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:05.525602 30163 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:05.526391 30163 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.530396 29068 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:05.553679 28967 catalog_manager.cc:5697] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 2, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:05.633430 30185 raft_consensus.cc:993] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: : Instructing follower 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:05.633936 30411 raft_consensus.cc:1081] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Signalling peer 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:05.635653 29144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:42044
I20260709 13:04:05.636313 29144 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:05.636750 29144 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.643631 29144 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.646301 29144 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:05.652536 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:05.653157 29216 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.658757 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:05.659379 29068 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:05.659691 29068 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:05.660498 29068 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.661499 29068 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.663167 29216 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:05.664314 29113 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:05.665625 30163 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:05.666141 30163 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:05.666903 30163 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.667935 29140 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:05.669993 29068 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:05.674160 29911 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:05.675019 29064 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:05.676488 30420 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:05.683262 30357 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:05.685490 28967 catalog_manager.cc:5697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:05.691102 30411 raft_consensus.cc:993] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:05.691615 30411 raft_consensus.cc:1081] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:05.693905 30362 raft_consensus.cc:2955] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.694932 29911 raft_consensus.cc:2955] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.700120 29222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79cdfc6a725145348b70034422ef477b"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:04:05.700711 29222 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:05.701061 29222 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.709156 29222 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.713070 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:05.713711 29068 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:05.714031 29068 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:05.712317 29064 raft_consensus.cc:2955] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:05.714804 29068 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.715770 29068 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.713111 28958 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a57ab99b58a448de88023e7166f78ce2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:05.724509 29068 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:05.726303 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:05.727156 30234 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:05.727619 30234 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:05.728399 30234 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.734160 28969 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 3 to 4, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New cstate: current_term: 1 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:05.746672 29222 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:05.754307 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet a57ab99b58a448de88023e7166f78ce2 with delete_type TABLET_DATA_TOMBSTONED (TS 6ad9faa1c14444b29c9db8e71f46e2d6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42954
I20260709 13:04:05.759420 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:05.760093 29140 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.766356 28969 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:05.769371 29140 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:05.776170 30458 tablet_replica.cc:333] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:04:05.777189 30458 raft_consensus.cc:2243] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:05.777853 30458 raft_consensus.cc:2272] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:05.782387 30458 ts_tablet_manager.cc:1916] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:05.797824 30458 ts_tablet_manager.cc:1929] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:05.798317 30458 log.cc:1199] T a57ab99b58a448de88023e7166f78ce2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/wals/a57ab99b58a448de88023e7166f78ce2
I20260709 13:04:05.801302 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet a57ab99b58a448de88023e7166f78ce2 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:05.834177 30420 raft_consensus.cc:993] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:05.834740 30163 raft_consensus.cc:1081] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:05.836098 29222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42970
I20260709 13:04:05.836556 29222 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:05.836843 29222 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:05.841208 29222 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.842826 29222 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:05.843508 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:05.843724 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:05.844079 29068 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:05.844295 29140 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:05.844632 29140 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:05.845323 29140 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.846280 29140 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:05.848721 29068 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:05.849730 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:05.850438 30449 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:05.850829 29140 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:05.850834 30449 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:05.851850 30449 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:05.860952 28969 catalog_manager.cc:5697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 2 to 3, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 3 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:05.975054 29222 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:05.976282 30420 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:05.988077 29064 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:05.990818 30362 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:06.043434 29064 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:06.044818 30449 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:06.048228 29140 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:06.050597 30234 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:06.079085 29216 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:06.081395 30420 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:06.094395 29068 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:06.096803 30420 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:06.151825 29144 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:06.153213 30443 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:06.162472 29068 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:06.164410 30443 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:06.234675 29144 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:06.235842 30234 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:06.245124 29068 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:06.246582 30234 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:06.351024 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:06.397501 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:06.434808 29222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:06.435415 29222 raft_consensus.cc:606] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Received request to transfer leadership to TS 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:06.437943 29222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c02e27bf25948329604215b11034f44"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:06.438429 29222 raft_consensus.cc:606] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Received request to transfer leadership to TS 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:06.440619 29144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:04:06.440999 29144 raft_consensus.cc:606] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:06.443060 29068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:06.443536 29068 raft_consensus.cc:606] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:06.445895 29144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:04:06.446362 29144 raft_consensus.cc:606] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Received request to transfer leadership to TS 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:06.448484 29222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:06.449117 29222 raft_consensus.cc:606] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:06.450867 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:06.452193 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:06.566486 30163 raft_consensus.cc:993] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:06.567008 30420 raft_consensus.cc:1081] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Signalling peer 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:06.568529 29908 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:54892
I20260709 13:04:06.569041 29908 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:06.569407 29908 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.579707 29908 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:06.582281 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:06.582891 29068 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.582777 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:06.583464 29140 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:06.583652 29908 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:06.583767 29140 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:06.584595 29140 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:06.585731 29140 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.589720 29068 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:06.590674 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:06.591387 30377 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:06.591861 30377 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:06.592500 29140 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:06.592721 30377 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:06.591813 30234 raft_consensus.cc:993] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:06.595660 30345 raft_consensus.cc:1081] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Signalling peer 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:06.597048 29140 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:42054
I20260709 13:04:06.597472 29140 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:06.597863 29140 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.600797 28969 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:06.604688 29140 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:06.606565 29140 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:06.607583 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:06.608181 29222 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:06.608482 29222 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:06.609233 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:06.609838 29068 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.609334 29222 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:06.611056 29222 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.615379 29068 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:06.617841 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:06.618607 30420 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:06.619395 30420 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:06.620157 29222 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:06.620779 30420 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:06.633303 28969 catalog_manager.cc:5697] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 2, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:06.667374 30163 raft_consensus.cc:993] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:06.667840 30420 raft_consensus.cc:1081] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:06.669198 29222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42970
I20260709 13:04:06.669644 29222 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:06.669922 29222 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.674520 29222 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:06.676095 29222 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:06.677273 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:06.677417 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:06.677917 29068 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.677986 29140 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:06.678339 29140 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:06.679191 29140 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:06.680361 29140 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.684526 29068 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:06.685720 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:06.686635 30345 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:06.686676 29140 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:06.687330 30345 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:06.688280 30345 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:06.703447 28969 catalog_manager.cc:5697] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:06.707891 30459 raft_consensus.cc:993] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:06.708526 30459 raft_consensus.cc:1081] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:06.713871 29216 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:04:06.714470 29216 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:06.714838 29216 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.722847 29216 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:06.725543 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:06.726171 29068 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:06.726436 29216 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:06.726529 29068 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:06.727981 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:06.728552 29140 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.728193 29068 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:06.729641 29068 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.735965 29140 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:06.736683 29068 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:06.737042 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:06.737860 30345 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:06.738427 30345 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:06.739259 30345 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:06.749742 28969 catalog_manager.cc:5697] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:06.825520 29140 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:06.831686 29068 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:06.831631 29216 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:04:06.832224 29112 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e68315e143764938be60ec716705c15f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.833122 30163 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:06.834151 30420 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:06.845043 30163 raft_consensus.cc:2955] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:06.846403 29064 raft_consensus.cc:2955] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:06.851830 29216 raft_consensus.cc:2955] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:06.861250 28969 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index -1 to 3, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:06.863318 29214 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:04:06.878209 29186 consensus_peers.cc:597] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0107c7a468e44f6b9ebd1ced2ae1ed1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.882349 29140 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:06.883245 29068 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:06.883781 30345 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:06.885205 30443 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:06.896629 30234 raft_consensus.cc:993] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:06.897505 30443 raft_consensus.cc:1081] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Signalling peer 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:06.899176 29908 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c02e27bf25948329604215b11034f44"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:54878
I20260709 13:04:06.899902 29908 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:06.900415 29908 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.907455 30443 raft_consensus.cc:2955] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:06.911773 29140 raft_consensus.cc:2955] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:06.915859 29908 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:06.925813 29908 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:06.927750 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:06.928331 29214 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:06.928597 29214 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:06.929311 29214 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:06.930464 29214 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.931670 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:06.932292 29064 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.937039 30234 raft_consensus.cc:993] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:06.938014 29214 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:06.939242 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:06.940294 30473 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:06.940941 30473 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:06.942092 30473 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:06.942951 28971 catalog_manager.cc:5697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:06.939409 30345 raft_consensus.cc:1081] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Signalling peer 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:04:06.945753 29064 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:06.945984 29908 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:54878
I20260709 13:04:06.946616 29908 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:06.946988 29908 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.955989 29908 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:06.961410 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:06.961459 29908 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:06.962245 29140 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.968304 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:06.968163 29068 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:06.968888 29214 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:06.969363 29214 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:06.968814 28969 catalog_manager.cc:5697] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:06.966095 29064 raft_consensus.cc:2955] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:06.970185 29214 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:06.977073 29214 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.984639 29140 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:06.985811 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:06.989907 30473 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:06.992121 30473 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:06.992412 29216 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:06.993091 30473 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:06.997400 29214 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:06.999106 29140 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:06.994416 30411 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:07.007905 29037 consensus_peers.cc:597] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 348ae3050d97486fa402a5da2bbdce46. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.008558 30459 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:07.023855 30479 raft_consensus.cc:2955] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:07.025180 29214 raft_consensus.cc:2955] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:07.028208 28969 catalog_manager.cc:5697] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:07.042903 28971 catalog_manager.cc:5697] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:07.047142 29214 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:07.047778 29139 raft_consensus.cc:2955] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:07.055500 29144 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:04:07.059067 29186 consensus_peers.cc:597] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 344aa5f73f0f4124b1512832a0bca2c9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:07.060081 30443 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:07.064889 29064 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:07.066941 30473 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:07.073683 29068 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:04:07.076053 30443 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:07.110172 30234 raft_consensus.cc:2955] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:07.112797 29144 raft_consensus.cc:2955] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:07.111866 29068 raft_consensus.cc:2955] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:07.123077 29140 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:07.131069 29072 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:07.132491 30420 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:07.133664 30443 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:07.135680 28971 catalog_manager.cc:5697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 2, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:07.138592 29071 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:07.140763 30487 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:07.180182 29139 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:07.185063 30473 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:07.159984 29215 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:07.235404 30163 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:07.287997 29144 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:07.297477 30487 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:07.327036 29072 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:07.327898 30513 ts_tablet_manager.cc:933] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:07.330256 30345 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:07.346422 30513 tablet_copy_client.cc:323] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.131:45851
I20260709 13:04:07.349246 29232 tablet_copy_service.cc:140] P 6ad9faa1c14444b29c9db8e71f46e2d6: Received BeginTabletCopySession request for tablet d0107c7a468e44f6b9ebd1ced2ae1ed1 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:51820)
I20260709 13:04:07.349844 29232 tablet_copy_service.cc:161] P 6ad9faa1c14444b29c9db8e71f46e2d6: Beginning new tablet copy session on tablet d0107c7a468e44f6b9ebd1ced2ae1ed1 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:51820: session id = 7e78ffbcc98242068a874a5d9e341477-d0107c7a468e44f6b9ebd1ced2ae1ed1
I20260709 13:04:07.369805 29232 tablet_copy_source_session.cc:215] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:07.356207 30516 ts_tablet_manager.cc:933] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:07.377604 30513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0107c7a468e44f6b9ebd1ced2ae1ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:07.385430 30516 tablet_copy_client.cc:323] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.130:35337
I20260709 13:04:07.387184 29157 tablet_copy_service.cc:140] P 730186a8c70f43f2a57d80163bbc37d2: Received BeginTabletCopySession request for tablet e68315e143764938be60ec716705c15f from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:60830)
I20260709 13:04:07.387701 29157 tablet_copy_service.cc:161] P 730186a8c70f43f2a57d80163bbc37d2: Beginning new tablet copy session on tablet e68315e143764938be60ec716705c15f from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:60830: session id = 7e78ffbcc98242068a874a5d9e341477-e68315e143764938be60ec716705c15f
I20260709 13:04:07.395083 29157 tablet_copy_source_session.cc:215] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:07.395998 30513 tablet_copy_client.cc:806] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:07.396903 30513 tablet_copy_client.cc:670] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:07.398082 30516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e68315e143764938be60ec716705c15f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:07.401993 30513 tablet_copy_client.cc:538] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:07.411679 30513 tablet_bootstrap.cc:492] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:07.414173 30516 tablet_copy_client.cc:806] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:07.414747 30516 tablet_copy_client.cc:670] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:07.419667 30516 tablet_copy_client.cc:538] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:07.428305 30516 tablet_bootstrap.cc:492] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:07.442423 30513 tablet_bootstrap.cc:492] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:07.443382 30513 tablet_bootstrap.cc:492] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:07.444132 30513 ts_tablet_manager.cc:1403] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.005s
I20260709 13:04:07.449405 30513 raft_consensus.cc:359] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:07.450328 30513 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:07.451009 30513 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:07.453074 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:07.453801 30513 ts_tablet_manager.cc:1434] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:04:07.455699 29072 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:07.458660 29232 tablet_copy_service.cc:342] P 6ad9faa1c14444b29c9db8e71f46e2d6: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-d0107c7a468e44f6b9ebd1ced2ae1ed1 received from {username='slave'} at 127.0.0.1:51820
I20260709 13:04:07.459071 29232 tablet_copy_service.cc:434] P 6ad9faa1c14444b29c9db8e71f46e2d6: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-d0107c7a468e44f6b9ebd1ced2ae1ed1 on tablet d0107c7a468e44f6b9ebd1ced2ae1ed1 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:07.460415 30473 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:07.478845 29214 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:07.480857 30473 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:07.492801 29140 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:07.493537 30516 tablet_bootstrap.cc:492] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:07.494477 30516 tablet_bootstrap.cc:492] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:07.495971 30516 ts_tablet_manager.cc:1403] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.018s
I20260709 13:04:07.498293 30473 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:07.500384 29215 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:07.503345 30473 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:07.519820 30516 raft_consensus.cc:359] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:07.522763 30516 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:07.523358 30516 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:07.526050 30516 ts_tablet_manager.cc:1434] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:04:07.527601 29157 tablet_copy_service.cc:342] P 730186a8c70f43f2a57d80163bbc37d2: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-e68315e143764938be60ec716705c15f received from {username='slave'} at 127.0.0.1:60830
I20260709 13:04:07.527999 29157 tablet_copy_service.cc:434] P 730186a8c70f43f2a57d80163bbc37d2: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-e68315e143764938be60ec716705c15f on tablet e68315e143764938be60ec716705c15f with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:07.599016 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:07.615538 30516 ts_tablet_manager.cc:933] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:07.617348 30516 tablet_copy_client.cc:323] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.129:41631
I20260709 13:04:07.618945 29082 tablet_copy_service.cc:140] P a936878f59c840a6bd176a1d2d7347b1: Received BeginTabletCopySession request for tablet 348ae3050d97486fa402a5da2bbdce46 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:40862)
I20260709 13:04:07.619498 29082 tablet_copy_service.cc:161] P a936878f59c840a6bd176a1d2d7347b1: Beginning new tablet copy session on tablet 348ae3050d97486fa402a5da2bbdce46 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:40862: session id = 7e78ffbcc98242068a874a5d9e341477-348ae3050d97486fa402a5da2bbdce46
I20260709 13:04:07.632339 29082 tablet_copy_source_session.cc:215] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:07.641259 30516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 348ae3050d97486fa402a5da2bbdce46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:07.666916 30513 ts_tablet_manager.cc:933] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:07.676765 29064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:07.677357 29064 raft_consensus.cc:606] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:07.676340 30513 tablet_copy_client.cc:323] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.131:45851
I20260709 13:04:07.680001 29232 tablet_copy_service.cc:140] P 6ad9faa1c14444b29c9db8e71f46e2d6: Received BeginTabletCopySession request for tablet 344aa5f73f0f4124b1512832a0bca2c9 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:51820)
I20260709 13:04:07.680547 29232 tablet_copy_service.cc:161] P 6ad9faa1c14444b29c9db8e71f46e2d6: Beginning new tablet copy session on tablet 344aa5f73f0f4124b1512832a0bca2c9 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:51820: session id = 7e78ffbcc98242068a874a5d9e341477-344aa5f73f0f4124b1512832a0bca2c9
I20260709 13:04:07.680744 29146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:04:07.681316 29146 raft_consensus.cc:606] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:07.684862 29071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:07.685400 29071 raft_consensus.cc:606] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:07.686869 29908 raft_consensus.cc:1217] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:07.688964 30509 raft_consensus.cc:993] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:07.690068 29232 tablet_copy_source_session.cc:215] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:07.695945 29214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:07.693480 30509 raft_consensus.cc:1081] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:07.701153 29214 raft_consensus.cc:606] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:07.704677 30516 tablet_copy_client.cc:806] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:07.705271 30516 tablet_copy_client.cc:670] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:07.705866 29215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:07.706449 29215 raft_consensus.cc:606] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:07.707780 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:07.709471 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:07.712992 30516 tablet_copy_client.cc:538] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:07.717871 29215 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42970
I20260709 13:04:07.718436 29215 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:07.718786 29215 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.720389 30513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 344aa5f73f0f4124b1512832a0bca2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:07.749722 30516 tablet_bootstrap.cc:492] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:07.751891 30513 tablet_copy_client.cc:806] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:07.755275 30513 tablet_copy_client.cc:670] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:07.760587 30513 tablet_copy_client.cc:538] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:04:07.773519 30513 tablet_bootstrap.cc:492] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:07.790468 30516 tablet_bootstrap.cc:492] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:07.791355 30516 tablet_bootstrap.cc:492] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:07.792032 30516 ts_tablet_manager.cc:1403] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.043s	user 0.022s	sys 0.011s
I20260709 13:04:07.794934 30516 raft_consensus.cc:359] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:07.795780 30516 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:07.796772 30516 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:07.800282 30516 ts_tablet_manager.cc:1434] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:07.812381 30513 tablet_bootstrap.cc:492] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:07.813465 30513 tablet_bootstrap.cc:492] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:07.814275 30513 ts_tablet_manager.cc:1403] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.041s	user 0.030s	sys 0.003s
I20260709 13:04:07.817608 30513 raft_consensus.cc:359] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:07.818544 30513 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:07.819227 30513 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:07.821274 30513 ts_tablet_manager.cc:1434] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:07.823006 29232 tablet_copy_service.cc:342] P 6ad9faa1c14444b29c9db8e71f46e2d6: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-344aa5f73f0f4124b1512832a0bca2c9 received from {username='slave'} at 127.0.0.1:51820
I20260709 13:04:07.823433 29232 tablet_copy_service.cc:434] P 6ad9faa1c14444b29c9db8e71f46e2d6: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-344aa5f73f0f4124b1512832a0bca2c9 on tablet 344aa5f73f0f4124b1512832a0bca2c9 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:07.830737 29082 tablet_copy_service.cc:342] P a936878f59c840a6bd176a1d2d7347b1: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-348ae3050d97486fa402a5da2bbdce46 received from {username='slave'} at 127.0.0.1:40862
I20260709 13:04:07.840672 29082 tablet_copy_service.cc:434] P a936878f59c840a6bd176a1d2d7347b1: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-348ae3050d97486fa402a5da2bbdce46 on tablet 348ae3050d97486fa402a5da2bbdce46 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:07.828918 29908 raft_consensus.cc:1217] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:04:07.849570 29215 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:07.858399 29215 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:07.859457 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:07.859900 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:07.860080 29908 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.860531 29146 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:07.860839 29146 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:07.861543 29146 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:07.862751 29146 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.866209 29908 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:07.867624 29186 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:07.868374 30487 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:07.868794 30487 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:07.869271 29146 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:07.869585 30487 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:07.890787 28967 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:07.931864 30459 raft_consensus.cc:993] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:07.932399 30459 raft_consensus.cc:1081] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:07.936429 29215 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:04:07.936975 29215 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:07.937314 29215 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.945047 29215 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:07.946785 29215 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:07.947470 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:07.947683 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:07.948091 29064 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:07.948226 29908 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.948442 29064 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:07.949211 29064 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:07.950266 29064 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.954459 29908 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:07.955444 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:07.956146 30234 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:07.956579 30234 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:07.957338 30234 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:07.957901 29064 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:07.969810 30487 raft_consensus.cc:993] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower a936878f59c840a6bd176a1d2d7347b1 to start an election
I20260709 13:04:07.970329 30234 raft_consensus.cc:1081] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Signalling peer a936878f59c840a6bd176a1d2d7347b1 to start an election
I20260709 13:04:07.971807 29064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:47974
I20260709 13:04:07.972400 29064 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:07.972780 29064 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:07.974189 28967 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:07.979275 29064 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:07.981220 29064 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:07.983853 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:07.984073 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:07.984427 29214 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:07.984774 29214 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:07.984809 29908 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:07.985543 29214 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:07.986773 29214 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:07.993561 29214 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:07.994594 29908 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:07.994596 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:07.996331 30411 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:07.996721 30411 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:07.997316 30411 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:08.011891 28967 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 3 to 4, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 4 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:08.153426 30459 raft_consensus.cc:993] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:08.153914 30459 raft_consensus.cc:1081] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:08.154083 29908 raft_consensus.cc:1217] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:08.159283 29214 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42964
I20260709 13:04:08.159965 29214 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:08.160358 29214 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.167997 29214 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:08.172811 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:08.173374 29140 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.174787 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:08.175343 29071 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:08.175637 29071 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:08.176337 29071 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:08.177408 29071 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.181788 29140 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:08.185789 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:08.182248 29214 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:08.186640 30234 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.184131 29071 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:08.187808 30234 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:08.190606 30234 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:08.210503 28967 catalog_manager.cc:5697] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:08.233008 30234 raft_consensus.cc:1064] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:08.234977 30234 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:08.242254 29071 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:08.242400 29140 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:08.244138 30345 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:08.245213 29911 raft_consensus.cc:1217] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:04:08.247320 29908 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:08.255589 30234 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:08.260819 30487 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:08.284528 29140 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:08.287163 30234 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:08.289670 30234 raft_consensus.cc:993] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:08.290127 30345 raft_consensus.cc:1081] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Signalling peer 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:08.292999 29144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:42054
I20260709 13:04:08.293522 29144 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:08.293874 29144 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:08.301901 29144 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:08.306598 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:08.308960 29144 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:08.310503 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:08.311249 29214 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:08.311589 29214 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:08.312466 29214 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:08.313704 29214 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:08.319327 30540 raft_consensus.cc:2955] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.323683 29140 raft_consensus.cc:2955] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.339241 29064 raft_consensus.cc:2955] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.348521 28971 catalog_manager.cc:5697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:08.369596 29214 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:08.372892 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:08.373996 29908 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:08.374655 30163 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:08.375144 30163 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:08.375519 30487 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:08.375851 30163 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:08.377245 29911 raft_consensus.cc:2955] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.411226 28967 catalog_manager.cc:5697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 2 to 3, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:08.424239 30362 raft_consensus.cc:1064] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:08.427085 30362 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:08.444075 29064 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:04:08.444327 29908 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:04:08.446245 29214 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:04:08.447436 30362 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:08.448251 30420 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:08.449291 30163 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:08.453285 29214 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:08.472467 29910 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:08.473897 30411 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:08.487272 29911 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:08.488700 29909 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:08.490579 30234 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:08.490830 30345 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:08.494202 29146 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:08.498104 30487 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:08.508883 30539 raft_consensus.cc:2955] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.513455 29911 raft_consensus.cc:2955] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.522204 30420 raft_consensus.cc:2955] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.522305 29140 raft_consensus.cc:2955] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.534287 29064 raft_consensus.cc:2955] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.536943 29071 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:08.538321 30234 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:08.544869 30459 raft_consensus.cc:1064] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:08.547686 30459 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:08.580641 29214 raft_consensus.cc:2955] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.592352 28971 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 3 to 4, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:08.613416 28957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d0107c7a468e44f6b9ebd1ced2ae1ed1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:08.653329 29908 raft_consensus.cc:2955] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:04:08.675418 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 105829us
I20260709 13:04:08.682021 29214 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:08.694454 28969 catalog_manager.cc:5697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.702536 29064 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.710511 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:08.720080 30534 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:08.725469 30539 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:08.766783 29140 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.774416 30539 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:08.786883 29140 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:08.794219 29071 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:08.794304 29910 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:08.796219 30362 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:08.799665 30362 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:08.813040 29059 tablet_service.cc:1558] Processing DeleteTablet for tablet d0107c7a468e44f6b9ebd1ced2ae1ed1 with delete_type TABLET_DATA_TOMBSTONED (TS a936878f59c840a6bd176a1d2d7347b1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47946
I20260709 13:04:08.816288 30163 raft_consensus.cc:2955] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.818966 29911 raft_consensus.cc:2955] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.830821 29214 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:08.830060 29071 raft_consensus.cc:2955] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.834647 30362 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:08.836747 30345 raft_consensus.cc:1064] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:04:08.839425 30345 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:08.858882 28971 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 4 to 5, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:08.879117 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:08.908974 28958 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e68315e143764938be60ec716705c15f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:08.927273 29910 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:08.928735 30234 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:08.931236 29910 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:08.935213 29216 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:08.941849 29140 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:08.948504 30570 tablet_replica.cc:333] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:04:08.949738 30570 raft_consensus.cc:2243] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:08.949572 30558 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:08.960780 30570 raft_consensus.cc:2272] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:08.960683 30557 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:08.982400 29071 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:08.992413 29071 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:08.985675 30540 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:04:09.013077 29071 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:09.016759 30163 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:09.058835 29146 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:04:09.060176 30570 ts_tablet_manager.cc:1916] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:09.065536 30411 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:09.094071 30577 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:09.102481 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet e68315e143764938be60ec716705c15f with delete_type TABLET_DATA_TOMBSTONED (TS 6ad9faa1c14444b29c9db8e71f46e2d6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42954
I20260709 13:04:09.174186 30570 ts_tablet_manager.cc:1929] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:09.197623 30570 log.cc:1199] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P a936878f59c840a6bd176a1d2d7347b1: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/wals/d0107c7a468e44f6b9ebd1ced2ae1ed1
W20260709 13:04:09.154909 30582 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:04:10.569720 30586 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.441s	user 0.001s	sys 0.000s
I20260709 13:04:10.488906 30565 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:10.565897 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 218) took 1576 ms. Trace:
I20260709 13:04:10.573442 29910 rpcz_store.cc:276] 0709 13:04:08.937023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.938048 (+  1025us) service_pool.cc:224] Handling call
0709 13:04:08.938690 (+   642us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.939134 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.939172 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:08.969709 (+ 30537us) log.cc:844] Serialized 564 byte log entry
0709 13:04:09.109037 (+139328us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:09.109085 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.109143 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.513433 (+1404290us) raft_consensus.cc:1685] finished
0709 13:04:10.513468 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.513686 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":137627,"threads_started":1,"wal-append.queue_time_us":539665}
I20260709 13:04:10.571529 30565 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.581811 30565 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:10.582499 30565 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.583089 30565 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.586320 30570 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47946 (request call id 76) took 1754 ms. Trace:
I20260709 13:04:10.594664 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 297) took 1632 ms. Trace:
I20260709 13:04:10.595252 29911 rpcz_store.cc:276] 0709 13:04:08.962730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.963352 (+   622us) service_pool.cc:224] Handling call
0709 13:04:08.968389 (+  5037us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:08.968720 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:08.968742 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:08.969709 (+   967us) log.cc:844] Serialized 564 byte log entry
0709 13:04:09.124060 (+154351us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:09.124096 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.124178 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.594340 (+1470162us) raft_consensus.cc:1685] finished
0709 13:04:10.594370 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.594583 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":154000,"threads_started":1,"wal-append.queue_time_us":1619915}
W20260709 13:04:10.604393 29872 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.533s	user 0.050s	sys 0.110s
W20260709 13:04:10.604935 29872 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.553s	user 0.051s	sys 0.110s
I20260709 13:04:10.601214 30583 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.613454 30583 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.591068 30570 rpcz_store.cc:276] 0709 13:04:08.812635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.812870 (+   235us) service_pool.cc:224] Handling call
0709 13:04:08.948122 (+135252us) ts_tablet_manager.cc:1151] Deleting tablet d0107c7a468e44f6b9ebd1ced2ae1ed1
0709 13:04:08.948185 (+    63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:04:09.090885 (+142700us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:09.173939 (+ 83054us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:10.493711 (+1319772us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10208,"tablet-delete.queue_time_us":134584,"thread_start_us":113000,"threads_started":1}
I20260709 13:04:10.615339 30565 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:10.672529 30565 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:10.676496 30565 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.685883 30588 tablet_replica.cc:333] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:04:10.616856 30583 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.688613 30588 raft_consensus.cc:2243] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:10.690035 30587 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.690922 30588 raft_consensus.cc:2272] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:10.691346 30587 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.695317 30587 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:10.688511 30583 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.698271 30587 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:10.702641 30583 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:10.697466 29204 thread.cc:641] tablet-delete [worker] (thread pool) Time spent creating pthread: real 1.593s	user 0.031s	sys 0.037s
W20260709 13:04:10.704077 29204 thread.cc:608] tablet-delete [worker] (thread pool) Time spent starting thread: real 1.601s	user 0.033s	sys 0.039s
I20260709 13:04:10.705252 30583 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:10.705997 30565 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:10.706941 30587 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.696732 30588 ts_tablet_manager.cc:1916] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:10.709620 30565 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.710556 30565 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.711892 30587 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.713022 30565 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:10.713724 29874 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.179s	user 0.000s	sys 0.038s
W20260709 13:04:10.714113 29874 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.000s	sys 0.038s
I20260709 13:04:10.717077 30593 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.717978 30593 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.719197 30587 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.720278 30587 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.721212 30593 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.723053 30587 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.729260 30583 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.730449 30583 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.732195 30591 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:10.733803 30583 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.734695 30565 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.733286 30591 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.735991 30565 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.738230 30591 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.739254 30565 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:10.749714 30587 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:10.750837 30587 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.752182 30583 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.753153 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1592) took 1678 ms. Trace:
I20260709 13:04:10.753530 30583 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.753753 29144 rpcz_store.cc:276] 0709 13:04:09.074820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.076622 (+  1802us) service_pool.cc:224] Handling call
0709 13:04:09.077270 (+   648us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.077680 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.077717 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.078815 (+  1098us) log.cc:844] Serialized 564 byte log entry
0709 13:04:09.128062 (+ 49247us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:09.128095 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.128149 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.752410 (+1624261us) raft_consensus.cc:1685] finished
0709 13:04:10.752471 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.752837 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":48936,"threads_started":1,"wal-append.queue_time_us":49302}
I20260709 13:04:10.756599 30593 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:10.757668 30583 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.757517 30593 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.761525 30593 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:10.766988 30594 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:10.767871 30594 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.770841 30594 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:10.771338 30587 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:10.780174 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1346) took 1649 ms. Trace:
I20260709 13:04:10.780731 29216 rpcz_store.cc:276] 0709 13:04:09.130960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.131099 (+   139us) service_pool.cc:224] Handling call
0709 13:04:10.779078 (+1647979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.779511 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.779547 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:10.779608 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:10.779657 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.779757 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.779891 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.784111 30595 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.785192 30595 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.788403 30595 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.797946 30595 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.800486 30595 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.804567 30588 ts_tablet_manager.cc:1929] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:04:10.805305 30588 log.cc:1199] T e68315e143764938be60ec716705c15f P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/wals/e68315e143764938be60ec716705c15f
I20260709 13:04:10.813257 30596 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:10.814338 30596 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.817526 30596 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:10.820698 30588 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42954 (request call id 74) took 1739 ms. Trace:
I20260709 13:04:10.821173 30588 rpcz_store.cc:276] 0709 13:04:09.080793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.088925 (+  8132us) service_pool.cc:224] Handling call
0709 13:04:10.684781 (+1595856us) ts_tablet_manager.cc:1151] Deleting tablet e68315e143764938be60ec716705c15f
0709 13:04:10.684955 (+   174us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:04:10.776189 (+ 91234us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:10.804432 (+ 28243us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:10.820527 (+ 16095us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1581606,"thread_start_us":1601326,"threads_started":1}
I20260709 13:04:10.823513 30594 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.824335 30594 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.826660 30594 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.827175 30594 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:10.830184 30596 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.831967 30596 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.833138 30597 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.834059 30597 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.834636 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1368) took 1814 ms. Trace:
I20260709 13:04:10.827831 30594 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.835155 29071 rpcz_store.cc:276] 0709 13:04:09.019623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.026930 (+  7307us) service_pool.cc:224] Handling call
0709 13:04:09.027204 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:09.027482 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.027509 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:09.027897 (+   388us) log.cc:844] Serialized 26 byte log entry
0709 13:04:09.097793 (+ 69896us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:09.098300 (+   507us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.098747 (+   447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.098829 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.833988 (+1735159us) raft_consensus.cc:1685] finished
0709 13:04:10.834031 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.834297 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":100,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":4814,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":12784,"thread_start_us":69585,"threads_started":1,"wal-append.queue_time_us":70171}
I20260709 13:04:10.838189 30594 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:10.839262 30595 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.841286 30597 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.842695 30598 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.843597 30598 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.849223 30598 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.851434 30600 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:10.852535 30600 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.855638 30600 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:10.856276 30600 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:10.857180 30600 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.859845 30600 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:10.865891 30602 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.866818 30602 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.869935 30602 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.870682 30600 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:10.871539 30600 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.872561 30601 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:10.881083 30600 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:10.881847 30599 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:10.909749 30599 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.910311 30601 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.917793 30601 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:10.918700 30599 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:10.884735 30602 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:10.920042 30602 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:10.959110 30606 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.962054 30597 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.963217 30597 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.966331 30597 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.960320 30606 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.971552 30606 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.974445 30597 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.977738 30598 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.978252 30597 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.978442 30598 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.980527 30597 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.977741 30595 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.982190 30596 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.980545 30598 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.982326 30595 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.983788 30602 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:10.984671 30595 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.989562 30607 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:10.989770 30608 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.991246 30596 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.989583 30597 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:10.980666 30594 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:10.994179 30594 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:10.996667 30594 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.991110 30608 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.005522 30608 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:10.992339 30596 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.007385 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1348) took 1874 ms. Trace:
I20260709 13:04:11.008209 30596 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:10.993376 30597 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.007979 29216 rpcz_store.cc:276] 0709 13:04:11.003590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.005958 (+  2368us) service_pool.cc:224] Handling call
0709 13:04:11.006476 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:11.006948 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.006986 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:11.007039 (+    53us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:11.007091 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:11.007162 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:11.007292 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:11.010890 30597 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.000579 30607 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.013615 30607 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:10.992509 30602 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.015123 30606 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.015403 30602 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.015890 30606 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.017864 30602 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.018020 30606 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.044296 30614 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.048908 30597 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.057473 30597 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.045957 30607 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.045008 30614 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.030280 30612 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:11.059646 30607 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.070369 30603 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.071170 30603 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.066723 30597 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.072731 30606 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.069829 30595 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.082742 30606 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.083396 30595 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.085090 30606 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.090703 30621 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:11.091424 30621 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.093823 30621 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.086253 30595 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.116493 30613 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:11.116720 30609 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.096086 30597 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.117631 30609 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.117359 30613 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.095759 30612 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.140136 30612 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.134011 30621 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:11.140836 30610 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.149891 30611 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.124779 30596 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.246065 30623 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.246374 30596 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.136185 30595 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.142611 30597 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.142515 30610 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.260246 30607 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:11.122786 30616 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.259956 30595 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.247041 30623 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.247107 30599 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:11.261668 30608 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.290323 30603 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:11.143273 30621 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.150811 30611 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.279713 30598 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.275224 30618 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.317767 30616 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.318838 30596 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.319618 30596 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.320312 30596 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.325373 30595 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.325841 30614 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.326402 30595 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.327518 30598 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.329725 30596 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.330580 30598 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.331825 30597 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.334254 30596 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:11.334995 30596 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.337225 30596 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.337479 30610 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.337761 30596 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.338403 30596 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.353216 30622 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:11.354053 30622 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.350318 30598 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.339241 30603 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.281361 30607 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.295671 30612 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.313124 30620 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.339363 30613 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.365137 30598 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.369416 30602 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.314785 30608 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.331650 30595 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.367245 30599 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.385004 30616 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.394656 30624 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.394888 30625 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:11.384048 30610 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.398201 30606 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.401889 30606 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.402506 28967 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 234) took 2293 ms. Trace:
I20260709 13:04:11.402887 30598 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.428413 30630 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:11.322120 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet e68315e143764938be60ec716705c15f (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:11.384290 30618 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.397080 30613 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.388135 30607 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.395454 30624 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.388298 30603 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.395630 30625 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.389609 30602 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.393793 30612 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.402683 30627 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:11.402926 28967 rpcz_store.cc:276] 0709 13:04:09.109105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.110980 (+  1875us) service_pool.cc:224] Handling call
0709 13:04:11.402262 (+2291282us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":2944}
I20260709 13:04:11.514092 30634 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.571949 30635 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.573073 30635 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.451943 30595 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.429637 30619 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.482705 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:11.451982 30608 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.402531 30626 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.482575 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:11.482914 30609 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.483107 30598 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:11.563046 30611 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:11.483283 28957 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 161323us
I20260709 13:04:11.515027 30634 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:11.483453 30610 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.570678 30622 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.606981 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:11.621819 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:11.399020 30620 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.607288 30596 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.595851 30623 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.595840 30619 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.596334 30627 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.644734 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:11.443023 30630 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.776515 30611 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.777518 30611 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.780229 30611 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.788571 30617 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.789371 30617 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.792193 30611 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.792989 30611 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.795583 30611 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.802855 30611 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.803761 30611 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.814728 30640 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:11.815495 30640 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.806360 30611 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.828786 30611 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.829567 30611 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.832315 30611 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.657816 30632 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.644495 30635 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.883121 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:11.656642 30602 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.886788 29218 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 1.
I20260709 13:04:11.654122 29222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:11.894119 30604 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.894325 29222 raft_consensus.cc:606] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:11.656466 30621 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.657307 30599 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.654263 30618 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.659142 30612 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:11.607167 30613 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.655262 30600 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.657723 30606 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.654460 30587 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.905428 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:11.655177 30615 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.912398 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:11.655360 30598 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.910058 30600 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.907533 30621 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:11.895017 30604 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.834667 30611 raft_consensus.cc:2955] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:11.919888 30587 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.921586 30600 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:11.921365 30621 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:11.924280 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:11.921728 29222 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 1.
I20260709 13:04:11.913932 30607 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:11.912328 30625 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.923763 30612 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.923938 30613 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.687053 30626 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.925967 29213 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 1.
I20260709 13:04:11.916244 30603 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.908218 30632 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.932956 29221 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:11.928823 30587 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.923802 30627 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.934926 30607 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:11.934414 30610 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.939038 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:11.942425 29146 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 1.
I20260709 13:04:11.941432 30598 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:11.925390 30615 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.948698 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:11.949018 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:11.949469 29147 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:11.950810 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:11.951025 30617 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.951449 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:11.943584 30624 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:11.940315 30612 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:11.947799 30634 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:11.951460 29148 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:04:11.946708 30607 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.034102 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:11.968114 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.034849 29072 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:11.968777 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.038810 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:11.944666 30630 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:12.041978 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.041234 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.041530 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:11.968611 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:11.983834 30640 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:12.043076 29072 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:12.045711 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.046005 30615 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:12.046166 29907 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:04:12.044454 30604 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:12.046659 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.047350 29065 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:12.047528 29140 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:12.048761 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.050693 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.052165 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.041978 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:11.949091 30620 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:12.055611 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.056041 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.056192 29072 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:12.056660 29070 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:12.056798 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.053378 29904 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:12.057456 29066 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:12.052788 29071 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:12.058779 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.058879 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.059350 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.059523 29073 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:12.054754 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.061291 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.069590 29070 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:12.061515 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.059967 29141 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:12.073351 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.073037 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.098925 29142 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:12.100003 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:11.968425 30626 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:12.062850 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.063012 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.063694 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.065676 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.066394 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:12.068404 29243 debug-util.cc:398] Leaking SignalData structure 0x7b0800301020 after lost signal to thread 27937
I20260709 13:04:12.070755 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.107883 29068 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:12.110124 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:12.110442 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08003002c0 after lost signal to thread 29005
I20260709 13:04:12.071973 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.104704 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.104992 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.104991 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.108562 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.108863 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.111088 29139 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:12.109972 29148 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:12.112278 29071 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:12.112453 29073 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:12.113691 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.115166 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.115907 29072 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:12.116168 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.117530 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.117707 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.118837 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.118952 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.119457 29148 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:04:12.119305 29141 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 1.
I20260709 13:04:12.121088 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.114398 29069 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:12.119887 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.122766 29037 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.125953 29142 raft_consensus.cc:2955] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:12.126804 30632 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:12.129284 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.129524 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.130225 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.131197 29064 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:12.132131 30640 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:12.132403 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.132573 30640 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:12.133033 30640 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.133066 29068 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:12.136368 30634 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:12.136905 30634 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:12.137403 30634 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.138803 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.139197 28968 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 257) took 1033 ms. Trace:
I20260709 13:04:12.139525 29145 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:12.140822 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.141426 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.142794 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.143224 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.143584 29140 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:04:12.144184 30617 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:12.144554 30617 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.144927 30617 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.149370 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.150054 30565 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:12.150481 29146 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:04:12.150771 30565 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.153695 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.154531 29217 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 1.
I20260709 13:04:12.139632 28968 rpcz_store.cc:276] 0709 13:04:11.105402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.105605 (+   203us) service_pool.cc:224] Handling call
0709 13:04:12.139016 (+1033411us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":281932,"reported_tablets":4}
I20260709 13:04:12.152210 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.200663 29147 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:12.208341 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.201403 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:12.211072 29903 raft_consensus.cc:2955] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:12.213533 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.217554 30587 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:12.218008 30587 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:12.218559 30587 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.218786 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.215788 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.213806 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.217944 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:12.223749 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:12.215691 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.224699 30626 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:12.225839 30610 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:12.226289 30610 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.226498 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.226681 30610 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.227874 30637 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:12.228256 30637 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.166208 30633 raft_consensus.cc:2955] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:12.228607 30637 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:12.235695 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:12.238716 29243 debug-util.cc:398] Leaking SignalData structure 0x7b080019cac0 after lost signal to thread 29097
W20260709 13:04:12.241010 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08000cd6a0 after lost signal to thread 29098
I20260709 13:04:12.244136 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:12.242072 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad860 after lost signal to thread 29172
I20260709 13:04:12.249341 30634 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.251008 30613 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:12.252020 30635 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:12.252518 30599 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.253752 28955 catalog_manager.cc:5028] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): tablet d0107c7a468e44f6b9ebd1ced2ae1ed1 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:12.259166 30635 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.260272 30619 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:12.254684 30606 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:12.261430 30635 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:12.261794 30634 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:12.262256 30613 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.257745 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.265588 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.265746 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.266741 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.257836 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.261495 30606 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:04:12.255219 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08000f93a0 after lost signal to thread 29173
I20260709 13:04:12.275254 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.268113 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.256300 30607 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:12.280303 30612 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:12.281001 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.281466 30565 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:12.281999 30565 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.282120 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.282442 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.282588 30599 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.283134 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.284380 30598 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:12.284868 30598 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.285470 30598 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:12.242002 30641 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:12.284873 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.284937 30599 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:12.278211 30606 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.297991 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.292114 30641 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:12.300627 30641 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.282495 30565 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.306293 30565 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:12.307256 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 65553.2us
I20260709 13:04:12.308604 29874 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:12.337466 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.340344 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.342321 30565 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:12.342787 30565 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:12.343243 30565 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:12.343941 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.344791 29904 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:12.307721 30613 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:12.354676 30617 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.346068 30637 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.360792 30637 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:12.361977 30606 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.362978 30617 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:12.364275 30606 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:12.370673 30641 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.370985 30565 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.374167 30565 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:12.356591 30610 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.376238 30641 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:12.376999 30610 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:12.377952 30587 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.380018 30587 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:12.383406 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.337s	user 0.009s	sys 0.029s
W20260709 13:04:12.383900 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.338s	user 0.009s	sys 0.029s
I20260709 13:04:12.385339 30598 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.399359 30598 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:12.385342 30640 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:12.400741 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08001877a0 after lost signal to thread 29246
I20260709 13:04:12.403674 30640 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:12.418977 30621 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:12.346068 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.499414 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.502319 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.502575 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.504388 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:12.505569 29871 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.506s (SENT) [suppressed 1 similar messages]
I20260709 13:04:12.505999 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.507380 29068 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:12.510051 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.512058 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.512863 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.517014 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.517674 29069 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:04:12.520042 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.521175 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:12.523352 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 236555us
I20260709 13:04:12.524968 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.525506 29908 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:12.526316 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.522562 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.529814 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.527676 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.532181 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.559329 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.560032 29146 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:04:12.560984 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.537384 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.531545 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.565289 29146 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:12.569694 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.570902 29073 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:12.573256 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.574592 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.575846 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:12.577543 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.578176 29065 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:12.578984 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.580006 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.581017 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.582015 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.582662 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.583451 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.584087 30621 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:12.584194 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.584715 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.584663 30621 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.585381 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.585968 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.586540 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.586933 30621 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:12.587095 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.587687 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.588066 30612 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:12.588333 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.588649 30612 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.588894 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.589905 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.590185 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.590610 30612 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:12.591291 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.592691 29065 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:12.595574 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.598765 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.599231 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.599485 29909 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:12.600750 29073 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:12.601063 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.602811 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.603941 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.604213 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.604925 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.598212 30621 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:12.605893 30621 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.607477 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.611120 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.612397 29148 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:12.611358 29907 raft_consensus.cc:2955] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:12.601136 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.613307 29909 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:04:12.615583 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.617024 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.617436 30628 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:12.617810 30628 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:12.618175 30628 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.618427 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:12.619858 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:12.621225 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:12.621881 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 313479us
I20260709 13:04:12.625942 30628 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.628264 30628 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:12.632293 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.632611 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.636224 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.637763 29144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:04:12.637934 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.638511 29144 raft_consensus.cc:606] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:12.640937 30633 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:12.642122 30619 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:12.642495 30619 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:12.642875 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.643479 30619 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.644891 30619 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.646131 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.646900 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.648718 30632 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:12.649137 30632 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:12.649500 30632 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:12.650843 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.646922 30641 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:12.656912 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.672194 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.672689 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.675513 30621 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:12.678675 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.684327 30632 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.687237 30601 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:12.688511 30601 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:12.688875 30601 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:12.689785 30601 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.690531 30628 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:12.691022 30601 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.593526 29069 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:12.694937 30632 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:12.696640 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.691766 30621 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:12.701650 30621 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:12.702286 30638 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:12.702666 30621 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.703071 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:12.703349 30638 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:12.703747 29069 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.703768 30638 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:12.704948 30638 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.703877 30621 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:12.706182 30638 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:12.721218 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.722815 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.723070 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.724249 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.724305 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.724866 30632 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:12.725188 30628 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:12.725651 29186 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.726444 30628 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.726639 29186 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.727355 30628 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:12.720353 29072 raft_consensus.cc:2955] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:12.730747 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:12.731482 29064 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:12.751477 29906 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:12.735584 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:12.770996 29142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.011s	sys 0.049s
W20260709 13:04:12.772315 29142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.012s	sys 0.050s
I20260709 13:04:12.772222 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.775674 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.777524 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.771998 30628 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:12.722330 30619 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:12.779706 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.780422 30632 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.782187 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.782931 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:12.784526 30643 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.785733 30643 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.777555 30607 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:12.786661 30607 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:04:12.786870 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 502988us
I20260709 13:04:12.787078 30607 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.789878 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.791508 30637 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:12.791932 30637 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:12.792490 30637 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.792753 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.794309 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.795753 29064 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:12.797158 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.798245 29069 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:12.798655 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.799417 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:12.800505 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.801256 30623 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:12.801654 30623 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.802002 30623 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:12.802322 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08000c0660 after lost signal to thread 29935
I20260709 13:04:12.807196 30598 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:12.807668 30598 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.808089 30598 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.809496 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.810250 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.811542 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.812317 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:12.813730 30628 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:12.814292 30628 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:12.814831 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.815625 30587 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:12.815237 30628 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.816042 30587 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:12.816522 30587 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.820798 30594 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:12.821236 30594 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.821619 30594 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:12.827564 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 583477us
I20260709 13:04:12.828567 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.837245 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.838901 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.840538 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:12.841995 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.842516 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.843206 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.848625 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.848850 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.868603 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:12.869314 29214 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.873392 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.876109 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:12.876523 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:12.877498 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 49046.8us
I20260709 13:04:12.876296 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.822824 30601 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:12.880041 29222 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.885027 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.886744 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.887064 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:12.894528 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:12.895367 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 113262us
I20260709 13:04:12.895612 29217 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:04:12.869813 30606 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:12.805039 30619 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.882691 30640 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:12.901486 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.906651 29875 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:12.907753 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 130305us
W20260709 13:04:12.908732 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.902258 29905 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:12.899876 30606 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.897192 30601 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:04:12.920046 30650 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:04:12.920572 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.921284 30642 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.903632 30640 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.921373 30565 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.922677 30640 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.911302 30606 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.923138 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:12.909022 30656 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:12.924358 30656 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.924750 30656 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.925256 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:12.922674 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.929132 29222 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:12.926571 30610 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:04:12.930819 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.931274 30610 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:12.931629 30610 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:12.932308 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.932534 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.926235 30654 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:12.933892 30654 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.932561 30610 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.934299 30654 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.934123 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.934953 29222 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:04:12.934088 29218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:12.936374 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.936589 29218 raft_consensus.cc:606] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:12.937182 29214 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:12.908464 30611 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:12.935472 30610 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:12.944077 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:12.949802 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 43582.3us
I20260709 13:04:12.961091 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:12.963045 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:12.944270 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.964798 29222 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
W20260709 13:04:12.965938 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 122840us
I20260709 13:04:12.971937 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:12.972632 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.964864 28968 catalog_manager.cc:5697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:12.973220 29215 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:12.974086 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.915050 30598 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.973439 29111 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.949661 30611 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:12.969231 30641 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.976117 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:12.967773 30565 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.928596 30637 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.969399 30651 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.981206 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:12.986027 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.988734 30643 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:12.983562 30607 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.984588 30657 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:12.998734 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.979238 30655 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:12.995247 30640 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:12.999012 30657 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.000366 30655 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.000562 30657 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.000813 30655 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.000998 30637 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:12.993602 30587 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:12.979493 30611 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.998018 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.003535 30630 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.003883 30630 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.004264 30630 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.991707 30618 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.001209 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:13.009101 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:12.997483 30594 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.011711 29213 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 2: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:12.989935 30623 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.023270 29219 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.008526 30637 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.024271 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.025315 29218 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:13.032999 30606 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.035458 30606 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.039677 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.008291 30601 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.044469 30654 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:13.050714 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 226429us
I20260709 13:04:13.052354 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.052855 30623 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.044533 30611 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:13.060060 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.062474 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:13.065701 29069 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:13.067795 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:13.071692 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.031791 30656 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.072891 29072 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.073824 30656 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.073432 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.074599 29213 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:04:13.076265 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.076311 30611 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.078399 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.078859 30652 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:13.079855 30652 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:13.080181 30652 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:13.081568 30607 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:13.081085 30652 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.082986 30652 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.084977 29871 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.088228 29213 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.090302 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.087756 30641 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.093503 30607 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:13.054177 30619 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:13.095043 30654 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.096107 30598 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.097271 30640 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.098654 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.098701 30630 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.094660 30607 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:13.102979 30657 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.105954 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.106308 30607 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:13.107357 30607 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.108551 30607 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.110206 30587 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.112587 29905 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.114490 30640 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:04:13.123425 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 160484us
I20260709 13:04:13.124305 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:13.129979 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.148965 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.150837 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.156687 30657 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.158062 30594 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:13.159183 30655 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.161666 30655 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.162232 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.163403 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:13.145583 30587 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.116067 30630 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.175443 30651 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.178642 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:13.179332 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:13.180083 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:13.151958 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 130908us
I20260709 13:04:13.172688 29066 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:13.181614 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:13.181927 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:13.182893 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.192387 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:13.192414 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 223973us
I20260709 13:04:13.164855 30642 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:13.195127 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.167466 30643 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.204836 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.206739 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.207398 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.204857 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.208248 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.208369 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.202548 30619 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:04:13.191037 30666 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:04:13.208391 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.212672 30610 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:13.200023 30655 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.225205 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.225927 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.197386 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.221997 29213 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:13.227856 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:13.228876 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.230247 30630 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:13.258838 30630 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:13.259413 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.225008 30610 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:13.232052 30619 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:13.235867 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.237267 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.248040 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.252113 29071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:13.261668 29071 raft_consensus.cc:606] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:13.261890 30666 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:13.260087 30630 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.262295 30666 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.262650 30619 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:13.262974 30630 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.263767 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.263752 30619 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.265079 30619 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.265681 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.266408 29217 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.267141 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.265120 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.268245 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.268618 30610 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.269291 29222 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:13.270080 30610 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.271621 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.272291 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.273834 29220 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:13.253904 29219 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:13.276017 29221 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:13.258117 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.258239 29905 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.262705 30666 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.256474 29218 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:13.256446 29215 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:13.276872 29213 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.279981 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:04:13.278623 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:13.260865 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 63553.8us
I20260709 13:04:13.283386 30652 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.283962 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:13.284221 30658 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:13.285319 30658 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:13.285645 30658 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:13.286573 30658 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.281594 29111 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.289153 30655 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.289991 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.292135 29111 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.290805 30658 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.306938 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.307145 30655 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:13.307523 30655 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.307871 30655 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.307942 30594 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.308692 29220 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.309489 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.310267 30594 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.310392 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.310659 30594 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.311117 30594 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.310909 30666 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.311338 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.312650 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.313410 30598 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.314026 30598 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:13.314380 30598 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.314738 30598 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.311136 30660 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.316597 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.316830 30660 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.316807 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.317457 30660 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.317857 29220 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.317996 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.318712 30667 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:04:13.319454 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.319315 30667 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:13.320101 30657 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:13.320600 30657 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.320950 30657 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.320184 30667 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:13.322239 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.327155 30666 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.328126 30598 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.319316 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.330416 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.331259 30610 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.331661 30610 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.332198 30610 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.332502 30657 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.334327 30654 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.334787 30654 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.335255 30654 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.335754 30655 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.339476 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.342751 29071 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.344166 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:13.344878 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.346402 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.347029 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.345382 29039 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.349121 30633 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.349287 30628 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.334793 30594 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.349779 30663 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:13.351315 30663 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.351672 30663 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:13.342721 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.349731 30658 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:13.362401 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.366801 30663 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.370340 30655 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:13.370826 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:13.371549 30657 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.372802 30594 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.372920 29214 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:13.374020 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.374321 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.376112 29220 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.375381 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.374362 30658 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:13.376927 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.377938 30658 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.380582 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.376166 30660 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.382122 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.382683 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.386695 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.387864 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.388787 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.389539 30652 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.390190 30607 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.390291 30598 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.391613 30654 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.393325 30633 raft_consensus.cc:993] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:13.393808 30633 raft_consensus.cc:1081] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Signalling peer 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:13.393877 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.394977 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:13.397730 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.400199 30610 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.405452 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.406518 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.408995 29215 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.398669 30654 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.414381 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.430018 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.430742 29217 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.411507 29214 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.412225 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.412297 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.431838 29070 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.432013 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.413108 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.396440 30619 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.434908 30667 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.415156 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.451800 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.452378 29069 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.415576 29072 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:13.411934 30663 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.416539 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.417730 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.454718 29143 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:13.455034 29068 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.409391 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.456130 29221 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.418330 29071 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:13.458133 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:13.420081 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.424456 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.425124 30610 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.460887 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.462829 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.465602 30630 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.466585 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.467401 30630 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.426842 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.468331 29213 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.427487 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.469252 29219 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:13.435895 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.437415 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.440268 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:13.440243 29070 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.467837 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.466516 30658 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.442651 29874 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.447100 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.449072 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.472283 30610 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:13.475200 30630 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.475368 30610 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:13.475852 30610 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.448330 28968 catalog_manager.cc:5697] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:13.477949 30658 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.413246 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.479743 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.480036 30643 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:13.480455 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 67290.6us
I20260709 13:04:13.480923 29038 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:13.481037 30611 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.481202 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.481822 30611 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:13.482190 30611 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:13.482369 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.482554 30611 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.482607 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:13.474481 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.486953 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:13.450652 30660 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.490589 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.490862 30643 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:13.491484 30587 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:13.458320 29073 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:13.491843 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.492033 30587 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:13.424113 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.493007 30587 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.494858 30652 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.464422 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:13.495635 30652 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.464286 30661 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:04:13.464598 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.497327 29188 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.425753 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.497749 30661 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:13.498301 30661 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:13.498394 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.498781 30660 raft_consensus.cc:993] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:13.499492 30630 raft_consensus.cc:1081] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:04:13.499591 30611 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.499709 30607 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.499498 30661 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.502004 30643 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:13.502197 30611 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.502547 30661 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.487399 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.488229 30633 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:13.488734 30619 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.488278 30662 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:13.511505 30633 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:13.511624 30662 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:13.512107 30633 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:13.512208 30662 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.512511 30633 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.492772 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.503710 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.514019 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.474916 30667 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:13.489531 30628 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.509835 29215 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:13.516968 29113 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:13.518326 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.516844 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.520196 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.520946 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:13.521485 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.521845 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:13.521857 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.523012 28967 catalog_manager.cc:5697] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:13.526011 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.527418 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.528326 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.528597 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.525684 29038 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:13.529897 28967 catalog_manager.cc:5697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 3 to 4, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 4 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:13.520285 30630 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.524143 30658 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.532709 30666 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.532833 30663 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.533228 30663 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:13.533638 30663 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.533412 30594 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:13.534178 30594 consensus_queue.cc:1243] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:04:13.534695 30594 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:13.535010 30594 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:13.532047 30617 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:04:13.537329 30658 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:13.537683 30658 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:13.538609 30658 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.520388 30660 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.517664 30654 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:13.519634 30611 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.546204 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.547224 30651 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.517273 30641 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.547828 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:13.554009 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.555169 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.560285 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.561355 29215 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:04:13.569062 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52291.2us
I20260709 13:04:13.548087 30611 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.577800 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.565930 30606 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.578523 30617 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.484643 30610 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.582013 29038 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.588721 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.588835 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.589783 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.590615 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.575304 30663 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.592095 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.595695 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.566376 30654 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.580575 30612 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.580127 30640 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:13.611701 30640 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:13.612438 30640 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.595563 30660 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.580324 30658 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.594269 30606 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:13.604538 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.636343 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:13.637632 29932 debug-util.cc:398] Leaking SignalData structure 0x7b08002e4260 after lost signal to thread 27937
I20260709 13:04:13.637773 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.535740 30594 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.503075 30643 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:13.639192 29932 debug-util.cc:398] Leaking SignalData structure 0x7b08002e4940 after lost signal to thread 29005
W20260709 13:04:13.607139 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.636343 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.600214 30598 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.592912 30611 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:13.594873 30617 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:13.661495 30610 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.662237 30610 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:13.662801 30610 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:13.663615 30610 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.666211 30594 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:13.650606 30663 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:13.594573 30637 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:13.594413 30666 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
W20260709 13:04:13.652299 29932 debug-util.cc:398] Leaking SignalData structure 0x7b0800375700 after lost signal to thread 29097
I20260709 13:04:13.685024 29188 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.700438 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.701200 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.599439 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.709775 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.713488 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.714556 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.715046 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.716070 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.716503 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.717262 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:13.717463 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:13.720906 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:13.739527 29144 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:04:13.702951 29932 debug-util.cc:398] Leaking SignalData structure 0x7b0800375720 after lost signal to thread 29098
W20260709 13:04:13.742563 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 143145us
I20260709 13:04:13.744747 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.746510 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.703074 30607 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.739395 30621 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.750181 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:13.703799 30637 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:13.709105 30619 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.741628 30666 consensus_queue.cc:1243] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:04:13.757318 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 166759us
I20260709 13:04:13.701798 30612 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:04:13.758265 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.765261 29038 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:13.743201 30598 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.727581 30640 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.747586 30667 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.752620 30630 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.778613 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.785367 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:13.757247 30619 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:13.792660 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:13.769215 30666 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.769584 30606 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:13.797436 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.774684 30607 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.766166 30611 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:13.775570 30617 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:13.801218 29147 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:13.791018 29932 debug-util.cc:398] Leaking SignalData structure 0x7b0800375f40 after lost signal to thread 29172
W20260709 13:04:13.803768 29932 debug-util.cc:398] Leaking SignalData structure 0x7b0800375f60 after lost signal to thread 29173
I20260709 13:04:13.804010 30611 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:13.805320 29932 debug-util.cc:398] Leaking SignalData structure 0x7b08000adac0 after lost signal to thread 29246
W20260709 13:04:13.806962 29932 debug-util.cc:398] Leaking SignalData structure 0x7b080019cf60 after lost signal to thread 29935
I20260709 13:04:13.774308 30654 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:04:13.795364 30619 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.810374 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.812364 30606 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.814913 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.816272 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.817555 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.818822 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.820107 30598 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:04:13.821278 30677 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:04:13.822623 30657 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:13.823011 30657 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:13.823457 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.824129 30680 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:13.824270 30655 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:13.824699 30680 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:13.825439 30655 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.825605 30680 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.826627 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.827877 30652 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:13.828118 30655 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:13.828729 30655 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:13.829044 30662 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.827951 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.802680 30667 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:13.831633 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:13.834218 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:13.828150 29142 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:42054
I20260709 13:04:13.834875 29142 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:13.835220 29142 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.835875 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:13.837416 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 72235.4us
I20260709 13:04:13.827940 29111 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.841138 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.842538 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.842819 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.844138 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.847007 29071 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.846669 30633 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.848284 29065 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.849233 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.849661 30633 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:13.850757 29065 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.853001 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.844213 28971 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:13.851658 29071 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:13.854429 29908 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:13.864763 29139 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:13.868994 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.869724 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.871207 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.871937 29214 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42970
I20260709 13:04:13.872555 29214 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:13.872932 29214 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:13.873883 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.879439 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:13.881062 29071 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:13.829730 30655 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.884589 29142 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.835711 30617 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:13.890589 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 177178us
I20260709 13:04:13.828390 30652 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:13.902716 29068 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:13.872170 30667 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.913429 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:13.918550 29218 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:13.919032 29218 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:13.921329 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:13.927737 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 51256.6us
I20260709 13:04:13.888765 30654 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.947314 29071 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:13.946053 29214 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:13.950201 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.944815 29218 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:13.951066 29222 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.964419 29214 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:13.881521 30633 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:04:13.967263 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.975652 30683 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.977314 30612 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:14.048763 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:13.969724 30594 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.963447 30628 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.871819 30637 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:14.057785 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 178207us
W20260709 13:04:14.058811 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 108556us
I20260709 13:04:14.059374 30666 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.059648 29141 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:13.999614 30598 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.972450 30667 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:04:14.065790 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:14.070796 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:14.090746 30678 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.091471 30679 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:14.123402 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 58310.8us
W20260709 13:04:14.210098 30661 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.002s
I20260709 13:04:14.215853 30675 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:04:14.106057 30689 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:14.219216 30633 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:14.219661 30633 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:14.220775 30633 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:14.068684 30640 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:14.226251 30633 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.092022 30682 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.149s
I20260709 13:04:14.093184 30687 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.213243 29139 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:04:14.254045 30641 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.005s
I20260709 13:04:14.254549 30641 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:14.264137 30684 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.359s	user 0.006s	sys 0.000s
I20260709 13:04:14.215437 30630 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:14.266883 30630 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:14.267979 30630 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:14.269582 30619 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:14.270836 29220 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:14.278973 29071 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:14.282384 30630 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.283191 29142 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:14.287354 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:14.290010 29908 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.291873 30643 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:04:14.303303 28971 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:14.310948 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:14.333279 29908 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:14.335094 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:14.336596 30654 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:14.337150 30654 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:14.337234 29218 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.338480 30654 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:14.340751 30662 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:14.402388 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.395706 30686 raft_consensus.cc:993] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: : Instructing follower 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:04:14.444654 29215 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:14.465721 30611 raft_consensus.cc:1081] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Signalling peer 730186a8c70f43f2a57d80163bbc37d2 to start an election
W20260709 13:04:14.470240 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 410501us
I20260709 13:04:14.480235 29219 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:14.479779 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:14.450824 30652 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:14.501214 29188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.003s	sys 0.016s
I20260709 13:04:14.470674 30675 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:04:14.479398 30619 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:14.505753 29218 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:14.504446 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:14.507552 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:14.503324 29188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.004s	sys 0.016s
W20260709 13:04:14.509634 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 588371us
I20260709 13:04:14.511660 30655 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.492089 30660 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.509178 30618 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.522861 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:14.534358 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.529389 29872 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.019s	sys 0.064s
I20260709 13:04:14.503572 30657 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:14.560158 29218 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:04:14.555634 29872 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.020s	sys 0.064s
I20260709 13:04:14.566377 30695 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:14.567185 30695 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:14.575686 29218 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:14.581741 30657 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.583580 29066 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:14.586706 30692 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:04:14.596922 30681 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.597666 30681 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:14.565197 30642 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
W20260709 13:04:14.615211 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.663153 29910 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:14.688391 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:14.699642 29216 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:14.684099 29071 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:14.613072 29221 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:04:14.743078 29217 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:14.756206 30698 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:14.607645 30641 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.149s
W20260709 13:04:14.656885 30708 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:04:14.707521 30587 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.704531 30652 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:14.685200 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:14.656972 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:14.600169 30643 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.172s
I20260709 13:04:14.764029 30687 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:04:14.794852 29905 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:14.771612 30661 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:14.776593 30587 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:14.781708 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:14.687096 30695 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:14.800060 30689 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.764256 30675 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:04:14.784111 30685 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:04:14.802335 30701 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:04:14.776926 30621 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:04:14.756896 30698 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:14.809142 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.797665 30661 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:14.822717 30658 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:14.822971 29906 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.048s	sys 0.101s
W20260709 13:04:14.823944 29906 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.048s	sys 0.101s
I20260709 13:04:14.823462 30658 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:14.813603 30618 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:04:14.773295 30683 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:04:14.868907 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:14.869554 29141 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:14.872299 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:14.862339 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:14.873961 30611 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:14.875385 30611 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:14.865149 30587 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:14.887852 29039 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.051s	user 0.057s	sys 0.053s
W20260709 13:04:14.889319 29039 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.057s	sys 0.053s
W20260709 13:04:14.889971 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 1.05408e+06us
I20260709 13:04:14.890851 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:14.913508 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:14.914237 29217 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:04:14.877166 30700 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:14.917119 29221 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:14.916643 30691 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.879628 30642 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:14.917160 29216 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:14.921875 29222 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:14.924157 29215 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:14.922721 28967 catalog_manager.cc:5697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 2 to 3, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:14.924142 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:14.917471 30621 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.935148 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:14.936024 29213 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:14.922396 29066 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:14.924187 30642 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.925802 30692 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:14.925658 30700 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:14.937907 30621 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:14.947376 30676 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:14.935292 30658 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:14.941154 30617 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:14.953416 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:14.951689 30587 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:14.950492 30692 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:14.943334 30642 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:14.919688 30661 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:14.988554 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:15.006803 30700 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:14.951531 30681 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:14.943053 30675 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:04:14.974166 30617 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.009454 30607 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.982805 30587 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.023449 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:15.025121 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.027765 29908 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:14.942850 30678 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:04:15.023626 30617 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.024214 30621 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:15.039563 30642 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.055186 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:15.056591 29908 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 2: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:15.059262 30628 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:15.059669 30628 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:15.065441 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:15.074916 29111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.013s	sys 0.053s
I20260709 13:04:15.011448 30676 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.052246 30695 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.054527 30698 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:15.076910 29111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.013s	sys 0.054s
I20260709 13:04:15.028990 30607 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:15.087651 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 574638us
I20260709 13:04:15.060567 30661 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.070758 30621 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.095629 30657 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:15.096995 30657 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:15.097419 30657 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:15.103155 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.113677 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:15.081247 29221 raft_consensus.cc:2955] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:15.109081 30666 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:04:15.077819 30662 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:15.129534 30662 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.129987 30662 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:15.129916 30630 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:15.130563 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:15.129536 30594 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:15.084332 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:15.132392 30594 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:15.133490 30697 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:15.133924 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:15.098423 30657 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.131075 30630 raft_consensus.cc:3055] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:15.132400 29907 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
W20260709 13:04:15.147567 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.140590 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 97266.6us
W20260709 13:04:15.142752 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 70992.2us
I20260709 13:04:15.150674 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:15.140283 30630 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:15.155020 30657 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.155519 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:15.154309 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:15.138715 30618 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:15.088429 30695 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.154276 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:15.077103 30628 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.155444 30630 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.162343 30628 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.162691 30630 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.126812 30621 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.157539 30678 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:15.194371 29039 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:15.198208 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:15.201326 30695 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:15.206199 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.191757 30658 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:15.212126 30667 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:15.212412 30676 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:15.212599 30667 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:15.213382 30594 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:15.213837 30594 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:15.213548 30667 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.214670 30667 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.214751 30594 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.223507 29214 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:15.218761 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:15.191769 30717 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:15.206744 30679 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:15.203797 30682 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:04:15.217767 30675 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:15.203401 30722 raft_consensus.cc:2955] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:15.249675 30716 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:15.250299 30716 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.223886 30607 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:15.219524 30697 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:15.256525 30618 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:15.234472 30594 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.258361 30700 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.229072 30642 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:15.231611 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:15.238690 30587 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:15.217322 30662 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.269434 30617 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.271526 30618 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:15.326978 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.265964 30697 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.335377 29071 raft_consensus.cc:2368] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3.
I20260709 13:04:15.315587 30676 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:15.321498 30621 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:15.320632 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:15.292372 29907 raft_consensus.cc:2955] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:15.325157 30683 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:15.348757 30703 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:15.353894 30662 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.340706 30685 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:15.346837 30679 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.374795 29142 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:15.376205 29143 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:15.345913 30722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:42954 (request call id 75) took 1025 ms. Trace:
I20260709 13:04:15.378053 29143 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:15.378381 29139 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:15.378082 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:15.360594 30617 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.379280 29140 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.382355 29141 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 3. Ops: []
I20260709 13:04:15.382946 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:15.385681 29143 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:15.386722 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:15.388068 29143 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:15.365068 30686 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:15.347299 30676 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.391088 29141 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:15.365803 30683 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.392803 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.394011 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.394706 29148 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:15.378429 30662 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:15.396350 29143 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:15.390195 30726 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:04:15.398453 29142 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:15.367904 30703 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.400324 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:15.401506 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:15.401716 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:15.402458 29146 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:15.405381 29143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:42044
I20260709 13:04:15.407491 29143 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:15.407886 30716 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:15.405917 30662 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.408088 29143 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.403455 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:15.411439 30662 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.378539 30722 rpcz_store.cc:276] 0709 13:04:14.314634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.445248 (+130614us) service_pool.cc:224] Handling call
0709 13:04:14.482880 (+ 37632us) log.cc:844] Serialized 501 byte log entry
0709 13:04:15.339827 (+856947us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11846,"raft.queue_time_us":12440,"raft.run_cpu_time_us":1289,"raft.run_wall_time_us":1289,"spinlock_wait_cycles":749440,"thread_start_us":36247,"threads_started":1,"wal-append.queue_time_us":89357}
W20260709 13:04:15.416709 30728 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:04:15.417415 30703 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:15.418170 30739 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:15.424204 29140 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
W20260709 13:04:15.422857 30732 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:04:15.422094 28957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 344aa5f73f0f4124b1512832a0bca2c9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:15.425201 29141 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:15.374377 30685 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.395414 30686 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.422201 30716 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:15.439225 29140 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:15.439818 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.440623 29148 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:15.426607 30703 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:15.441843 29141 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:15.442044 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:15.418857 30739 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.446117 29146 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:15.450352 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.451951 30598 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:15.452603 30598 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.452947 30685 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.455029 30598 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:15.378707 30679 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.446373 30703 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.455826 30598 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:15.457685 30679 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.457660 30598 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.440769 30618 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:15.463963 29139 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:04:15.429642 30683 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:15.481415 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.487454 30730 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:04:15.440493 30716 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.493145 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.513800 29145 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:15.499624 30739 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.516973 29220 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:15.518399 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:15.519699 30662 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:15.523068 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.523494 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.524037 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.529042 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.529794 29217 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:15.532050 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:15.532567 29218 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:04:15.498461 30611 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.501682 30686 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:15.508670 30676 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:15.470901 30738 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:15.509626 30598 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.537592 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.532533 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:15.552740 29214 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.513578 29143 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.539750 29219 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:15.542099 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:15.554839 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.555430 29219 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:15.556205 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.520241 30662 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.545950 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:15.556751 29219 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
W20260709 13:04:15.545495 30740 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
W20260709 13:04:15.590916 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 45044.8us
I20260709 13:04:15.542392 30700 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:15.598477 29143 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:15.600549 30680 raft_consensus.cc:2955] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:15.604279 30621 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:15.605273 30722 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.565727 30667 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:15.547024 30640 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:04:15.609457 30666 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:15.605927 30738 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.566524 29147 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:15.552279 29217 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:15.615513 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:15.617573 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 75551.6us
W20260709 13:04:15.623721 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.629814 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.616766 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 82983.9us
I20260709 13:04:15.638294 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:15.660614 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.620337 30673 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:15.668869 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.642731 30666 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:15.679905 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.644063 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:15.655525 30685 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.658787 30693 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:15.635927 30722 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.693195 29218 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:15.661904 30598 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:04:15.690933 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.698546 29214 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:15.641892 30667 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.700325 29144 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:15.701184 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:15.701804 29215 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.661896 30630 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:04:15.679062 30703 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:15.704833 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.691785 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:15.719316 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.721511 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.721853 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:15.722154 29141 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:15.722536 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:15.722781 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.723076 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.724737 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.726886 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.737774 30738 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.745575 30682 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:15.746218 30682 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.746690 30666 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.747813 30666 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.749372 30689 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.749594 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:15.749912 30689 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.751703 30652 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:15.751832 30701 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:15.753165 30652 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:15.753911 30621 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:15.754078 30699 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:15.754343 30652 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:15.754999 30699 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:15.755087 30628 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:15.753279 30607 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:15.691064 30685 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.712157 30745 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.775847 29188 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:15.742148 29905 raft_consensus.cc:2368] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4.
I20260709 13:04:15.696399 30693 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.788664 30598 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.790354 30689 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:15.790613 30607 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:15.755442 30699 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:15.755494 30628 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:15.764772 30717 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:15.824208 30717 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:15.809351 30607 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.825155 30607 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:15.825309 30717 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.826723 30717 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.826757 30628 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.829293 30598 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.830265 30692 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:15.833773 30693 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:15.839816 30699 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.841305 30699 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.842154 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.843192 29219 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:15.733749 30700 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:04:15.772522 30716 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.799100 29146 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:15.864470 30682 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:15.821961 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.868297 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.868944 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.775995 30756 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:15.702628 29907 raft_consensus.cc:2955] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:15.823355 30667 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.867475 30628 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:15.876327 30756 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:15.877430 30756 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.881948 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:15.885984 29906 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.855798 29215 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:15.908396 30662 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.784471 30745 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:15.912763 30745 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:15.909301 30598 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.916508 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.919680 30722 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:15.920957 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.922294 29215 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 3 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:04:15.928588 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:15.929517 29146 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:15.928872 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.961014 29147 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:15.961719 30692 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.962401 30692 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.966089 30692 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:15.968511 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.925212 30756 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.971335 29147 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:15.973572 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.926625 30598 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:15.960605 30667 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:15.976300 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 326521us
W20260709 13:04:15.977137 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 201388us
I20260709 13:04:15.979435 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.980888 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:15.981077 29147 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 1.
I20260709 13:04:15.981484 29145 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:15.982553 30598 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:15.985143 30598 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:15.986327 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.989280 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:15.989945 29215 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:15.997371 29146 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:16.012470 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:16.009696 29039 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:16.025848 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:16.027128 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:16.028563 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:16.030714 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:16.031632 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 336002us
I20260709 13:04:16.039570 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:16.040131 29910 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:16.035507 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:16.038146 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:16.004601 30757 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.045310 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:16.032187 30701 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:16.029073 30679 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:16.031903 30628 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:16.031669 30682 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:16.064366 29142 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:15.988701 30667 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.041986 30661 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:16.069115 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:16.050675 30565 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:16.082844 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:16.063423 30750 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.063450 29139 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:16.071381 30642 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:16.066680 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:16.041807 30751 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:04:16.061260 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:16.079227 30628 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:16.027817 30662 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
W20260709 13:04:16.105654 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.092094 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:16.092317 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:16.084204 30565 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:16.079696 30699 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:16.070142 30661 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:16.050365 30757 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.104331 29143 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 2: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:16.056115 30701 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:16.150851 30757 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:16.090626 30679 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:04:16.083539 30761 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:16.168298 30607 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.171373 30607 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:16.090837 30697 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:16.091768 30722 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:16.142580 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 51457.1us
I20260709 13:04:16.145776 30565 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.089407 30643 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.090080 30674 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:16.060768 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:16.149384 30682 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:04:16.190491 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 129798us
W20260709 13:04:16.181789 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 148837us
I20260709 13:04:16.148649 30661 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:16.158530 30701 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.195093 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:16.199534 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:16.194814 30674 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.186379 29906 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:16.197980 30587 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:16.104856 30750 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:16.199344 30682 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:16.205323 30674 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.195060 30643 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:16.205770 30587 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:16.207553 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:16.346481 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:16.214232 30621 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:16.227267 29910 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:16.253839 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:04:16.350298 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 96554.3us
I20260709 13:04:16.272502 28967 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 269) took 1664 ms. Trace:
I20260709 13:04:16.351501 30598 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:16.352190 30598 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:16.275746 28969 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 2 to 3, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 3 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:04:16.353084 30598 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.278771 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:16.268157 30565 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:16.355379 29148 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:16.279080 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:16.281368 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:16.232695 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:16.284312 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:16.357122 29139 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:16.357995 29145 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.284600 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:16.360832 29903 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:16.284893 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:16.302239 30661 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:16.308641 30674 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:16.304389 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:16.366318 29902 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:16.366544 30674 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:16.366796 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:16.314539 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:16.280112 29144 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:16.211308 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:16.207574 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:16.368687 29143 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:16.213629 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:16.212827 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:16.277068 30643 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:16.216337 30667 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:16.224238 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:16.285331 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:16.294055 30701 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.370272 30716 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:16.373100 30716 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:16.373577 30716 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.373764 30701 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:16.351084 28967 rpcz_store.cc:276] 0709 13:04:14.608305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.609010 (+   705us) service_pool.cc:224] Handling call
0709 13:04:16.272400 (+1663390us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:14.946251 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:15.062025 (+115774us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:15.070040 (+  8015us) write_op.cc:435] Acquired schema lock
0709 13:04:15.075168 (+  5128us) tablet.cc:662] Decoding operations
0709 13:04:15.132519 (+ 57351us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:15.132572 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:15.132599 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:15.132801 (+   202us) tablet.cc:701] Row locks acquired
0709 13:04:15.132821 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:04:15.132978 (+   157us) write_op.cc:270] Start()
0709 13:04:15.133120 (+   142us) write_op.cc:276] Timestamp: P: 1783602255132933 usec, L: 0
0709 13:04:15.133141 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:04:15.241233 (+108092us) log.cc:844] Serialized 541 byte log entry
0709 13:04:15.284111 (+ 42878us) op_driver.cc:464] REPLICATION: finished
0709 13:04:15.429786 (+145675us) write_op.cc:301] APPLY: starting
0709 13:04:15.432112 (+  2326us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:15.437326 (+  5214us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:15.444022 (+  6696us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:15.448616 (+  4594us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:15.448774 (+   158us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:04:15.448817 (+    43us) write_op.cc:312] APPLY: finished
0709 13:04:15.449010 (+   193us) log.cc:844] Serialized 23 byte log entry
0709 13:04:15.461548 (+ 12538us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:15.465840 (+  4292us) write_op.cc:454] Released schema lock
0709 13:04:15.467035 (+  1195us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:15.479527 (+ 12492us) spinlock_profiling.cc:243] Waited 64.1 ms on lock 0x7b5400018954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b08e28 00007f783a543581 00007f783a542d58 00007f783a5479f1 00007f783a5479a9 00007f783a547939 00007f783a547901 00007f783a546b2d 000000000038f354 000000000038f199
0709 13:04:15.507766 (+ 28239us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:15.507818 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":748962,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":140447,"apply.run_cpu_time_us":2112,"apply.run_wall_time_us":61576,"num_ops":1,"prepare.queue_time_us":107,"prepare.run_cpu_time_us":9700,"prepare.run_wall_time_us":538343,"replication_time_us":129517,"spinlock_wait_cycles":147317120,"thread_start_us":140415,"threads_started":1}]]}
W20260709 13:04:16.318338 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.299482 30770 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:16.223795 30686 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.375219 30770 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:16.216226 28971 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 159) took 2427 ms. Trace:
I20260709 13:04:16.375603 30686 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.376308 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:16.371587 29213 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:16.377533 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:16.367938 29220 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:16.378476 30761 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:16.378876 30761 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:16.376185 28971 rpcz_store.cc:276] 0709 13:04:13.789149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.789356 (+   207us) service_pool.cc:224] Handling call
0709 13:04:16.216153 (+2426797us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:14.440790 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:14.441576 (+   786us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:14.441623 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:04:14.441651 (+    28us) tablet.cc:662] Decoding operations
0709 13:04:14.442173 (+   522us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:14.442275 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:14.442316 (+    41us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:04:14.442678 (+   362us) tablet.cc:701] Row locks acquired
0709 13:04:14.442705 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:04:14.442892 (+   187us) write_op.cc:270] Start()
0709 13:04:14.443043 (+   151us) write_op.cc:276] Timestamp: P: 1783602254442839 usec, L: 0
0709 13:04:14.443074 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:04:14.444830 (+  1756us) log.cc:844] Serialized 919 byte log entry
0709 13:04:15.060023 (+615193us) op_driver.cc:464] REPLICATION: finished
0709 13:04:15.311125 (+251102us) write_op.cc:301] APPLY: starting
0709 13:04:15.312685 (+  1560us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:15.322825 (+ 10140us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:15.354364 (+ 31539us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:15.360505 (+  6141us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:15.364928 (+  4423us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:04:15.369134 (+  4206us) write_op.cc:312] APPLY: finished
0709 13:04:15.389200 (+ 20066us) log.cc:844] Serialized 29 byte log entry
0709 13:04:15.397730 (+  8530us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:15.404990 (+  7260us) write_op.cc:454] Released schema lock
0709 13:04:15.412706 (+  7716us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:15.425379 (+ 12673us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:15.425412 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":448674,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":236059,"apply.run_cpu_time_us":3658,"apply.run_wall_time_us":122503,"num_ops":2,"prepare.queue_time_us":132896,"prepare.run_cpu_time_us":3626,"prepare.run_wall_time_us":19958,"replication_time_us":600926,"thread_start_us":270651,"threads_started":2}]]}
I20260709 13:04:16.379251 30761 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:16.286712 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:16.381839 30770 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.388368 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:16.388999 29143 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.389508 30565 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:16.390676 30612 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:16.391127 30612 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.391449 30612 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.392028 30686 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:16.397522 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:16.388729 29112 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:16.401775 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:16.401911 29188 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:16.402539 29148 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:16.407135 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:16.395776 29143 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:16.408390 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:16.412938 29142 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:16.411767 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:16.214205 30771 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:04:16.416939 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 70590us
I20260709 13:04:16.417934 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:16.418411 29145 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:16.247905 30587 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:16.214262 30736 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:16.422565 29188 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:04:16.423403 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 224743us
I20260709 13:04:16.429601 29037 leader_election.cc:400] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4.
I20260709 13:04:16.429913 29037 leader_election.cc:403] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:16.403187 30667 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:16.211313 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:16.267163 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:16.266724 30745 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.426028 30661 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:16.222172 30750 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:16.222393 30685 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:16.425397 29039 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:16.396616 30621 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:16.431200 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:16.432693 29139 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:16.266783 30746 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.414279 30701 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.430022 30676 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:16.436687 30756 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:16.417136 30717 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:16.417271 30722 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:16.424116 30607 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:16.419955 30697 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:16.451977 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 392794us
I20260709 13:04:16.423667 30715 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:16.414255 30652 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:04:16.426673 30565 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:16.207875 30628 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.426815 30751 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:16.408689 30749 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:04:16.501019 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 69903.9us
I20260709 13:04:16.438687 30662 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.420950 30658 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:04:16.426998 30683 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:04:16.239054 30682 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.439947 30736 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:16.520614 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.497334 30728 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:04:16.489374 30699 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:04:16.548568 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 122676us
I20260709 13:04:16.405426 30754 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.101s
I20260709 13:04:16.614259 30756 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:16.505808 30686 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:16.505189 30680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:47946 (request call id 77) took 3060 ms. Trace:
W20260709 13:04:16.564085 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 131878us
I20260709 13:04:16.627805 29874 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:16.484933 30745 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.494592 30716 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:16.547505 30611 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:16.644310 30611 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:16.645648 30716 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:16.645390 30611 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.503636 30701 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:16.572476 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 173638us
W20260709 13:04:16.575944 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 187230us
I20260709 13:04:16.577570 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:16.516537 30746 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.650942 29112 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:16.599674 30715 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.653393 29113 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:04:16.656037 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.553335 30722 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.660331 30715 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.665138 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:16.624490 30771 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:16.668231 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:16.673139 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:16.618619 30749 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:16.624091 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.505952 30717 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.631383 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:16.631798 30651 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:04:16.632361 30779 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:04:16.632467 30641 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:16.635478 30770 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:16.635977 30686 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:16.636303 30612 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.626804 30680 rpcz_store.cc:276] 0709 13:04:13.438086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.438190 (+   104us) service_pool.cc:224] Handling call
0709 13:04:13.475358 (+ 37168us) log.cc:844] Serialized 501 byte log entry
0709 13:04:14.349789 (+874431us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b2c000014c8. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835415a0a 00007f7835415693 00007f7835415631 00007f7836b49b52 00007f7836b49cf5 00007f7836a86691 00007f7836a66eaa 00007f7836a6dd53 00007f7836a6dcd9 00007f7836a6dc69
0709 13:04:16.493926 (+2144137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12989,"raft.queue_time_us":153778,"raft.run_cpu_time_us":4052,"raft.run_wall_time_us":613777,"spinlock_wait_cycles":53282048,"thread_start_us":295834,"threads_started":3,"wal-append.queue_time_us":188617,"wal-append.run_cpu_time_us":1767,"wal-append.run_wall_time_us":1007159}
I20260709 13:04:16.647786 29141 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
W20260709 13:04:16.650956 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 73506.4us
I20260709 13:04:16.666694 30630 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:16.698921 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:16.700161 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:16.623028 30736 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.614830 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:16.658948 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:16.704160 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 89500.4us
I20260709 13:04:16.612715 30565 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:16.680927 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 49619.7us
I20260709 13:04:16.710980 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:16.630759 30699 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.690919 30683 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:16.664000 30685 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:16.741691 29139 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:16.701699 29145 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:16.685392 30717 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:16.697883 30641 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.709836 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:16.710098 29073 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:16.704375 29216 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:16.708907 30667 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.715516 29144 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:16.717111 30751 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:16.717602 29066 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:16.717538 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:16.718633 30746 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:16.770925 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:16.754487 30762 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:16.720160 29071 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:16.720559 30610 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:16.711542 30658 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:16.748871 29185 leader_election.cc:400] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3.
I20260709 13:04:16.784948 29185 leader_election.cc:403] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:16.786087 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 54362.9us
I20260709 13:04:16.747618 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:16.708267 29064 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:16.788311 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:16.758249 30641 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.654129 30775 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:16.756582 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:16.761345 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.760403 29068 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:16.751114 30685 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.765306 30749 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:16.706830 30697 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.706437 30679 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:16.767503 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:16.770386 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:16.770500 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:16.719692 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:16.779333 29216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "082c896dd04e451198eba8392e70fba7"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:16.749526 30683 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.753407 30628 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:16.790269 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:16.792117 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44600.1us
I20260709 13:04:16.793469 30778 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:16.893158 29073 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:16.763404 30751 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:16.894194 30751 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.894335 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:16.895581 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:16.896189 29073 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:16.762622 28955 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 348ae3050d97486fa402a5da2bbdce46 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:16.813532 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:16.693286 30651 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:16.820585 30796 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:04:16.821347 30784 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260709 13:04:16.821836 30782 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:04:16.700977 30756 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.889091 30769 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:16.773999 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:16.890506 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:16.990618 30643 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:16.891009 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.021135 29068 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:16.786777 30658 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:16.805126 30685 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:16.804381 30680 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:16.822587 30774 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:16.787958 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:16.825048 30640 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.970815 30776 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.221s
I20260709 13:04:16.886186 30683 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.805694 30775 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:16.756937 30667 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:16.968674 30779 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:04:16.996178 30750 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.893308 30778 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.962025 30610 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:16.824007 30697 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:17.002553 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.957926 30628 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:16.985895 30617 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:04:16.967979 28955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 208451us
I20260709 13:04:16.721323 30771 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:16.985536 30679 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:04:17.003890 30794 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:04:17.017968 29070 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:16.779579 30700 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:16.826444 30686 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.009439 30701 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.028038 30643 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.035280 30640 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.024814 30795 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:17.130278 30700 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.018652 30722 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.028231 30658 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.023265 30745 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.047883 30776 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:04:17.050705 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:17.036454 30587 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.598s	user 0.006s	sys 0.006s
I20260709 13:04:17.150367 30795 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.123623 30679 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.091624 30799 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3.
I20260709 13:04:17.097136 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.029374 29905 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:17.107265 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1676) took 1045 ms. Trace:
I20260709 13:04:17.170197 30700 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.032945 30775 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.039680 30774 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:04:17.099059 30761 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.719s	user 0.012s	sys 0.000s
I20260709 13:04:17.048831 30612 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.030813 30680 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.023452 30796 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:17.126744 30782 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:04:17.119427 30699 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.046841 30750 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.035002 30769 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.159655 30587 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.163395 30643 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.179181 30700 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.164276 30795 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.164279 30679 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:17.205961 29908 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:17.123775 30789 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:17.170083 29147 rpcz_store.cc:276] 0709 13:04:16.062066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.062865 (+   799us) service_pool.cc:224] Handling call
0709 13:04:16.063131 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.063437 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.063475 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.065996 (+  2521us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.316312 (+250316us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:16.342262 (+ 25950us) log.cc:844] Serialized 14 byte log entry
0709 13:04:16.342547 (+   285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.342648 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:17.107047 (+764399us) raft_consensus.cc:1685] finished
0709 13:04:17.107070 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:17.107206 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":652,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":7265,"raft.queue_time_us":1256,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":69866,"thread_start_us":249939,"threads_started":1,"wal-append.queue_time_us":372452}
I20260709 13:04:17.099761 30617 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.189278 30774 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.160073 30776 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:17.153949 30701 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:04:17.184964 30797 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.213s	user 0.001s	sys 0.001s
I20260709 13:04:17.185798 30621 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.175278 30610 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.188735 30651 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.201541 30751 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.166371 30745 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.151111 30792 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:17.185854 30640 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:17.201295 30750 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.204248 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1388) took 1014 ms. Trace:
I20260709 13:04:17.261164 30769 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.208705 29064 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:17.181123 30791 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.201941 30667 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.181636 30658 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.216463 30666 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.182137 30761 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.235611 29037 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:17.203114 30628 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.230461 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:17.231660 30787 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:17.208830 30700 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.229699 30679 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.259874 29146 rpcz_store.cc:276] 0709 13:04:16.189748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.189879 (+   131us) service_pool.cc:224] Handling call
0709 13:04:16.190166 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.190499 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.190528 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.196195 (+  5667us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.483794 (+287599us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:16.496980 (+ 13186us) log.cc:844] Serialized 14 byte log entry
0709 13:04:16.546361 (+ 49381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.546460 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:17.203969 (+657509us) raft_consensus.cc:1685] finished
0709 13:04:17.204001 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:17.204162 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53750,"prepare.queue_time_us":18613,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":1603,"raft.queue_time_us":59242,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":11184,"thread_start_us":280619,"threads_started":1,"wal-append.queue_time_us":400215}
I20260709 13:04:17.189003 30697 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:17.270350 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 219403us
I20260709 13:04:17.274353 29073 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:17.275434 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.259299 30795 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:17.276146 29070 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:17.266522 30750 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.263697 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:17.279217 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.280851 29070 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:17.216617 30736 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.252774 30701 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:17.283010 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 282033us
I20260709 13:04:17.260221 30745 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.286825 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:17.287237 30745 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.287438 29213 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:17.288700 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:17.267871 30787 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.289569 29215 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:17.289901 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:17.290462 29219 raft_consensus.cc:1813] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Handling vote request from an unknown peer a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:17.291013 29219 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:04:17.291842 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:17.292883 29219 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.289602 29217 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:04:17.294152 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:17.294215 30667 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:17.295689 30667 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:17.296057 30667 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.296396 30667 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.296600 29218 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:17.302912 29219 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:17.303458 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:17.276791 30778 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.268652 30666 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.283839 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.314592 30787 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.315088 30787 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:17.312039 30756 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:17.270777 30679 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.307893 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:17.234167 30789 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.281760 30795 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.299201 30697 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
W20260709 13:04:17.323563 30662 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.703s	user 0.012s	sys 0.000s
I20260709 13:04:17.263195 30792 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.345851 30761 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.343317 30662 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4.
I20260709 13:04:17.323866 30756 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.352483 30792 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:17.354816 29871 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.508s (SENT) [suppressed 24 similar messages]
I20260709 13:04:17.355923 30815 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:04:17.313282 30745 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:04:17.269392 30776 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.361526 30812 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:17.261963 30791 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.343868 30680 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.343437 30587 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.353739 30674 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.357123 30769 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:17.336038 30775 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.366037 30792 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.391223 30804 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:17.384188 30776 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:17.405052 30777 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.834s	user 0.004s	sys 0.010s
I20260709 13:04:17.406814 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1436) took 1549 ms. Trace:
I20260709 13:04:17.389930 30658 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.403345 30565 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.385046 30674 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.395215 30751 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.396363 30745 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:17.407318 29222 rpcz_store.cc:276] 0709 13:04:15.857221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.857373 (+   152us) service_pool.cc:224] Handling call
0709 13:04:15.857889 (+   516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.900466 (+ 42577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:15.900513 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.905671 (+  5158us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.430554 (+524883us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:16.430591 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.430653 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:17.402928 (+972275us) spinlock_profiling.cc:243] Waited 40.3 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:17.406420 (+  3492us) raft_consensus.cc:1685] finished
0709 13:04:17.406448 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:17.406731 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2672,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":932,"spinlock_wait_cycles":92695808,"thread_start_us":469528,"threads_started":1,"wal-append.queue_time_us":585279}
I20260709 13:04:17.392375 30792 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.386554 30769 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.382081 30812 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:17.422890 30807 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:04:17.422086 30701 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.403249 30808 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:17.420053 30745 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:17.427868 30736 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.407866 30750 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:17.467399 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.452453 30701 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.453270 30666 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.452453 30598 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:17.460501 30679 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:17.520372 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.453601 30808 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.460851 30791 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.464787 30795 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:17.665385 30715 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 1.003s	user 0.005s	sys 0.006s
I20260709 13:04:17.552929 30769 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:17.682348 30789 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:17.687054 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 376230us
W20260709 13:04:17.677500 30641 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.850s	user 0.008s	sys 0.004s
I20260709 13:04:17.688902 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:17.692253 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:17.693879 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:17.558027 30792 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.680100 30810 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.652271 30598 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:17.687844 30607 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.081s	user 0.024s	sys 0.036s
I20260709 13:04:17.679510 30795 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:17.652508 30701 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:17.703204 30685 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.663s	user 0.006s	sys 0.007s
I20260709 13:04:17.682862 30674 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.677049 30715 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:17.718605 30607 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.114s	user 0.025s	sys 0.036s
W20260709 13:04:17.727782 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 372980us
W20260709 13:04:17.723369 30683 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.002s
I20260709 13:04:17.671813 30745 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.735275 30821 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:17.714051 30808 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:17.728772 30717 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.960s	user 0.013s	sys 0.000s
I20260709 13:04:17.745652 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:17.749272 29222 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:17.749003 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:17.750407 29218 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:17.746387 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:17.752063 29215 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
W20260709 13:04:17.753664 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59854.2us
I20260709 13:04:17.706302 30641 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.728986 30607 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:04:17.737185 30783 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.170s	user 0.007s	sys 0.012s
I20260709 13:04:17.722159 30795 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.758945 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:17.725646 30810 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.741842 30630 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:17.756342 29902 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:17.762274 29910 raft_consensus.cc:2368] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4.
I20260709 13:04:17.724160 30685 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.735484 30823 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:17.737393 30598 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:17.758302 30826 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:17.756376 30745 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.762497 30812 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:17.764353 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:17.754593 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:17.768409 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:17.736349 30683 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.761850 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:17.754437 30717 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.780628 30823 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.784865 30717 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.785072 30823 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:17.763564 30830 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:17.779217 29222 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:17.779224 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:17.754511 30821 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:17.780140 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 75797.3us
I20260709 13:04:17.781533 30598 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:17.789757 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.786499 29113 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:17.789388 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.789588 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:17.790577 29140 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:04:17.790692 29142 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.790786 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.791610 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.791759 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.792209 29139 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:17.792421 29145 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:17.793241 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.792914 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.793557 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:17.794250 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.795085 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:17.789935 30788 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:17.797302 30598 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.797856 30598 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.798024 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.799153 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.799269 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.799501 30700 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.799748 29070 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:17.800522 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.800745 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.801172 29066 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:17.788407 30607 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.802158 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.803308 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.786983 30830 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.804467 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:17.804677 30830 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.805445 30617 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:17.805969 29067 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:17.823441 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.823900 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.824556 29145 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:17.825579 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.826961 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.828181 29144 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:17.774176 30667 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.837877 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.792625 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:17.840380 29146 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
W20260709 13:04:17.840667 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 48098.6us
I20260709 13:04:17.845722 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:17.846338 29140 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:17.832615 30697 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:17.858114 30795 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:17.859215 29145 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:17.859750 30697 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.863996 30697 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.865360 30701 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:17.845222 30792 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:17.867887 29147 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:17.845973 30823 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:17.874418 29142 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:17.845044 30824 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:17.898831 30641 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.920994 30715 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.902616 30824 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:17.938221 30824 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.890410 30749 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.940304 30749 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.943182 30749 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:17.920650 30788 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:17.927651 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 106819us
I20260709 13:04:17.938409 30832 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:17.892339 30792 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.944201 30788 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:17.997668 30832 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:17.922133 30823 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.049726 30792 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.050158 30746 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.051029 30746 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.073585 30667 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.083206 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:18.094715 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:18.040124 30757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.472s	user 0.009s	sys 0.026s
I20260709 13:04:18.097054 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.097723 29068 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.084520 30821 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.090039 30685 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:18.098235 30757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.530s	user 0.009s	sys 0.027s
I20260709 13:04:18.096565 30745 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.099449 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.097777 30775 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.111284 30832 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.109578 30745 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.113819 30745 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.114225 30788 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.114676 30683 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.116142 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.123782 30788 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.123982 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.124779 29221 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:18.126130 29113 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:18.124558 30788 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.130594 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.102210 30810 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:18.128445 29067 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:18.138366 30823 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.139375 30757 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:18.139880 30757 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.144368 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.144734 30757 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.142917 30788 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.145176 29221 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:18.147181 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.148725 29068 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
W20260709 13:04:18.146024 29141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 158) took 1859 ms (client timeout 1934 ms). Trace:
I20260709 13:04:18.149884 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1463) took 1437 ms. Trace:
I20260709 13:04:18.150475 29073 rpcz_store.cc:276] 0709 13:04:16.712237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.896802 (+184565us) service_pool.cc:224] Handling call
0709 13:04:16.897085 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.276890 (+379805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.149557 (+872667us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b540031e7d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:18.149805 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862781184}
W20260709 13:04:18.149642 29141 rpcz_store.cc:269] 0709 13:04:16.286342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.286487 (+   145us) service_pool.cc:224] Handling call
0709 13:04:18.145849 (+1859362us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54002e8bd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:18.145943 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612734208}
I20260709 13:04:18.110044 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.157833 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.161893 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.162076 29141 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:18.163125 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.165112 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.166544 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.167193 29142 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:18.167059 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.168084 29146 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:18.168201 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.169384 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.169952 29144 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:18.170463 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:18.171532 29071 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1459) took 1460 ms (client timeout 1897 ms). Trace:
W20260709 13:04:18.171986 29071 rpcz_store.cc:269] 0709 13:04:16.711303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.157601 (+1446298us) service_pool.cc:224] Handling call
0709 13:04:18.171470 (+ 13869us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:18.172848 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.173385 29221 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:18.174353 29143 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:18.176710 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:18.156627 30745 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:18.179536 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:18.180936 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.146083 30830 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.181568 29144 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:18.181883 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:18.164213 30682 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:18.182670 30792 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:18.176177 30756 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:18.176688 30757 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:18.175107 30810 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:18.186576 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:18.182964 30745 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.187858 30832 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:18.188658 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.200734 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.201094 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:18.201328 29071 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:18.178705 30821 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:18.189929 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:18.186955 30607 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:18.177302 30823 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:18.195660 29140 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:18.180614 30838 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.197700 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.174979 29069 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:18.203356 29214 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:18.202457 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:18.204288 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:18.195371 30788 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:18.202956 30745 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.213109 30838 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.217720 30838 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.193553 30792 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:18.198427 30598 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.219300 29139 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:18.220490 30598 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:18.220834 29910 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 341) took 1556 ms (client timeout 1912 ms). Trace:
W20260709 13:04:18.221263 29910 rpcz_store.cc:269] 0709 13:04:16.664770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.664881 (+   111us) service_pool.cc:224] Handling call
0709 13:04:18.220621 (+1555740us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003fdc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:18.220752 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2521154432}
I20260709 13:04:18.222191 30745 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.208496 29222 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.198652 30780 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:18.205379 30832 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.208046 30641 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:18.203948 30701 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:18.210387 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 357) took 1461 ms. Trace:
I20260709 13:04:18.208218 30823 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.211001 30808 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:18.207108 30821 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.214031 30788 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:18.267347 30824 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.218411 30792 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:18.269173 30821 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.269553 30824 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.267366 30808 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:18.270293 30808 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.208666 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:18.198467 30682 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:18.272953 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64368us
I20260709 13:04:18.272751 30697 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.274328 30830 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.274442 30682 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.204375 30756 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:18.266448 30598 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:18.276372 30697 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.276593 30598 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.278223 30745 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.279172 30598 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.280885 30795 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:18.281316 30795 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.281716 30795 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.206271 30810 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.287006 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.284809 29909 rpcz_store.cc:276] 0709 13:04:16.748983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.749905 (+   922us) service_pool.cc:224] Handling call
0709 13:04:16.771999 (+ 22094us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.022368 (+250369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:17.022403 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:17.022447 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:17.022482 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.210112 (+1187630us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54004e61d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:18.210162 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.210289 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576112256}
I20260709 13:04:18.287966 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.280918 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.293609 29070 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:18.289752 29143 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:18.294034 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.292827 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.294747 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.294898 29145 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 1.
I20260709 13:04:18.289885 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.267699 30788 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.296813 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.306141 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1731) took 1014 ms. Trace:
I20260709 13:04:18.294895 29071 raft_consensus.cc:2368] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3.
I20260709 13:04:18.306525 29220 rpcz_store.cc:276] 0709 13:04:17.291317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.291697 (+   380us) service_pool.cc:224] Handling call
0709 13:04:18.280768 (+989071us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:18.306071 (+ 25303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9819392}
I20260709 13:04:18.300565 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:18.301435 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.303028 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.284448 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.204715 30607 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:18.299968 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:18.308694 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:18.315349 29145 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:18.315773 29112 leader_election.cc:400] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4.
I20260709 13:04:18.302752 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:18.316150 29112 leader_election.cc:403] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:04:18.312264 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.309352 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.205294 30757 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.424503 29911 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:18.468155 29073 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
W20260709 13:04:18.423933 29908 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 334) took 2216 ms (client timeout 1798 ms). Trace:
I20260709 13:04:18.316581 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:18.320984 29143 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:18.487393 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1729) took 1196 ms. Trace:
I20260709 13:04:18.343204 29222 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
W20260709 13:04:18.477119 29908 rpcz_store.cc:269] 0709 13:04:16.207168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.207296 (+   128us) service_pool.cc:224] Handling call
0709 13:04:18.423559 (+2216263us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b54005a4e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:18.423793 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1924432768}
I20260709 13:04:18.288796 30838 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.474248 29146 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
W20260709 13:04:18.392516 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 79534.5us
W20260709 13:04:18.492034 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 172792us
I20260709 13:04:18.548748 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1448) took 1841 ms. Trace:
I20260709 13:04:18.550187 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.304704 30756 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:04:18.511444 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.579507 29073 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:18.315502 30607 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:18.549352 29072 rpcz_store.cc:276] 0709 13:04:16.706859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.707044 (+   185us) service_pool.cc:224] Handling call
0709 13:04:16.760862 (+ 53818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:16.789641 (+ 28779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.792691 (+  3050us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:16.792761 (+    70us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:16.792815 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.548515 (+1755700us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b54001d9054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:18.548562 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.548689 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65615232}
I20260709 13:04:18.493829 29217 rpcz_store.cc:276] 0709 13:04:17.290864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.294165 (+  3301us) service_pool.cc:224] Handling call
0709 13:04:18.475355 (+1181190us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:18.475585 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.475822 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:18.475859 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:18.483700 (+  7841us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:18.483786 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.483888 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.487300 (+  3412us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8383360}
I20260709 13:04:18.578202 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:18.598944 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.585659 29145 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
W20260709 13:04:18.477622 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 161136us
I20260709 13:04:18.311102 30830 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:18.323701 30715 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:18.584566 30736 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.538430 30823 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:18.305425 30641 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.602813 30830 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.609998 30641 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.610891 30697 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.611431 30697 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.600160 30824 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4.
I20260709 13:04:18.328716 29067 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:18.474880 30832 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.305294 30780 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.589874 30621 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:18.590184 30692 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:18.395092 30701 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:18.415680 30788 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.390862 30821 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.474933 30745 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:18.592720 30757 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.406117 30808 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.392230 30792 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.325011 30791 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.270s
I20260709 13:04:18.579398 30838 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:18.487751 30785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.259s	user 0.006s	sys 0.022s
I20260709 13:04:18.550360 30758 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:04:18.603103 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.628715 30715 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:18.606287 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.606585 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.615746 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.311800 30682 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.290s
I20260709 13:04:18.613672 30697 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.611887 30830 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.642743 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.605495 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.650671 29144 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:18.652511 29068 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:04:18.653904 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 95270.6us
W20260709 13:04:18.648149 30785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.508s	user 0.013s	sys 0.024s
I20260709 13:04:18.659845 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:18.615600 30736 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.613425 30795 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.654436 29213 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:18.647737 30832 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.651008 30758 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:18.664099 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.682554 30758 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:18.665102 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60873.7us
I20260709 13:04:18.647658 30745 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:18.684264 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.685001 30745 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.668589 30850 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.659600 30676 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.646562 30821 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.673162 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.693133 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.693302 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.676483 30848 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:18.679482 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.680024 30832 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.682080 30795 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.685851 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.654153 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.686652 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:18.700881 29217 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:18.701311 30745 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:18.694294 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:18.701741 30745 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:18.701860 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.702157 30745 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.702494 29217 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:18.702565 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:18.687171 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.703495 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.703953 29072 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:18.704495 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.705024 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.706450 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.708832 30697 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:18.709036 30854 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:18.709549 30697 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:18.709844 30854 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.687943 29222 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:18.688222 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:18.712530 29143 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.710831 30697 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.689138 29073 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:18.705066 29217 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:18.716761 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.717767 29073 raft_consensus.cc:2393] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:18.718750 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.707715 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.720132 29072 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:18.689339 29145 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:18.689467 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:18.687114 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.721045 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.721657 29146 raft_consensus.cc:2410] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:18.721896 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.722971 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.723310 30854 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.670468 30780 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.691138 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.723891 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.725932 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.726857 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.727541 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.728103 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.728902 29071 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:04:18.729985 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.694072 29220 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:18.671159 30701 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.691192 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:18.667490 30692 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:18.733031 29073 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:18.733603 30692 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:18.733935 30786 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:18.697418 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:18.734252 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.734812 29073 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.734815 29068 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:18.735885 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.734028 30692 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:18.736063 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 44154.8us
I20260709 13:04:18.738812 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:18.742739 30786 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:18.743129 29143 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:18.743218 30786 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.743475 29068 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:18.744489 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:18.724120 30854 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:18.744710 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:18.745055 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.685451 30838 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:18.690511 30846 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:18.747597 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:18.750134 29067 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:18.750514 30697 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:18.749764 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:18.745566 29068 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:04:18.760795 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.743826 30786 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.695523 30676 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.695216 30662 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:18.767525 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 113254us
I20260709 13:04:18.756431 29071 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:18.675179 30756 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.840690 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:18.768951 29215 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:18.858544 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.859081 29068 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:18.769824 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.861284 29220 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:18.861842 29068 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:18.868155 29073 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:18.869889 29071 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 4. Ops: []
W20260709 13:04:18.850351 30860 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.107s	user 0.003s	sys 0.003s
I20260709 13:04:18.690137 30850 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:18.877061 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 174731us
I20260709 13:04:18.760439 30854 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 3 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:18.874575 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.758916 30846 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:18.869242 29220 raft_consensus.cc:2955] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:18.882719 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:18.883028 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:18.848093 30795 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:18.841764 30791 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:18.898792 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:18.912555 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 162833us
I20260709 13:04:18.924041 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:18.848263 29069 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:18.867738 30682 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:18.872539 30697 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:18.931597 29068 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:18.922437 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:18.933490 29217 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.929869 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:18.766378 30745 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.938120 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.920089 29066 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 3. Ops: []
I20260709 13:04:18.949196 29222 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:18.949690 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:18.950459 29215 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:18.871948 30662 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.920779 30845 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.769344 30715 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:18.959108 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 89784.1us
I20260709 13:04:18.935698 30788 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:18.964766 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:18.936975 30676 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.967748 29072 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:18.932173 30808 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:18.935484 30678 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:04:18.969118 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 103409us
I20260709 13:04:18.964718 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:18.955404 30792 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:18.970773 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:18.929723 30795 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.974918 29222 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:18.978216 29215 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:18.935952 30682 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.932274 30791 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:18.985455 29222 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:18.979007 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.986548 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:18.987035 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:18.989221 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.989342 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:18.990067 29215 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:18.989892 29222 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
W20260709 13:04:18.974720 30798 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 2.006s	user 0.012s	sys 0.000s
I20260709 13:04:18.991865 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:18.991865 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:18.994330 30795 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:18.957254 30692 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.024353 30850 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.970302 30678 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:19.062430 30662 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:19.062813 30745 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:19.063241 30678 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:19.064713 30678 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.064888 30859 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:19.065361 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:19.065943 29214 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.067391 30848 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:19.067261 30786 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:19.042371 30736 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:18.979880 30846 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:19.048662 30862 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:04:19.114053 30786 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:19.050081 30832 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:19.050539 30821 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:19.119328 29222 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.040272 30863 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:18.996035 30856 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:19.123106 30856 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:19.124191 30856 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.973278 30854 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:18.981596 30686 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:18.962232 30845 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.062140 30692 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:19.160393 29214 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:19.161701 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:19.034821 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:19.162799 29215 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:19.162921 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:19.107661 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1687) took 2703 ms. Trace:
I20260709 13:04:19.067972 30848 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.069852 30792 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.073431 30607 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:19.069038 30791 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:19.071520 30758 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.062150 30682 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:19.128732 29217 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:19.211272 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:19.211871 29217 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:19.212711 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:19.213300 30607 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:19.138118 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 169344us
W20260709 13:04:19.138541 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 208745us
I20260709 13:04:19.215168 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:19.193207 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50791us
W20260709 13:04:19.157330 30621 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.010s
I20260709 13:04:19.206848 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:19.218294 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:19.219239 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:04:19.204797 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 321846us
W20260709 13:04:19.209584 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.203325 30630 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:19.221799 30630 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:19.222260 30630 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:19.213436 29217 raft_consensus.cc:2368] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3.
I20260709 13:04:19.209597 30792 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.244717 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:19.216300 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:19.143780 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.150286 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:19.209178 30686 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:19.221164 29874 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:19.231232 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1457) took 1490 ms. Trace:
I20260709 13:04:19.231346 30848 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:19.233811 30845 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:19.233691 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:19.247874 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:19.245690 29222 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:19.257738 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:19.248514 29220 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:19.249907 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:19.259223 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:19.259217 29140 raft_consensus.cc:2368] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3.
W20260709 13:04:19.234845 29902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 356) took 2488 ms (client timeout 1839 ms). Trace:
I20260709 13:04:19.236088 30758 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:19.236925 30630 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:19.163489 29148 rpcz_store.cc:276] 0709 13:04:16.404389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.404544 (+   155us) service_pool.cc:224] Handling call
0709 13:04:16.404876 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.502889 (+ 98013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.502921 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.565006 (+ 62085us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.692937 (+127931us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:16.754877 (+ 61940us) log.cc:844] Serialized 14 byte log entry
0709 13:04:16.755099 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.755196 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:18.811145 (+2055949us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:19.107242 (+296097us) raft_consensus.cc:1685] finished
0709 13:04:19.107348 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.107519 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":61635,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":26473,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":11626,"spinlock_wait_cycles":4950912,"thread_start_us":188061,"threads_started":2,"wal-append.queue_time_us":209154}
I20260709 13:04:19.237697 30804 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:19.201216 30868 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.208251 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:19.267597 29214 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.239632 29218 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:19.268186 29216 raft_consensus.cc:606] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:19.268564 29216 raft_consensus.cc:614] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:19.228204 30678 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:19.246249 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 1397) took 1457 ms. Trace:
I20260709 13:04:19.250792 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:19.252300 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:19.253209 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:19.249444 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:19.260823 29064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1464) took 2511 ms (client timeout 1917 ms). Trace:
I20260709 13:04:19.261482 29113 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:19.255173 29219 rpcz_store.cc:276] 0709 13:04:17.740826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.741004 (+   178us) service_pool.cc:224] Handling call
0709 13:04:17.741387 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.767084 (+ 25697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:17.767117 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:17.767155 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:17.767182 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.230988 (+1463806us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b540009f6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:19.231044 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.231173 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58691328}
I20260709 13:04:19.264329 30864 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:19.320020 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 58633.8us
I20260709 13:04:19.320801 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:19.321738 29145 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.320046 29147 rpcz_store.cc:276] 0709 13:04:17.788813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.789546 (+   733us) service_pool.cc:224] Handling call
0709 13:04:19.246059 (+1456513us) spinlock_profiling.cc:243] Waited 77.2 ms on lock 0x7b54003a48d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:19.246197 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177618688}
I20260709 13:04:19.246562 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:19.258311 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:19.323050 29217 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.323882 29215 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:04:19.320752 29064 rpcz_store.cc:269] 0709 13:04:16.743540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.787972 (+ 44432us) service_pool.cc:224] Handling call
0709 13:04:19.254683 (+2466711us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b540012aa54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:19.254797 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964916992}
I20260709 13:04:19.256784 30830 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:19.325449 29113 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:04:19.270305 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 403480us
I20260709 13:04:19.330763 29214 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
W20260709 13:04:19.333170 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 83766.8us
I20260709 13:04:19.337744 29222 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:19.356046 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:19.264462 30821 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:04:19.336694 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 83601.6us
I20260709 13:04:19.262981 30770 raft_consensus.cc:2764] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:04:19.363169 29219 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:19.368499 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.364986 29874 leader_election.cc:400] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3.
I20260709 13:04:19.369473 29874 leader_election.cc:403] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:19.365466 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:19.371060 29038 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:19.265470 30745 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:19.255774 30875 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:19.266206 30876 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:19.267061 30848 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:04:19.359305 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 96708.7us
I20260709 13:04:19.375795 30745 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:19.372704 29217 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:19.267995 30804 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:19.367416 29145 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:19.381327 30866 raft_consensus.cc:2764] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:04:19.336007 30697 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:19.332544 30678 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:19.269277 30812 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.335743 30852 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:19.384143 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:19.261168 29902 rpcz_store.cc:269] 0709 13:04:16.745962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.749833 (+  3871us) service_pool.cc:224] Handling call
0709 13:04:19.234625 (+2484792us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b540061fdd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:19.234776 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2099221120}
I20260709 13:04:19.335355 30867 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:19.370394 30830 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:19.373478 30850 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:19.362651 30859 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:19.400903 30848 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.403425 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:19.370579 30826 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:04:19.408375 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.269758 30863 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.118s
I20260709 13:04:19.384649 29219 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:19.398394 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:19.381532 30821 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:19.381798 30876 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.376026 30789 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:04:19.540481 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 215099us
I20260709 13:04:19.383194 30875 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:19.391491 30864 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:19.392151 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:19.400768 30697 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:19.394672 30630 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:19.404961 30678 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.323740 30868 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:19.556391 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 191530us
W20260709 13:04:19.543664 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 167949us
I20260709 13:04:19.402751 30852 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:19.425684 30787 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:19.561103 30787 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.561561 30787 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.426591 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:19.427536 30812 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:19.548530 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 145213us
I20260709 13:04:19.576690 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:19.549743 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:19.551343 30676 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.384406 30745 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:19.555781 29875 leader_election.cc:400] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3.
I20260709 13:04:19.390897 30866 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.157s
I20260709 13:04:19.539602 30826 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:19.590261 30826 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.560531 30852 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.558211 30864 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:19.565622 30854 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:19.568137 29904 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:19.568728 30821 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:19.549103 30876 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:19.571076 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:19.572198 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1396) took 1282 ms. Trace:
W20260709 13:04:19.547747 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 167387us
I20260709 13:04:19.606737 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:19.516040 30867 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:19.579663 29139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 1407) took 1786 ms (client timeout 2092 ms). Trace:
I20260709 13:04:19.582422 29113 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:19.394892 30686 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.556964 30697 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.552345 30875 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.585721 30884 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:19.582676 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:19.587303 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:19.598537 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:19.558418 30881 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:04:19.607813 29139 rpcz_store.cc:269] 0709 13:04:17.793165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.794026 (+   861us) service_pool.cc:224] Handling call
0709 13:04:19.579416 (+1785390us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b540027ced4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:19.579589 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":975703808}
I20260709 13:04:19.596724 30630 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:19.608309 29113 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:19.592383 30676 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.612375 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:19.625222 29070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 123) took 1323 ms (client timeout 1586 ms). Trace:
W20260709 13:04:19.625799 29070 rpcz_store.cc:269] 0709 13:04:18.287021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.314405 (+ 27384us) service_pool.cc:224] Handling call
0709 13:04:19.570902 (+1256497us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5800005198. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:19.610624 (+ 39722us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352182656}
I20260709 13:04:19.617027 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:19.620553 30881 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.620651 30810 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:19.603240 30868 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:19.631258 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44083.6us
I20260709 13:04:19.631250 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:19.651517 30891 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:19.661870 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 87384.2us
I20260709 13:04:19.605844 30883 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:19.668255 30630 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:19.668540 30893 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:19.613775 30884 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:19.630872 30866 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:19.639447 30863 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:19.634124 30853 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:19.681059 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 98488.2us
W20260709 13:04:19.677205 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 68963.3us
I20260709 13:04:19.669266 30722 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:04:19.664479 30792 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:19.630635 30881 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.665349 30868 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:19.689416 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:19.681388 30686 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:19.674754 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:19.669561 30883 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:19.647588 29904 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:04:19.602857 30830 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.685861 30893 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:19.627718 30886 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:19.685443 30758 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:19.692796 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:19.606624 29066 rpcz_store.cc:276] 0709 13:04:18.289775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.852436 (+562661us) service_pool.cc:224] Handling call
0709 13:04:18.873121 (+ 20685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.957564 (+ 84443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.571947 (+614383us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b54003219d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:19.572104 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44334336}
I20260709 13:04:19.682967 30866 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.647200 30787 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.677676 30662 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:19.675209 30630 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:19.846607 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1403) took 1541 ms. Trace:
W20260709 13:04:19.842701 29218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1463) took 2079 ms (client timeout 1515 ms). Trace:
W20260709 13:04:19.912425 29218 rpcz_store.cc:269] 0709 13:04:17.763609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.763808 (+   199us) service_pool.cc:224] Handling call
0709 13:04:19.842491 (+2078683us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400282dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:19.842603 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3360571776}
I20260709 13:04:19.844027 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1728) took 2553 ms. Trace:
I20260709 13:04:19.847504 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 12) took 3068 ms. Trace:
I20260709 13:04:19.729146 30850 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:19.947474 30826 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.769183 30892 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:19.917879 29213 rpcz_store.cc:276] 0709 13:04:17.290627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.292898 (+  2271us) service_pool.cc:224] Handling call
0709 13:04:18.279892 (+986994us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:18.280271 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.667244 (+386973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.843751 (+1176507us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b5400283cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:19.843947 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":871106176}
W20260709 13:04:19.791152 30880 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:04:19.691680 30676 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:19.822273 30789 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3.
I20260709 13:04:19.846026 30804 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:19.734850 30758 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:19.726609 30864 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:19.726395 30686 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:19.739629 30853 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:19.806953 30770 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:19.798755 30889 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:19.936116 29216 rpcz_store.cc:276] 0709 13:04:16.778755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.779158 (+   403us) service_pool.cc:224] Handling call
0709 13:04:19.847296 (+3068138us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b5400282dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b07853 00007f783bad560a 00007f783505b7d4 00007f783505b762 00007f783505b691 00007f783505b60c 00007f783505a8c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:19.847416 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5480887680}
I20260709 13:04:19.980993 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1400) took 1690 ms. Trace:
I20260709 13:04:19.936355 29072 rpcz_store.cc:276] 0709 13:04:18.305284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.720755 (+415471us) service_pool.cc:224] Handling call
0709 13:04:19.846412 (+1125657us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54001d8b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:19.846551 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":565474688}
I20260709 13:04:19.982999 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.016355 29910 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:04:20.015277 29071 rpcz_store.cc:276] 0709 13:04:18.291008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.858329 (+567321us) service_pool.cc:224] Handling call
0709 13:04:18.868980 (+ 10651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.873485 (+  4505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.949405 (+1075920us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b54002085d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:19.980911 (+ 31506us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23748864}
I20260709 13:04:19.734830 30886 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:20.021190 30886 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.799042 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:20.022908 29904 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:20.024132 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:04:19.812712 30882 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:04:19.985400 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:19.885187 30887 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:19.818308 30888 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:20.034782 30888 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.017598 30770 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:20.036597 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1467) took 2772 ms. Trace:
I20260709 13:04:20.037107 30888 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:19.689747 30868 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:19.897648 30883 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:19.743074 30848 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:19.989004 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 353) took 3239 ms. Trace:
I20260709 13:04:19.918365 30866 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:20.050289 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64992.9us
W20260709 13:04:19.990235 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 300858us
I20260709 13:04:19.827601 30745 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.040616 29911 rpcz_store.cc:276] 0709 13:04:16.744975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.754348 (+  9373us) service_pool.cc:224] Handling call
0709 13:04:18.424224 (+1669876us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.469282 (+ 45058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.954977 (+1485695us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54005a4e50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:19.984118 (+ 29141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3804326656}
I20260709 13:04:19.996413 30678 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:20.003228 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 352850us
I20260709 13:04:19.894145 30852 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:19.987234 30863 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:20.174896 29215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:20.175621 29215 raft_consensus.cc:606] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:20.020547 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.176183 30863 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.177726 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.179646 29910 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:04:20.182601 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:19.842921 30876 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:20.012876 30758 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.034715 30887 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:20.035094 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:19.813920 30867 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.043138 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:20.043336 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:20.235741 29064 raft_consensus.cc:2368] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3.
I20260709 13:04:20.236127 30758 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:20.043676 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:20.236238 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:19.788761 30884 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:19.948758 30850 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:20.056035 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:20.036998 29065 rpcz_store.cc:276] 0709 13:04:17.264477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.264620 (+   143us) service_pool.cc:224] Handling call
0709 13:04:19.254882 (+1990262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.256860 (+  1978us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.256890 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:19.256929 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:19.256956 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.257019 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.036358 (+779339us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b540012aa50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:20.036497 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1652,"raft.queue_time_us":5727,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":4576563072}
I20260709 13:04:20.059302 30868 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:20.061589 29069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 108) took 1779 ms (client timeout 1829 ms). Trace:
I20260709 13:04:19.789893 30893 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:20.063277 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 81347.8us
I20260709 13:04:20.064249 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:20.038003 30770 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:20.111078 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 420803us
I20260709 13:04:20.156106 30812 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:20.138500 30875 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:20.183667 30866 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:20.161146 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:20.161371 30886 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:20.162233 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:20.163570 30697 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.008s
I20260709 13:04:20.164388 30686 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.164661 30787 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.165061 30900 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:20.165818 30716 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.165906 30821 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.166605 30892 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:20.168169 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.168282 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:20.170203 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.168231 30678 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.171664 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.172045 30902 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:20.172322 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:20.173667 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.174660 29066 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:20.063423 30857 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:20.197176 30826 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.196290 30852 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.178821 30883 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:20.228363 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.173729 30881 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.229336 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:20.184742 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.228710 29186 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:20.570111 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.401002 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 165) took 2115 ms. Trace:
I20260709 13:04:20.573325 29144 rpcz_store.cc:276] 0709 13:04:18.285718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.288138 (+  2420us) service_pool.cc:224] Handling call
0709 13:04:18.288422 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.658616 (+370194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.400749 (+1742133us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b5400342454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:20.400939 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832658304}
I20260709 13:04:20.574949 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.512172 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.576918 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 348) took 1255 ms. Trace:
I20260709 13:04:20.577353 29908 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:20.577545 29910 rpcz_store.cc:276] 0709 13:04:19.321948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.184403 (+862455us) service_pool.cc:224] Handling call
0709 13:04:20.576840 (+392437us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.579332 29143 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:20.580020 29145 raft_consensus.cc:2393] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:20.577859 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 270) took 1063 ms. Trace:
I20260709 13:04:20.239701 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:20.581018 29908 rpcz_store.cc:276] 0709 13:04:19.514650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.511807 (+997157us) service_pool.cc:224] Handling call
0709 13:04:20.577805 (+ 65998us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.572573 29073 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:20.400995 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 194) took 1273 ms. Trace:
I20260709 13:04:20.582296 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:20.582422 29219 rpcz_store.cc:276] 0709 13:04:19.127188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.368741 (+241553us) service_pool.cc:224] Handling call
0709 13:04:19.384473 (+ 15732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.540082 (+155609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.400768 (+860686us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b54002832d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:20.400929 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35529088}
I20260709 13:04:20.576180 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:20.583762 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:20.583983 29144 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:20.586392 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 355) took 1190 ms. Trace:
I20260709 13:04:20.586916 29911 rpcz_store.cc:276] 0709 13:04:19.396389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.052253 (+655864us) service_pool.cc:224] Handling call
0709 13:04:20.586337 (+534084us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.589314 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:20.236487 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:20.251706 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1475) took 1319 ms. Trace:
I20260709 13:04:20.586069 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:20.580355 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:20.592079 29221 rpcz_store.cc:276] 0709 13:04:18.932813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.933076 (+   263us) service_pool.cc:224] Handling call
0709 13:04:18.955818 (+ 22742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.051680 (+ 95862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.051716 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:19.051763 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:19.051806 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.251387 (+1199581us) spinlock_profiling.cc:243] Waited 95.6 ms on lock 0x7b54003ccdd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:20.251447 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.251628 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219809024}
I20260709 13:04:20.594956 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.595803 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.596395 29142 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:04:20.597378 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.599435 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.600713 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:20.602771 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 546826us
I20260709 13:04:20.602828 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:20.604171 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.246383 29904 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:20.605727 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 273) took 1057 ms. Trace:
W20260709 13:04:20.484225 30905 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:04:20.606137 29904 rpcz_store.cc:276] 0709 13:04:19.548276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.023731 (+475455us) service_pool.cc:224] Handling call
0709 13:04:20.605660 (+581929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.615442 29072 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:20.249975 30796 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:20.331833 30832 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:20.600178 30857 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:20.426329 30888 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.473542 30849 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.614398 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.638981 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 263) took 1241 ms. Trace:
I20260709 13:04:20.639416 29910 rpcz_store.cc:276] 0709 13:04:19.397655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.614128 (+1216473us) service_pool.cc:224] Handling call
0709 13:04:20.638899 (+ 24771us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.249576 30746 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.236877 30876 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.584935 30850 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:20.616922 29141 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:20.617213 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:20.252859 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.510840 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.659615 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.610874 29142 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:20.599107 30821 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:20.237458 30887 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.610351 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.594383 30902 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.599534 30900 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:20.599151 30892 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.602259 30716 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.609448 30868 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:20.633718 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:20.609401 30812 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:20.614398 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:20.618847 30826 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:20.616605 30787 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.721458 30787 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.652661 29145 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:20.594002 30697 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.724634 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.652696 29144 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:20.634915 30884 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:20.632865 30796 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:20.578297 29069 rpcz_store.cc:269] 0709 13:04:18.282679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.284174 (+  1495us) service_pool.cc:224] Handling call
0709 13:04:20.061396 (+1777222us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54002ad354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:20.061528 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":847366912}
I20260709 13:04:20.660218 29871 leader_election.cc:400] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3.
I20260709 13:04:20.661612 29146 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:20.665146 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 268) took 1266 ms. Trace:
W20260709 13:04:20.677764 29038 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:47974 dropped due to backpressure: service queue is full with 50 items
I20260709 13:04:20.691466 30875 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.692425 30857 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:20.693724 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 272) took 1146 ms. Trace:
I20260709 13:04:20.695807 30821 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.641788 30888 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.633086 30832 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:20.700343 29903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 343) took 1379 ms (client timeout 1564 ms). Trace:
I20260709 13:04:20.645081 30849 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.702548 30892 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.660354 30885 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:20.653471 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.717499 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:20.798808 29064 raft_consensus.cc:2376] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4. Re-sending same reply.
I20260709 13:04:20.725243 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.726066 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:20.726428 29140 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
W20260709 13:04:20.730854 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 502171us
I20260709 13:04:20.799932 29142 raft_consensus.cc:2368] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3.
I20260709 13:04:20.752624 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:20.800026 29144 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.731087 30697 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.753336 29905 rpcz_store.cc:276] 0709 13:04:19.546910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.585374 (+ 38464us) service_pool.cc:224] Handling call
0709 13:04:20.252642 (+667268us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:20.693636 (+440994us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4881664}
I20260709 13:04:20.804524 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:20.804848 30849 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.805671 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 275) took 1256 ms. Trace:
I20260709 13:04:20.806219 29905 rpcz_store.cc:276] 0709 13:04:19.548841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.804172 (+1255331us) service_pool.cc:224] Handling call
0709 13:04:20.805588 (+  1416us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.712805 30812 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:20.807749 30812 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.777587 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:20.778475 29148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.019s	sys 0.014s
I20260709 13:04:20.731364 30796 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:20.789233 29910 rpcz_store.cc:276] 0709 13:04:19.399026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.640425 (+1241399us) service_pool.cc:224] Handling call
0709 13:04:20.665070 (+ 24645us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:20.809315 29148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.020s	sys 0.014s
W20260709 13:04:20.770854 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 113855us
I20260709 13:04:20.793951 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:20.799671 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 212305us
I20260709 13:04:20.810513 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.769349 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.796550 29067 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:20.780277 30870 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:20.826433 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.816991 29146 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:20.768893 30826 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:20.796739 30787 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.788939 29038 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:47974 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 133)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1503)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1426)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1438)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1422)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1430)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1500)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1442)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1504)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1436)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1417)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1425)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1449)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1428)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1427)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1454)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1456)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1432)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1457)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1418)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1450)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1448)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1447)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1421)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1435)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1459)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1499)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1455)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1441)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1440)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1501)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1453)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 138)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 142)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1420)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1434)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1437)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1451)
I20260709 13:04:20.846305 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:20.846930 29147 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:20.827170 30796 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:20.787891 29903 rpcz_store.cc:269] 0709 13:04:19.320499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.324415 (+  3916us) service_pool.cc:224] Handling call
0709 13:04:20.510601 (+1186186us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:20.700267 (+189666us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8808192}
W20260709 13:04:20.833284 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 231508us
I20260709 13:04:20.842278 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:20.842377 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.857756 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:20.858541 29146 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:20.811781 29142 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:20.807780 30900 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.861637 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.862545 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.864171 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:20.843150 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 211365us
I20260709 13:04:20.817937 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:20.866117 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48285.2us
I20260709 13:04:20.867348 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:20.845397 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.872949 30676 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:20.873433 30853 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:20.873955 30678 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.889437 29112 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:20.878263 30846 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:20.880410 29094 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
I20260709 13:04:20.881631 29871 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:20.883359 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:20.884935 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.885242 29147 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:20.897140 29142 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:20.897207 29146 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:20.887132 29144 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:20.855763 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 267) took 1453 ms. Trace:
I20260709 13:04:20.832805 30849 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:04:20.845247 30870 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.898926 29144 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:20.896390 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:20.900738 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:20.887957 30770 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:20.909819 29139 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:20.909245 29146 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:20.902283 29147 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:20.912639 29144 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:20.888281 29907 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 345) took 1567 ms (client timeout 1809 ms). Trace:
I20260709 13:04:20.916441 29147 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:20.919348 29147 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:04:20.921042 30888 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.921865 29072 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:20.923110 30870 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.926512 30832 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.927119 30864 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:20.913065 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 264) took 1515 ms. Trace:
I20260709 13:04:20.927845 29905 rpcz_store.cc:276] 0709 13:04:19.397933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.816665 (+1418732us) service_pool.cc:224] Handling call
0709 13:04:20.910793 (+ 94128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.928956 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.929626 29905 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 1.
I20260709 13:04:20.930685 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.889083 30850 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:20.932350 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:20.933307 29147 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:20.933804 29148 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:20.899372 30885 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:20.935438 29142 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:20.899513 30854 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:20.932056 30826 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:20.899024 30771 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:04:20.916337 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.953833 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 277) took 1039 ms. Trace:
I20260709 13:04:20.954525 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.900120 29904 rpcz_store.cc:276] 0709 13:04:19.398786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.629872 (+1231086us) service_pool.cc:224] Handling call
0709 13:04:20.852148 (+222276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.956614 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.957943 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.959813 29904 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:20.962616 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 278) took 1047 ms. Trace:
I20260709 13:04:20.964630 30881 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.941051 30854 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:20.967502 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:20.916004 29907 rpcz_store.cc:269] 0709 13:04:19.321074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.324424 (+  3350us) service_pool.cc:224] Handling call
0709 13:04:20.730412 (+1405988us) spinlock_profiling.cc:243] Waited 9.65 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:20.887685 (+157273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22200576}
I20260709 13:04:20.932044 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:20.969786 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.972296 30745 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:20.964962 29904 rpcz_store.cc:276] 0709 13:04:19.915215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.957672 (+1042457us) service_pool.cc:224] Handling call
0709 13:04:20.962549 (+  4877us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.937712 30864 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.031759 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:21.032701 30864 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:20.951944 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.033888 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 352) took 1711 ms. Trace:
I20260709 13:04:20.916178 30770 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.037297 30770 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.893992 30678 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.038763 30692 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.039175 30678 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.933120 30788 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.039299 30692 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.040228 30788 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.042287 30788 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.942134 30716 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.961781 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.043325 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 354) took 1680 ms. Trace:
I20260709 13:04:21.043417 30692 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:21.043676 29908 rpcz_store.cc:276] 0709 13:04:19.362734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.961445 (+1598711us) service_pool.cc:224] Handling call
0709 13:04:21.043263 (+ 81818us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:21.044634 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 143973us
I20260709 13:04:21.044819 29908 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:21.045082 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 263899us
I20260709 13:04:21.045373 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 367) took 1708 ms. Trace:
I20260709 13:04:21.045774 29908 rpcz_store.cc:276] 0709 13:04:19.336859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.044352 (+1707493us) service_pool.cc:224] Handling call
0709 13:04:21.044651 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.045167 (+   516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.045327 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.969996 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.974690 30853 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:20.954349 29903 rpcz_store.cc:276] 0709 13:04:19.914868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.882818 (+967950us) service_pool.cc:224] Handling call
0709 13:04:20.953740 (+ 70922us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.946754 30771 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:20.943691 30850 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.955483 29910 raft_consensus.cc:2376] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3. Re-sending same reply.
I20260709 13:04:20.977702 30902 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.980587 30854 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:20.981583 30883 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:20.937577 30893 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:20.983884 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.986799 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:20.993028 30866 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.994750 30886 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:20.998597 30804 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.971606 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.054889 29907 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:21.055399 30796 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:21.056334 29906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 266) took 1657 ms (client timeout 1731 ms). Trace:
W20260709 13:04:21.056807 29906 rpcz_store.cc:269] 0709 13:04:19.398520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.585289 (+186769us) service_pool.cc:224] Handling call
0709 13:04:20.983648 (+1398359us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:21.056267 (+ 72619us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4673920}
I20260709 13:04:21.057389 30796 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:21.058697 29909 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:21.059010 30676 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:21.060026 29909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 349) took 1737 ms (client timeout 1605 ms). Trace:
W20260709 13:04:21.060395 29909 rpcz_store.cc:269] 0709 13:04:19.322172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.367889 (+ 45717us) service_pool.cc:224] Handling call
0709 13:04:20.239505 (+871616us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:21.059835 (+820330us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b54004b50d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:21.059911 (+    76us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091486848}
I20260709 13:04:21.061924 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 261) took 1664 ms. Trace:
I20260709 13:04:21.062278 29909 rpcz_store.cc:276] 0709 13:04:19.397059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.061179 (+1664120us) service_pool.cc:224] Handling call
0709 13:04:21.061462 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.061636 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.061659 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.061701 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:21.061730 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.061782 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.061873 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.064024 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 370) took 1505 ms. Trace:
I20260709 13:04:21.050554 30692 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:04:21.005640 29902 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:21.013094 30812 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.065730 30692 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:21.066133 30692 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:21.067512 30812 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:21.050246 29905 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
W20260709 13:04:21.068646 29902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 353) took 1743 ms (client timeout 1758 ms). Trace:
I20260709 13:04:21.069406 29905 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:21.070647 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 372) took 1492 ms. Trace:
I20260709 13:04:21.049355 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 279) took 1133 ms. Trace:
I20260709 13:04:21.055423 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 274) took 1506 ms. Trace:
I20260709 13:04:21.072108 29910 rpcz_store.cc:276] 0709 13:04:19.915511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.809896 (+894385us) service_pool.cc:224] Handling call
0709 13:04:20.954321 (+144425us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:21.049303 (+ 94982us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4370688}
I20260709 13:04:21.048261 30853 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:21.049294 30922 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:21.073669 30922 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:21.064320 29909 rpcz_store.cc:276] 0709 13:04:19.558625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.063290 (+1504665us) service_pool.cc:224] Handling call
0709 13:04:21.063540 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.063713 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.063742 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.063779 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:21.063812 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.063865 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.063970 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:21.069062 29902 rpcz_store.cc:269] 0709 13:04:19.325211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.561536 (+236325us) service_pool.cc:224] Handling call
0709 13:04:20.034960 (+473424us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8eee 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:21.068473 (+1033513us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54006ffc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:21.068585 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1655090304}
I20260709 13:04:21.117688 29070 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:04:21.118456 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.064703 29903 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:21.058727 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 260) took 1662 ms. Trace:
I20260709 13:04:21.156355 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1502) took 1098 ms. Trace:
I20260709 13:04:21.052608 29911 raft_consensus.cc:1813] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Handling vote request from an unknown peer a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:20.914268 29112 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:21.126471 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 229994us
I20260709 13:04:20.899782 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:20.920173 29144 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:04:21.136548 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 326869us
I20260709 13:04:21.146283 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 127) took 1103 ms. Trace:
I20260709 13:04:21.048995 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:21.025098 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.171109 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1710) took 1001 ms. Trace:
I20260709 13:04:21.172466 29911 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:21.050714 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.173542 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.024231 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:21.018672 29213 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:21.175101 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.175812 30883 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.176345 29221 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:21.177716 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.178169 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:21.178541 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.179312 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.179663 29073 raft_consensus.cc:2368] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3.
I20260709 13:04:21.022383 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.180192 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:21.180395 29213 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:21.180884 29217 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:21.182003 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.182461 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:21.151453 29871 leader_election.cc:336] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.166s (SENT)
W20260709 13:04:21.000115 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.020145 29218 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:21.183903 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.185081 29213 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:04:21.178972 29221 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.185766 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.186379 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.187326 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:21.034446 29904 rpcz_store.cc:276] 0709 13:04:19.322869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.032949 (+1710080us) service_pool.cc:224] Handling call
0709 13:04:21.033340 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.033534 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.033566 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.033619 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:21.033651 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.033724 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.033833 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.071179 29905 rpcz_store.cc:276] 0709 13:04:19.578611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.068713 (+1490102us) service_pool.cc:224] Handling call
0709 13:04:21.069122 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.070254 (+  1132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.070561 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.072496 29907 rpcz_store.cc:276] 0709 13:04:19.548573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.971222 (+1422649us) service_pool.cc:224] Handling call
0709 13:04:21.055374 (+ 84152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.048573 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 368) took 1680 ms. Trace:
I20260709 13:04:21.175086 28969 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 260) took 5492 ms. Trace:
I20260709 13:04:21.189926 29908 rpcz_store.cc:276] 0709 13:04:19.368112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.047254 (+1679142us) service_pool.cc:224] Handling call
0709 13:04:21.047550 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.048258 (+   708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.048287 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.048331 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:21.048366 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.048425 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.048526 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.074698 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 373) took 1469 ms. Trace:
I20260709 13:04:21.049039 30771 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:21.191694 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:20.892459 29037 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:40862 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1422)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1445)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1430)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1500)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1442)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1504)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1446)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1436)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1417)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1425)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1509)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1449)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1433)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1428)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1427)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1431)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1454)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1456)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1432)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1457)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1418)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1450)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1448)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1429)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1447)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1421)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1435)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1459)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1499)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1455)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1441)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1440)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1501)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1453)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1511)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 138)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 142)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1420)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1434)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1437)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1505)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1506)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1508)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1510)
I20260709 13:04:21.192137 29910 rpcz_store.cc:276] 0709 13:04:19.605066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.073568 (+1468502us) service_pool.cc:224] Handling call
0709 13:04:21.074004 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.074252 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.074282 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.074323 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:21.074353 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.074414 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.074603 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.074048 30853 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.067219 30692 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.193657 30771 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.077015 30922 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.197079 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:21.198381 29071 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:21.199723 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.158430 30676 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:04:21.170228 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 121195us
I20260709 13:04:21.021150 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:21.176019 29222 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:20.999305 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.207243 29215 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 1.
I20260709 13:04:21.207465 29218 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.208591 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:21.209486 29215 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.181128 29142 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.212448 29220 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.217526 30883 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:21.182672 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 181) took 1931 ms. Trace:
I20260709 13:04:21.214028 30758 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:21.218313 30758 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:21.218089 29141 rpcz_store.cc:276] 0709 13:04:19.251256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.251358 (+   102us) service_pool.cc:224] Handling call
0709 13:04:19.251614 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.656177 (+1404563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.182046 (+525869us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54003b38d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:21.182421 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3139539712}
I20260709 13:04:21.218847 30758 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.188172 29217 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:21.220304 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.192778 29213 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:21.195482 30692 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.126027 29143 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:21.156921 29070 rpcz_store.cc:276] 0709 13:04:20.058192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.235993 (+177801us) service_pool.cc:224] Handling call
0709 13:04:21.155949 (+919956us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b54002d4454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:21.156223 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1231806848}
I20260709 13:04:21.171605 29139 rpcz_store.cc:276] 0709 13:04:20.169292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.169410 (+   118us) service_pool.cc:224] Handling call
0709 13:04:20.169672 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.910246 (+740574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.170520 (+260274us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b54001bd5d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:21.170897 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1701675776}
I20260709 13:04:21.195420 30853 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.175137 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.207265 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 133) took 1161 ms. Trace:
I20260709 13:04:21.000389 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:21.208458 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:20.995776 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:21.156814 29906 rpcz_store.cc:276] 0709 13:04:19.396774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.057798 (+1661024us) service_pool.cc:224] Handling call
0709 13:04:21.058157 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.058352 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.058386 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.058431 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:21.058471 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.058536 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.058668 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.221113 29217 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:21.227283 29073 rpcz_store.cc:276] 0709 13:04:20.044658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.160235 (+1115577us) service_pool.cc:224] Handling call
0709 13:04:21.205864 (+ 45629us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.195590 30771 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.226323 29140 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:21.230211 29222 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:21.228255 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.236536 29142 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:21.236462 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:21.231084 30676 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:21.244052 29216 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.164433 29066 rpcz_store.cc:276] 0709 13:04:20.042367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.050213 (+  7846us) service_pool.cc:224] Handling call
0709 13:04:20.063995 (+ 13782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.594755 (+530760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.145655 (+550900us) spinlock_profiling.cc:243] Waited 9.9 ms on lock 0x7b54002ad350. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:21.145971 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22767488}
W20260709 13:04:21.244240 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.246546 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1503) took 1187 ms. Trace:
I20260709 13:04:21.250576 29064 rpcz_store.cc:276] 0709 13:04:20.058715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.225842 (+1167127us) service_pool.cc:224] Handling call
0709 13:04:21.246482 (+ 20640us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:21.244998 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 330689us
I20260709 13:04:21.258265 29214 raft_consensus.cc:1813] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Handling vote request from an unknown peer 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:21.259202 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.253444 29069 raft_consensus.cc:2376] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4. Re-sending same reply.
I20260709 13:04:21.259904 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.255149 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1722) took 1021 ms. Trace:
I20260709 13:04:21.261445 29214 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:21.257470 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.263098 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.263808 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:21.265198 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:21.265482 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.260809 29143 rpcz_store.cc:276] 0709 13:04:20.233297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.594719 (+361422us) service_pool.cc:224] Handling call
0709 13:04:21.254905 (+660186us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54003b4054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:21.255080 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1218283904}
I20260709 13:04:21.268160 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.269176 29140 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:21.272004 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 369) took 1859 ms. Trace:
I20260709 13:04:21.272460 29903 rpcz_store.cc:276] 0709 13:04:19.379121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.051493 (+1672372us) service_pool.cc:224] Handling call
0709 13:04:21.064247 (+ 12754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.167273 (+103026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.238182 (+ 70909us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.273574 30676 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.274564 29906 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:21.284782 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.285827 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1500) took 1228 ms. Trace:
I20260709 13:04:21.286231 29065 rpcz_store.cc:276] 0709 13:04:20.057470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.284197 (+1226727us) service_pool.cc:224] Handling call
0709 13:04:21.285730 (+  1533us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.287290 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.288471 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:21.289893 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.290777 29072 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:21.292306 29068 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:21.296452 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.296511 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:21.298280 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:21.299147 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 60872.1us
I20260709 13:04:21.300674 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.302114 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.303035 29065 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:21.291882 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.292425 29219 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
W20260709 13:04:21.305081 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 48408.3us
I20260709 13:04:21.307802 29142 raft_consensus.cc:2393] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:21.309015 29216 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:21.266656 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.311757 29218 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:21.312284 29222 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:21.315376 29066 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:21.308112 29071 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:21.320111 30812 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:21.320313 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.321835 29070 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:21.320817 30812 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:21.325076 29068 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:21.326081 30812 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.326097 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.329187 29069 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:21.329532 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:21.331490 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.316582 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.333261 30893 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:21.333905 30893 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:21.335813 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:21.336665 29139 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:21.336620 29186 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:21.337391 29217 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:21.324646 30746 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.338865 29217 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.338737 29186 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:21.339989 30918 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:21.340397 30918 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:21.340525 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.341099 30918 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.341230 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.341663 29065 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:21.342409 29903 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:21.340320 30929 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:21.344029 29139 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:21.343607 28968 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 2 to 3, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 3 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.338783 30746 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.352926 29143 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.335199 30893 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.352979 29213 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:21.357848 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.359464 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:21.361428 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.362097 29143 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:21.366443 29140 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.364337 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.368008 30898 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:04:21.363636 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.370920 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:21.371203 30898 consensus_queue.cc:1243] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:21.371793 30898 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:21.372148 30929 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:21.372206 30898 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:21.373665 30898 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.374112 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:21.375093 30898 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.363746 29143 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:21.376891 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.365386 29139 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.377300 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.378086 29069 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:21.379320 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.380681 29906 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:04:21.381975 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.370599 29073 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:21.380517 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.388069 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.389189 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:21.390025 29217 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.390116 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.392809 29066 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:21.396696 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.396966 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:21.397441 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.395536 29221 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:21.395673 29220 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:21.402343 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.403059 29219 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:21.403504 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 103151us
I20260709 13:04:21.379331 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.405413 30886 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.408252 29068 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:21.407761 30886 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.409775 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.410526 30886 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:21.410593 29068 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 1.
I20260709 13:04:21.411196 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.411787 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.412469 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.413318 29072 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:21.414372 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.420228 29070 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:21.421180 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.422051 29070 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:21.422976 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:21.424028 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:21.463604 30922 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:21.465981 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:21.466897 30922 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.428408 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.467463 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:21.468101 29065 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:21.468622 30697 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:21.468994 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:21.470270 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:21.471284 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:21.406070 28968 catalog_manager.cc:5697] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 1 to 2, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.429461 30771 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:21.472681 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:21.472755 28968 catalog_manager.cc:5697] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 1 to 2, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.473712 29113 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:21.473991 30883 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:21.475000 29070 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.432394 29069 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:21.482052 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:21.482971 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.483309 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.415447 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:21.433921 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.486840 29215 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:21.440130 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:21.441323 30746 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:21.490295 29218 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.441687 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:21.441996 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:21.442878 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.445421 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.492370 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.492797 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.493335 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:21.494472 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.494668 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.495240 29908 raft_consensus.cc:2376] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3. Re-sending same reply.
I20260709 13:04:21.495612 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:21.496174 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:21.446163 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:21.455986 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:21.459574 29222 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.460012 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.467365 30922 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:21.475729 30883 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:21.476359 30821 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.190858 28969 rpcz_store.cc:276] 0709 13:04:15.682091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.697731 (+ 15640us) service_pool.cc:224] Handling call
0709 13:04:21.174786 (+5477055us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:16.661342 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:16.963060 (+301718us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:16.965686 (+  2626us) write_op.cc:435] Acquired schema lock
0709 13:04:16.965717 (+    31us) tablet.cc:662] Decoding operations
0709 13:04:17.018492 (+ 52775us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:17.020981 (+  2489us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:17.022324 (+  1343us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:17.039845 (+ 17521us) tablet.cc:701] Row locks acquired
0709 13:04:17.044748 (+  4903us) write_op.cc:260] PREPARE: finished
0709 13:04:17.118007 (+ 73259us) write_op.cc:270] Start()
0709 13:04:17.130017 (+ 12010us) write_op.cc:276] Timestamp: P: 1783602257103142 usec, L: 0
0709 13:04:17.138580 (+  8563us) op_driver.cc:348] REPLICATION: starting
0709 13:04:17.139351 (+   771us) log.cc:844] Serialized 541 byte log entry
0709 13:04:18.279099 (+1139748us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f7833a030ac 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b
0709 13:04:19.622798 (+1343699us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b5400018ea0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a77090 00007f7836a846b8 00007f7836a842fa 00007f7836b08965 00007f7836b08e09 00007f783a543581 00007f783a542d58 00007f783a5479f1 00007f783a5479a9 00007f783a547939 00007f783a547901
0709 13:04:19.680055 (+ 57257us) op_driver.cc:464] REPLICATION: finished
0709 13:04:20.184124 (+504069us) write_op.cc:301] APPLY: starting
0709 13:04:20.184212 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:20.184314 (+   102us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:20.184334 (+    20us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:20.184608 (+   274us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:20.184761 (+   153us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:04:20.184802 (+    41us) write_op.cc:312] APPLY: finished
0709 13:04:20.185005 (+   203us) log.cc:844] Serialized 23 byte log entry
0709 13:04:20.226075 (+ 41070us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:20.226168 (+    93us) write_op.cc:454] Released schema lock
0709 13:04:20.226206 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:20.226667 (+   461us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:20.226694 (+    27us) write_op.cc:454] Released schema lock
0709 13:04:20.503595 (+276901us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b5400018954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0a32a 00007f7836a8ff14 00007f7836a8fec3 00007f7836a8fe38 00007f7836a8fdf3 00007f7836a8f0dc 00007f7836ab8e16 00007f7836aa0cbb 00007f7836a8b39d 00007f7836a8ed75
Metrics: {"mutex_wait_us":512069,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":366740,"apply.run_cpu_time_us":13292,"apply.run_wall_time_us":131968,"num_ops":1,"prepare.queue_time_us":271524,"prepare.run_cpu_time_us":52660,"prepare.run_wall_time_us":2974129,"raft.queue_time_us":191305,"raft.run_cpu_time_us":26821,"raft.run_wall_time_us":1554388,"replication_time_us":2508051,"spinlock_wait_cycles":1733785600,"thread_start_us":2189813,"threads_started":5,"wal-append.queue_time_us":1550206,"wal-append.run_cpu_time_us":31453,"wal-append.run_wall_time_us":2261113}]]}
I20260709 13:04:21.476958 30788 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:21.477228 29067 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:21.477768 29220 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:21.307793 29214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:21.473654 30771 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:21.478945 29219 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:21.471426 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:21.488535 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:21.507874 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.493489 29143 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
W20260709 13:04:21.446691 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 48458.9us
I20260709 13:04:21.412581 29068 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:21.508967 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.509492 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1499) took 1452 ms. Trace:
I20260709 13:04:21.500574 29221 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.501157 30922 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.504848 29218 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:21.505290 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63741.1us
I20260709 13:04:21.492110 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.502542 30821 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:21.509833 29068 rpcz_store.cc:276] 0709 13:04:20.057107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.411506 (+1354399us) service_pool.cc:224] Handling call
0709 13:04:21.509416 (+ 97910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.506717 30771 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.507750 29214 raft_consensus.cc:606] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:21.501368 30883 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:04:21.513952 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 146013us
I20260709 13:04:21.514093 29073 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:21.515568 29216 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:21.517005 29139 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:21.503463 28968 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 4 to 5, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 5 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.506472 30788 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:21.519511 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:21.520053 29214 raft_consensus.cc:614] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:21.522972 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.521011 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:21.523833 29215 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:21.511404 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.528136 29071 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:21.516968 30883 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.527084 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:21.514858 30853 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:21.517171 30771 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.515570 30821 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.539331 29215 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:21.528497 30929 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:21.540712 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 142) took 1490 ms. Trace:
I20260709 13:04:21.544617 29072 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:04:21.542570 29064 rpcz_store.cc:276] 0709 13:04:20.047154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.505032 (+1457878us) service_pool.cc:224] Handling call
0709 13:04:21.524755 (+ 19723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.532140 (+  7385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.532168 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.536890 (+  4722us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:21.536930 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.537008 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.537104 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.546427 29218 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.549316 29218 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:21.545159 29072 pending_rounds.cc:85] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 2
I20260709 13:04:21.533732 30853 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:21.541383 28968 catalog_manager.cc:5697] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 2 to 3, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 3 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.550551 29072 pending_rounds.cc:107] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:04:21.561779 29874 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:21.566211 29072 raft_consensus.cc:2889] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:21.575022 29073 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:21.561426 30796 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:21.565018 29903 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:04:21.578985 29216 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:21.581357 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.583043 30692 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:21.583639 30692 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:21.561105 30846 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:04:21.584812 30692 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.544018 30886 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:04:21.588129 29064 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:21.588297 29218 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:21.559554 30853 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.574108 29069 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.593256 29065 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:21.597965 29065 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.602429 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1420) took 1022 ms. Trace:
I20260709 13:04:21.601042 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:21.604653 29064 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.602905 29072 rpcz_store.cc:276] 0709 13:04:20.574556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.515951 (+941395us) service_pool.cc:224] Handling call
0709 13:04:21.537221 (+ 21270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.596602 (+ 59381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.596811 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.609131 29064 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:21.611889 29064 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:21.612479 29908 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:21.554872 30918 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.625200 29222 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:21.596256 29066 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:04:21.626291 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 49561.9us
I20260709 13:04:21.627070 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.632709 29065 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:21.632979 29039 leader_election.cc:400] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3.
I20260709 13:04:21.633461 29039 leader_election.cc:403] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:21.645291 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.647300 30918 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:21.649528 29072 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:21.665112 30888 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:21.629006 30929 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:21.670090 30929 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.671661 30746 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:21.672390 30746 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.673969 30866 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:21.674574 30746 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:21.675462 30886 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:21.676674 30888 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:21.677170 30888 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:21.678292 30888 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.681180 29214 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:21.627744 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 138) took 1581 ms. Trace:
W20260709 13:04:21.683127 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 139062us
I20260709 13:04:21.564738 30788 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.686933 29073 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:21.688898 29217 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:21.689720 29213 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:21.690706 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 98397.5us
I20260709 13:04:21.681020 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.696375 29070 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:21.697053 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1501) took 1639 ms. Trace:
W20260709 13:04:21.681169 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 163481us
I20260709 13:04:21.697467 29070 rpcz_store.cc:276] 0709 13:04:20.057763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.422667 (+1364904us) service_pool.cc:224] Handling call
0709 13:04:21.696996 (+274329us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:21.702725 30937 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:04:21.695715 30888 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:21.730866 30853 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.732102 30870 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3.
I20260709 13:04:21.733026 30944 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:21.733677 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:21.734673 30866 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.735091 30866 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:21.735201 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:21.735489 30866 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.738829 30886 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.739392 30886 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:21.739859 30886 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.740561 29070 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:21.742606 29071 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:21.718921 30918 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:21.747473 29068 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 1. Current term is 3. Ops: []
I20260709 13:04:21.686653 29069 rpcz_store.cc:276] 0709 13:04:20.046050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.506710 (+1460660us) service_pool.cc:224] Handling call
0709 13:04:21.548158 (+ 41448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.595985 (+ 47827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.624764 (+ 28779us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.728408 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:21.756515 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 175234us
I20260709 13:04:21.756961 30821 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.757967 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:21.752077 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 135470us
I20260709 13:04:21.759331 30821 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:21.761871 30947 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:21.772722 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.787392 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 86948.1us
W20260709 13:04:21.784780 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 139307us
I20260709 13:04:21.750578 30918 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.750222 30853 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:21.723117 30771 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:21.796455 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.801450 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:21.794807 30918 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.802980 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:21.783710 30883 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:21.808665 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47291.6us
I20260709 13:04:21.787992 30951 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 13:04:21.814667 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 51137.8us
I20260709 13:04:21.815601 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:21.816561 30948 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:04:21.824935 30942 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:21.826192 30942 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.826658 30942 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:21.826927 30771 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:21.827039 30942 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:21.808943 30758 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.590s	user 0.005s	sys 0.008s
I20260709 13:04:21.828274 30758 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.829092 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:21.829444 30922 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.830662 30758 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:21.831318 29219 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:21.833469 30940 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:21.825287 30883 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:21.835959 30948 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:21.836439 30948 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:21.837473 30948 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.838783 30948 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.843607 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:21.844115 30943 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:21.844623 30943 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:21.845046 30943 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.845933 30886 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.846788 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:21.843238 30866 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.831647 30922 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:21.852962 30886 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:21.853406 30940 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:21.853955 30940 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:21.854369 30940 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.857559 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:21.859192 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:21.868058 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:21.870277 30922 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:21.871284 30941 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:21.876987 30758 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:21.877285 30953 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.877583 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:21.885471 30947 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.885978 30947 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:21.886364 30947 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.889159 30866 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:21.900254 30944 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:21.901135 30918 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.903085 30944 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.877691 30953 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:21.904186 30953 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.904206 29217 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:21.905488 30922 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:21.907925 30918 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:21.895674 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:21.909884 30950 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.910491 30950 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.912143 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:21.912967 30922 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.912889 30950 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:21.913535 30922 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:21.913765 30950 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:21.915469 30950 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.916074 30950 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.918423 30950 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:21.920120 30948 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:21.913383 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:21.925683 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 199815us
W20260709 13:04:21.922996 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 76285.9us
I20260709 13:04:21.928164 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:21.925474 30950 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:21.914002 30922 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.932132 29039 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:21.921399 30948 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:04:21.937146 29112 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:21.937902 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.938529 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:21.949353 29908 raft_consensus.cc:2376] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3. Re-sending same reply.
I20260709 13:04:21.939188 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.939540 30942 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.940788 30788 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:21.940737 30940 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.940692 30676 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:21.941957 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.957659 29911 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:21.942018 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:21.942499 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:21.942657 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:21.937153 30943 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:21.944443 29909 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:21.971707 29902 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:21.957435 30676 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:21.978641 30676 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:21.979866 30676 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.980827 30883 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.960400 30866 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.960210 30821 raft_consensus.cc:476] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Not starting pre-election -- already a leader
I20260709 13:04:21.968887 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:21.952396 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:21.936630 30918 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.968214 29904 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.968884 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:21.969135 29141 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:21.922928 30944 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:22.071724 30918 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:22.130687 30854 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:04:22.135496 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 400352us
I20260709 13:04:22.122459 30886 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:04:22.133188 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.134723 30854 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:22.131047 30950 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:22.121888 30948 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:22.133955 30918 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:22.169864 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 201108us
I20260709 13:04:22.159264 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:22.150183 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.170358 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 57762.9us
W20260709 13:04:22.146948 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 191566us
W20260709 13:04:22.181046 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.149070 30944 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:22.173483 30948 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:22.169308 30940 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:22.170107 30866 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:22.617030 29143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.028s	sys 0.021s
W20260709 13:04:22.617445 29143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.028s	sys 0.021s
I20260709 13:04:22.619071 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:22.150969 30898 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.165975 30956 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:04:22.645641 30898 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:22.646412 30956 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:22.646914 30956 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.159533 30854 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.151141 30953 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.649170 30854 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.650640 30953 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:22.163386 30947 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:22.652892 30947 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:22.206185 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 47062.4us
I20260709 13:04:22.654225 29038 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:22.654296 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:22.655030 30866 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:22.647751 30898 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:22.241688 30967 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:22.211333 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1761) took 1071 ms. Trace:
I20260709 13:04:22.254838 30942 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:22.266506 30852 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:04:22.243495 30965 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:22.658134 30852 consensus_queue.cc:1243] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:04:22.658746 30852 raft_consensus.cc:3055] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:22.658977 30965 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:22.659229 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1434) took 2057 ms. Trace:
I20260709 13:04:22.275133 29904 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:22.659545 30965 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:22.276600 29067 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:22.659703 29068 rpcz_store.cc:276] 0709 13:04:20.601392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.516361 (+914969us) service_pool.cc:224] Handling call
0709 13:04:21.699965 (+183604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.750724 (+ 50759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.567858 (+817134us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x7b540012a554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:22.659045 (+ 91187us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540012a550. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:22.659179 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475723904}
I20260709 13:04:22.270236 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:22.660584 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1437) took 2047 ms. Trace:
I20260709 13:04:22.270507 30971 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:22.309341 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:22.661000 29067 rpcz_store.cc:276] 0709 13:04:20.613088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.524035 (+910947us) service_pool.cc:224] Handling call
0709 13:04:22.275872 (+751837us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b5800005198. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:22.276150 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.660328 (+384178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.660535 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8182144}
I20260709 13:04:22.661983 30971 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:22.662211 30870 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:22.313323 30746 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:22.662392 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:22.662597 30971 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:22.663187 29067 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:22.662886 30746 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:22.302114 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.664563 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:22.665005 30746 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:22.364667 30948 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.665532 30746 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:22.363756 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:22.674024 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 310340us
I20260709 13:04:22.675315 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:22.435330 29073 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.008s	sys 0.011s
W20260709 13:04:22.676080 29073 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.008s	sys 0.011s
I20260709 13:04:22.677358 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:22.439491 29111 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.548s (SENT) [suppressed 57 similar messages]
W20260709 13:04:22.678309 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 238835us
I20260709 13:04:22.460436 30918 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:22.683153 30918 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:22.170565 30922 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:22.685181 30796 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.103s	user 0.009s	sys 0.009s
W20260709 13:04:22.685545 30796 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.009s	sys 0.009s
I20260709 13:04:22.685793 30796 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:22.468421 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:22.687074 29221 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:22.686771 30796 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:22.688166 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:22.689196 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:22.684806 30979 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:22.690194 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:22.469722 29140 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:22.470623 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:22.694288 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:22.695290 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:22.471380 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:22.696187 29222 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:22.696887 29222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:22.697372 29222 raft_consensus.cc:606] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:22.697662 29222 raft_consensus.cc:614] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:22.468683 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:22.699067 29217 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:22.700106 29217 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:22.472507 29216 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:22.481906 30950 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.484267 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:22.479765 30943 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:22.488780 30962 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.491528 30788 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.704967 30962 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.705541 30788 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:22.491966 30963 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:22.707393 30962 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:22.493098 29146 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:22.493503 30951 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:22.508698 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:22.709038 30943 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:22.709395 30943 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:22.704130 30950 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.509461 30964 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:22.512452 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:22.513209 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:22.513792 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1506) took 1654 ms. Trace:
I20260709 13:04:22.513967 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:22.515684 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:22.523007 30758 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:22.724699 30758 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:22.523638 30676 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.543s	user 0.008s	sys 0.004s
I20260709 13:04:22.528896 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:22.531869 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:22.714758 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:04:22.533574 29910 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:04:22.533835 30935 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.929s	user 0.003s	sys 0.009s
I20260709 13:04:22.534919 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:22.734469 29911 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 2: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:22.536648 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:22.538506 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:22.549024 30957 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:22.562050 28968 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 237) took 7020 ms. Trace:
I20260709 13:04:22.580628 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:22.581768 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:22.582326 29141 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:22.737648 29148 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:22.583720 29139 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:22.588610 30771 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:22.601135 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1757) took 1463 ms. Trace:
I20260709 13:04:22.609483 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:22.610363 30846 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.026s	user 0.015s	sys 0.002s
I20260709 13:04:22.169760 30883 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:22.487493 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:22.736533 28968 rpcz_store.cc:276] 0709 13:04:15.589728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.603018 (+ 13290us) service_pool.cc:224] Handling call
0709 13:04:22.561986 (+6958968us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:22.246233 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:22.247008 (+   775us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:22.247056 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:04:22.247075 (+    19us) tablet.cc:662] Decoding operations
0709 13:04:22.248173 (+  1098us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:22.248336 (+   163us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:22.248367 (+    31us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:04:22.249076 (+   709us) tablet.cc:701] Row locks acquired
0709 13:04:22.249126 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:04:22.249322 (+   196us) write_op.cc:270] Start()
0709 13:04:22.249469 (+   147us) write_op.cc:276] Timestamp: P: 1783602262249247 usec, L: 0
0709 13:04:22.249513 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:04:22.250308 (+   795us) log.cc:844] Serialized 2094 byte log entry
0709 13:04:22.269230 (+ 18922us) op_driver.cc:464] REPLICATION: finished
0709 13:04:22.280891 (+ 11661us) write_op.cc:301] APPLY: starting
0709 13:04:22.281016 (+   125us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:22.281194 (+   178us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:22.281397 (+   203us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:22.282335 (+   938us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:22.282684 (+   349us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:04:22.282744 (+    60us) write_op.cc:312] APPLY: finished
0709 13:04:22.283097 (+   353us) log.cc:844] Serialized 47 byte log entry
0709 13:04:22.283303 (+   206us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.283412 (+   109us) write_op.cc:454] Released schema lock
0709 13:04:22.283476 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:22.291615 (+  8139us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.291656 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":576173,"reported_tablets":6,"spinlock_wait_cycles":2304,"child_traces":[["op",{"apply.queue_time_us":11358,"apply.run_cpu_time_us":2964,"apply.run_wall_time_us":3730,"num_ops":5,"prepare.queue_time_us":32929,"prepare.run_cpu_time_us":11002,"prepare.run_wall_time_us":14096,"raft.queue_time_us":7770,"raft.run_cpu_time_us":4495,"raft.run_wall_time_us":23135,"replication_time_us":19552,"thread_start_us":63543,"threads_started":4,"wal-append.queue_time_us":7883}]]}
I20260709 13:04:22.760167 30758 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:22.629467 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:22.634915 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:22.761121 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 85919.2us
I20260709 13:04:22.761459 29070 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:22.189443 30886 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:22.656736 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:22.657693 29219 rpcz_store.cc:276] 0709 13:04:21.140035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.507280 (+367245us) service_pool.cc:224] Handling call
0709 13:04:21.519671 (+ 12391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.831718 (+312047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.211062 (+379344us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400339fd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:22.211255 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716292736}
I20260709 13:04:22.286299 29064 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:22.314265 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:22.661242 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 391060us
W20260709 13:04:22.365461 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.665937 30948 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.666324 29902 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:22.765600 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 154) took 1031 ms. Trace:
I20260709 13:04:22.666991 30967 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.667487 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:22.660595 30965 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:22.663810 30971 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.636525 28971 catalog_manager.cc:5697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 3, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:22.661034 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:22.673022 30956 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:22.710166 29215 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:22.764566 30886 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.720811 29072 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.691166 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:22.720160 29071 rpcz_store.cc:276] 0709 13:04:20.858925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.532409 (+673484us) service_pool.cc:224] Handling call
0709 13:04:21.555555 (+ 23146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.743035 (+187480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.513547 (+770512us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400185954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:22.513715 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429758848}
I20260709 13:04:22.707654 30788 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:22.735743 29218 raft_consensus.cc:2368] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 3. Current term is 4.
I20260709 13:04:22.774683 29143 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:22.738694 29073 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
W20260709 13:04:22.739883 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.732257 30962 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:04:22.741138 29213 rpcz_store.cc:276] 0709 13:04:21.138241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.478115 (+339874us) service_pool.cc:224] Handling call
0709 13:04:21.689495 (+211380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.690113 (+   618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.600822 (+910709us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54000799d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:22.601017 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485409024}
I20260709 13:04:22.686796 30922 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:22.721463 30870 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:22.727596 30918 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:22.780122 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 599466us
I20260709 13:04:22.731607 30950 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:22.742383 30846 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.158s	user 0.015s	sys 0.002s
I20260709 13:04:22.743484 30746 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.748253 29905 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 4 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:04:22.740947 30771 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:22.749850 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:22.750690 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:22.748291 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:22.749189 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:22.765053 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:22.766096 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 109476us
I20260709 13:04:22.659092 30852 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:22.767166 29064 rpcz_store.cc:276] 0709 13:04:21.732663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.285756 (+553093us) service_pool.cc:224] Handling call
0709 13:04:22.286055 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.763517 (+477462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.763708 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:22.710290 30943 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:22.715689 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.809031 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.808920 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:22.771615 29068 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:22.768514 30967 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:22.816931 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:22.772926 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:22.818480 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:22.818837 29219 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 2: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:22.776540 30965 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.819926 30852 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:22.820523 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 366) took 1227 ms. Trace:
I20260709 13:04:22.821213 30852 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.820945 29905 rpcz_store.cc:276] 0709 13:04:21.592625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.599180 (+  6555us) service_pool.cc:224] Handling call
0709 13:04:22.528602 (+929422us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:22.820438 (+291836us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94353152}
I20260709 13:04:22.822180 30676 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:22.823235 30866 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:22.823596 30866 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:22.824518 29068 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:22.824498 30866 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.825729 30866 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.825809 29069 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:22.767580 30964 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:22.827227 29064 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:22.828392 30771 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.829725 30771 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.832209 30870 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:22.833413 30870 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.835794 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:22.837422 29220 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:22.837906 29065 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:22.838963 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1512) took 1165 ms. Trace:
I20260709 13:04:22.839293 29065 rpcz_store.cc:276] 0709 13:04:21.673467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.512198 (+838731us) service_pool.cc:224] Handling call
0709 13:04:22.838782 (+326584us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400383e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:22.838897 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255541120}
I20260709 13:04:22.769214 30898 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:22.840485 29222 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:22.770987 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.726651 30893 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:22.844040 29072 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:22.770848 30821 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:22.787654 30962 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:22.787814 29215 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:22.787567 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:22.858047 29071 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:22.795495 29213 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:22.774536 30886 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.797031 29148 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:22.797510 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:22.799250 29186 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:22.779615 30788 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:22.812619 29145 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:22.813577 30746 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:22.816280 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:22.666118 30978 consensus_queue.cc:1175] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" responder_term: 3 status { last_received { term: 2 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:04:22.773460 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 286046us
I20260709 13:04:22.769634 30883 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:22.848503 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.849033 29068 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:22.849586 30965 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:22.849817 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:22.849484 30893 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.851550 30692 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:22.859951 29214 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:22.858856 30962 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:04:22.885540 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 86366.8us
I20260709 13:04:22.890064 29186 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:22.886252 29219 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:22.897053 30984 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.897996 30984 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:22.901206 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:22.901721 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:22.906793 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:22.873242 30964 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:22.904846 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:22.817809 30967 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:22.903776 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 93124.1us
I20260709 13:04:22.883662 30853 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:22.913759 29910 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:22.905689 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:22.934350 30888 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:22.934823 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:22.935670 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 50733.5us
I20260709 13:04:22.900573 28971 catalog_manager.cc:5697] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.886175 30898 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.885094 30692 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:22.931758 29874 leader_election.cc:400] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 3. Current term is 4.
I20260709 13:04:22.952878 29874 leader_election.cc:403] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:04:22.900676 30950 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:22.950286 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:22.951217 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.887513 30746 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
W20260709 13:04:22.951318 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 61330.1us
W20260709 13:04:22.954479 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.954770 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:22.956576 29186 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:22.910006 30943 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:22.960577 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59453.7us
I20260709 13:04:22.906366 30866 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:22.907941 30965 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:22.963121 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:22.932390 30853 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.946020 30898 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:22.967278 29185 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:22.898674 30987 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
W20260709 13:04:22.973291 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 192546us
I20260709 13:04:22.972230 30866 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.975462 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:22.952507 30922 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:22.978883 30888 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:22.907704 30788 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:22.982523 30893 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:22.963982 30676 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:22.958602 30940 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:22.986459 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:22.970857 29903 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:22.976817 30812 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:22.966838 30979 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:22.960884 30883 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:22.966960 30796 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:22.976768 30853 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.974344 30821 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:22.967397 30846 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.977039 30942 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 3. Current term is 4.
I20260709 13:04:22.988211 30866 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.990408 30950 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.008047 30984 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:23.009143 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 115338us
I20260709 13:04:23.010663 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:22.995894 30676 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:23.021646 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:23.002883 30979 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:22.971413 30964 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.005272 30951 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.022939 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.009128 30812 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:23.022976 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:04:23.029815 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.994192 30929 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:04:23.019070 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:23.032670 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:23.030913 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.038125 30821 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.031811 30922 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:23.040062 30812 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.038630 30788 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:22.997218 30962 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:23.045241 29871 leader_election.cc:336] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.164s (SENT)
I20260709 13:04:23.045709 30993 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:23.045652 30965 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.027499 30979 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:23.024948 30990 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:23.020502 30888 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.065416 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:23.073462 30964 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.074139 30964 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:23.043388 30796 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:04:23.078258 30962 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:23.079178 30991 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.078259 30853 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.046861 30984 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:23.080368 30984 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:23.080757 30984 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.081632 30853 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.082227 29066 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:23.082648 29066 pending_rounds.cc:85] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 1
I20260709 13:04:23.082891 29066 pending_rounds.cc:107] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:23.083139 29066 raft_consensus.cc:2889] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:23.064796 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 114591us
W20260709 13:04:23.087186 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 65618.2us
I20260709 13:04:23.034318 30987 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.043700 30953 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:23.016734 30846 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.108585 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:23.122089 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:23.124069 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 101168us
W20260709 13:04:23.122825 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 48256.9us
I20260709 13:04:23.034066 30951 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.126664 29139 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:23.091638 30788 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:23.090803 30990 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:23.131371 29185 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:23.091936 30746 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:23.109036 30987 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:23.139376 30929 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:23.138722 30676 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.135371 30821 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:04:23.132102 30788 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.163233 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:23.133311 30853 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:23.126277 30852 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:04:23.079769 30991 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.183593 29909 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:23.140620 30947 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:23.163151 30992 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:04:23.169242 30788 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.221932 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:23.128121 30986 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:04:23.168946 30853 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.162799 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:23.162422 30985 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.213492 30993 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.160495 30746 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.162302 30989 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:23.246483 30821 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:23.160717 30971 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:04:23.277487 30999 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.111s
I20260709 13:04:23.279122 30999 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.279506 30999 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:23.281479 30940 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:23.281890 30940 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:23.282820 30940 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.284678 30991 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.253710 30796 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.291584 30796 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.294162 30796 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:23.228008 30883 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.167019 30898 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.232043 30853 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.236240 30866 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.263090 30997 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:04:23.245626 30993 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:23.263871 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08001ec280 after lost signal to thread 27937
W20260709 13:04:23.316382 29094 debug-util.cc:398] Leaking SignalData structure 0x7b0800139ea0 after lost signal to thread 29005
W20260709 13:04:23.235805 30988 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:04:23.267778 30989 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:23.248242 30990 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.271072 30967 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:23.271351 30948 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.503s	user 0.006s	sys 0.002s
W20260709 13:04:23.272119 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.273572 29905 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:04:23.273980 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 53513.9us
W20260709 13:04:23.276791 31000 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:04:23.297066 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 155393us
W20260709 13:04:23.314414 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 78750.9us
I20260709 13:04:23.331341 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:23.319594 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08000b7320 after lost signal to thread 29097
I20260709 13:04:23.317487 30940 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:23.321105 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:23.324031 29219 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:23.323305 29214 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:23.335381 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:23.324493 29216 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:23.327070 30976 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:04:23.335415 29213 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 2: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:04:23.332134 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08000b7340 after lost signal to thread 29098
W20260709 13:04:23.341092 29094 debug-util.cc:398] Leaking SignalData structure 0x7b080017e680 after lost signal to thread 29172
W20260709 13:04:23.341609 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08001a90a0 after lost signal to thread 29173
W20260709 13:04:23.343186 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08003a89e0 after lost signal to thread 29246
W20260709 13:04:23.345147 29094 debug-util.cc:398] Leaking SignalData structure 0x7b0800327f40 after lost signal to thread 29935
W20260709 13:04:23.320876 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.346726 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.368268 29094 debug-util.cc:398] Leaking SignalData structure 0x7b0800308580 after lost signal to thread 29936
W20260709 13:04:23.369978 29094 debug-util.cc:398] Leaking SignalData structure 0x7b080020df20 after lost signal to thread 30667
W20260709 13:04:23.371796 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08003a79a0 after lost signal to thread 30927
W20260709 13:04:23.373901 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08003a7960 after lost signal to thread 30931
W20260709 13:04:23.375001 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08003a7a00 after lost signal to thread 30933
W20260709 13:04:23.375974 29094 debug-util.cc:398] Leaking SignalData structure 0x7b08003a77e0 after lost signal to thread 30934
W20260709 13:04:23.393168 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 61929.7us
W20260709 13:04:23.348465 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 188010us
I20260709 13:04:23.394054 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:23.397770 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 180824us
I20260709 13:04:23.333060 30948 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.398904 29112 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:23.350457 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.322460 30886 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.339514 30812 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.409533 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:23.410566 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.412569 30886 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.412358 30979 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:23.414855 31002 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:04:23.309389 30987 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.392598 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:23.416815 30993 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.417233 30968 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:23.417817 29904 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:23.263882 30985 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.418707 31017 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:04:23.419471 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.343636 30971 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:23.420477 30942 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:23.434106 30883 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.422360 30951 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.424808 30893 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:23.531293 30893 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:23.425592 30821 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:23.430136 30984 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.440356 30999 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.440398 30991 consensus_queue.cc:1059] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:04:23.444759 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:23.446717 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:23.392086 30943 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
W20260709 13:04:23.489223 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.486202 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:23.393424 30771 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:04:23.446547 30898 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.421752 30866 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.552600 29143 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:23.490398 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.432260 30812 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.529863 30852 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:23.535085 30821 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.460409 31017 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:23.433298 30948 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:23.555254 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 108657us
I20260709 13:04:23.538537 30883 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:23.532946 30989 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 13:04:23.567734 30994 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.407s	user 0.013s	sys 0.000s
I20260709 13:04:23.572206 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:23.542290 30987 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.552412 30993 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.584810 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 87439.4us
I20260709 13:04:23.536367 30971 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.565259 30821 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:23.562101 30676 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:23.646581 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 74438.5us
W20260709 13:04:23.638887 30950 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.568s	user 0.010s	sys 0.000s
I20260709 13:04:23.652390 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:23.641033 30922 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.641239 30943 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.585399 30999 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.584995 31010 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:23.651408 31009 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:04:23.668210 31018 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:04:23.668370 30979 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:23.668800 30971 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:23.669611 30812 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:23.646966 29902 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
W20260709 13:04:23.670590 31016 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:04:23.671278 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:23.667608 31009 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:23.671489 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:23.653266 30993 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.674293 30993 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:23.670463 31010 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.676517 31010 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.677170 30948 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.651859 30676 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.657270 30950 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:23.678452 30676 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.679674 31030 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:23.680264 31030 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:23.679968 30788 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.681001 31009 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.682159 30788 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.672456 29907 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:23.653308 31007 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:23.682299 31009 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:23.693782 29910 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:23.661741 30943 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.659845 30922 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.668398 31029 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.657815 31027 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:23.671547 30985 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:23.707679 30929 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260709 13:04:23.708149 30929 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:23.708793 30929 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:23.709277 30929 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.711372 30929 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:23.704406 31029 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:23.713336 30853 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.713441 31029 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.714733 31029 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.717913 30853 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.685657 30967 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:23.713460 30984 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:23.730232 30951 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:23.731055 31019 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:23.731446 31019 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:23.732283 31019 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:23.703634 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 306021us
I20260709 13:04:23.702117 29141 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:23.705436 30943 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:23.705856 30997 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:23.706437 31005 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:23.718169 30846 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:23.694437 31012 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:04:23.723465 30948 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:23.724764 30922 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:23.733458 31019 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:23.735515 30942 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:23.681170 31030 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.742712 30870 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:23.738469 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:23.755051 29874 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:23.782104 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:23.793030 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:23.794771 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 139445us
I20260709 13:04:23.795591 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:23.738127 30942 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:23.795791 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:23.796206 29218 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:23.796918 29213 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:23.799890 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:23.800194 30993 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.800587 30967 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:23.800735 30993 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.801265 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:23.801164 30967 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.801930 29903 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:23.802943 30993 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:23.803232 30967 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:23.803471 30993 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.804749 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:23.805332 29215 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:23.806324 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:23.807102 29220 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:23.807027 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:23.797821 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:23.807744 29219 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:23.807941 29214 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:23.808861 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:23.774794 30940 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.809504 29215 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:23.772944 30997 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.810015 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:23.788758 30943 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.808877 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:23.806021 29037 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:23.803331 29907 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:23.808779 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:23.811743 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:23.811839 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 73442us
I20260709 13:04:23.749732 30922 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:04:23.789055 30846 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.750538 30853 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:23.814357 30979 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.814702 30771 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:23.816007 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:23.819293 30758 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:23.819999 30947 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:23.821005 30997 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:04:23.821281 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.812294 29222 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:23.797834 30984 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:23.702770 31017 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.831590 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:23.792203 30978 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:04:23.869338 30676 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.833961 30929 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.834537 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:23.871163 29070 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:23.820472 30940 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.872500 29071 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:23.873257 30940 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:23.833348 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:23.836663 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:23.836743 31032 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:23.837838 30989 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.838761 30985 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:23.839426 31010 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.852178 29220 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:23.853410 29215 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:23.794327 30812 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:04:23.749178 30870 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:23.804395 30993 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.829975 31034 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:23.868352 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:23.866153 29222 pending_rounds.cc:85] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 1
I20260709 13:04:23.958168 29222 pending_rounds.cc:107] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:23.958441 29222 raft_consensus.cc:2889] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:23.839809 31026 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:23.873412 30854 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:23.971747 30854 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:23.972419 30692 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.972910 30854 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.973680 30692 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:23.793423 30883 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:23.878602 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:23.973824 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:23.974480 29213 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:23.975071 30951 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:23.975466 30951 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:23.975993 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:23.976322 30951 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.977334 30951 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:23.879320 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60463.4us
I20260709 13:04:23.944731 29218 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:23.946033 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:23.997417 29068 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:23.873713 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:23.854465 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:23.948253 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:23.830025 30943 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.865197 30853 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:23.953361 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:23.962584 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:24.011827 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:24.012739 29071 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:23.969709 29214 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:24.013293 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:23.970999 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:24.014204 29220 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:23.997321 29221 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:04:23.947327 30979 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:23.998384 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:23.971144 30846 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:24.002242 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:23.952584 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.020605 29148 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:24.004235 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:24.008239 29143 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:24.020629 29217 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:23.865830 30888 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:04:24.012123 29067 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:23.967756 30746 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:24.012506 30991 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:24.012065 29145 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:24.000053 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 119926us
I20260709 13:04:24.037810 29186 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:24.012554 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 134075us
I20260709 13:04:24.018271 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.029748 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:23.959515 31034 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:24.034471 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.035686 30771 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.043723 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:24.043844 31036 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.203s
I20260709 13:04:24.044245 30771 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.044939 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:24.045315 31036 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:24.045979 31036 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:24.046046 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:24.046423 30771 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.016886 30676 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.055622 29222 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:24.033620 31032 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:24.029776 30963 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:24.062266 29068 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:23.951330 30950 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:04:24.035704 29904 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:04:24.063313 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.060772 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:24.063575 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:24.066741 29070 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.036633 30968 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:24.047554 31034 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:24.075181 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:24.049433 31027 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:24.037195 30991 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.251681 30991 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:24.071743 29186 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:24.056090 31036 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:24.067157 29071 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:24.073056 29217 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:24.003057 30997 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.060001 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:24.070449 31032 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.068843 30963 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:24.147315 31042 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:04:24.077853 30948 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.081732 30922 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:24.121258 30968 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.071482 30821 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:24.232234 31035 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.290s	user 0.005s	sys 0.000s
I20260709 13:04:24.235507 30853 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:24.292400 30853 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.235908 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.236693 30993 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.237277 29066 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:24.118463 30985 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:04:24.237880 30943 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:24.238787 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.308322 29068 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:24.239492 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.287164 31027 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.238050 31019 consensus_meta.cc:287] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.501s	user 0.007s	sys 0.005s
I20260709 13:04:24.118022 31007 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:24.242484 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:24.242892 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:24.243700 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:24.296793 30893 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:24.244616 30942 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.305891 30951 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.078289 30746 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:04:24.248076 30987 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.597s	user 0.000s	sys 0.013s
W20260709 13:04:24.248535 31037 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.170s	user 0.003s	sys 0.003s
I20260709 13:04:24.121920 31029 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:04:24.121626 31009 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
W20260709 13:04:24.252970 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 181309us
I20260709 13:04:24.267279 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.383083 31043 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:24.383653 31043 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:04:24.275121 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.280139 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:24.384500 31043 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:24.385073 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 104994us
I20260709 13:04:24.272794 31036 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:24.385573 31043 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:24.385792 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:24.386121 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.294226 30948 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.277217 30796 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:24.276872 30967 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.279659 31032 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:24.278970 30812 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.279192 30963 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:24.240312 29072 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:04:24.288133 30922 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.309099 29221 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:24.311067 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.313000 29073 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:24.292840 30821 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:24.340296 31024 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.312516 29064 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:24.355451 29069 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:24.343328 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.294051 30999 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:24.067268 29902 raft_consensus.cc:2410] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:24.295430 30853 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.245178 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:24.357219 29067 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:04:24.357846 30942 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.358304 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.363185 30991 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.436929 30991 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.365417 30971 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:24.367139 30854 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:24.369501 30870 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.296252 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.387385 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:24.444373 30870 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:24.444743 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 57439.3us
I20260709 13:04:24.355412 30893 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.342425 30968 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.397207 30692 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:24.451555 30692 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.452046 30692 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:24.453404 29905 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:24.392944 30951 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.394881 30796 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.320358 31027 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.415830 29068 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:24.422533 31030 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:24.423941 30999 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:24.424734 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.423324 31034 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.385829 31036 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:24.448392 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.415660 30922 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:24.405871 29146 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:24.415280 30812 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.460210 31026 raft_consensus.cc:2764] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:04:24.449013 30943 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:24.464670 29065 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:24.432041 29067 pending_rounds.cc:85] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 6
I20260709 13:04:24.505329 29067 pending_rounds.cc:107] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:04:24.505661 29067 raft_consensus.cc:2889] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:24.506302 30893 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:24.466035 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 220926us
I20260709 13:04:24.498672 30968 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.507154 30968 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.507357 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:24.421905 31024 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.509174 30968 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.509845 31024 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:24.509934 31026 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:24.510571 31026 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:24.511531 31034 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:24.447652 30948 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.460320 30947 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.558189 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.558526 29146 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:04:24.504299 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.457773 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:24.472090 30796 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:24.402165 30967 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.513974 30692 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.515033 31009 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:24.515347 29069 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:24.515512 30940 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:24.515808 30991 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:24.517079 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.584335 29144 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.523339 31010 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.525487 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:24.525624 31017 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:24.511432 29141 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:24.472608 31030 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:24.533244 31027 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.589567 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.511411 31026 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:24.549783 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.554266 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.496451 30929 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:24.561019 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:24.570889 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 205627us
I20260709 13:04:24.571765 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.570578 30692 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:24.597499 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.597894 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.586241 30922 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.598963 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:24.599110 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.599753 29902 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:24.600221 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.600656 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:24.601282 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.600221 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:24.588632 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63216.1us
I20260709 13:04:24.602314 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.603041 30922 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:24.602975 29140 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 2: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:24.604424 29911 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:24.588783 30948 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:24.571400 29905 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:24.605312 30948 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.605695 30948 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:24.605733 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.606391 29141 raft_consensus.cc:2376] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6. Re-sending same reply.
I20260709 13:04:24.607180 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.593098 29148 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:24.608633 29908 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:24.595970 29146 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:24.609490 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.612684 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.584164 30940 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:24.564627 30947 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.604763 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.608445 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.635530 29143 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:24.636695 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:24.636744 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.616948 29144 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:24.637351 29068 raft_consensus.cc:2376] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4. Re-sending same reply.
I20260709 13:04:24.640158 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.640774 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.641537 29068 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:24.609170 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:24.609483 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:24.643138 29903 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:24.643308 29067 raft_consensus.cc:2376] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4. Re-sending same reply.
I20260709 13:04:24.644603 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.645892 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.646656 29073 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:24.647742 29070 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:04:24.648236 29070 pending_rounds.cc:85] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 4
I20260709 13:04:24.648509 29070 pending_rounds.cc:107] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:04:24.648772 29070 raft_consensus.cc:2889] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:24.640884 29072 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:24.642469 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.613654 30963 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.652084 30940 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.653117 30940 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:24.654759 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.613961 30929 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.656126 30963 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.652123 31043 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:04:24.608762 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.660894 29147 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:24.679056 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:24.644603 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.689101 30853 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.690259 30812 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:24.681893 30758 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:04:24.692124 30951 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.695058 29903 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:24.695703 31009 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:24.694697 31036 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.700237 30951 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.700623 31009 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:24.700958 30951 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.703015 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.703609 29147 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:24.705777 29142 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:24.694705 30967 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.706772 30951 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.709679 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.711973 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:24.712167 29186 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:24.667882 30963 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:24.639254 30922 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 13:04:24.730875 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44972.6us
I20260709 13:04:24.725040 29143 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:24.724943 29147 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:24.725232 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:24.726918 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:24.686929 30950 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:24.733625 30951 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:24.732937 30788 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:04:24.732909 30967 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.710093 31036 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:24.774792 29911 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:04:24.733003 30758 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:04:24.788347 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.730453 30979 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:24.775118 29038 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:24.793911 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:24.794767 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:24.680948 31049 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 13:04:24.682359 31030 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:24.798657 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.776686 30999 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:24.787063 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.788448 29141 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:24.791667 29143 raft_consensus.cc:2376] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3. Re-sending same reply.
I20260709 13:04:24.803574 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.803580 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.804358 29143 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:24.792485 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.797547 29902 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:24.747792 30963 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.801385 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.783020 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:24.746775 29905 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:24.784832 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:24.801617 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:24.890066 29909 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:24.801121 30866 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:24.748656 30996 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:04:24.854409 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:24.854934 31061 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:04:24.856334 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:25.002225 29143 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:24.805506 30950 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
W20260709 13:04:24.858112 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 103048us
I20260709 13:04:24.858654 30985 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:24.860131 30922 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:04:24.860276 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 62217.6us
I20260709 13:04:24.860795 30788 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:24.790362 30991 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.885416 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.801700 30971 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:24.800307 30870 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:24.886653 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:24.853268 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:24.892701 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:25.048914 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 156302us
W20260709 13:04:25.049623 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.790198 30967 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:24.894431 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 92887.9us
I20260709 13:04:25.056443 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:25.057302 29143 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:24.923285 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:24.884929 31030 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:24.937863 30929 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:24.939826 30943 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.941425 30812 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:24.942914 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:25.082818 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:24.948321 31059 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.478s	user 0.010s	sys 0.000s
I20260709 13:04:24.806020 30993 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.955744 29064 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:04:24.947680 31032 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.515s	user 0.007s	sys 0.000s
I20260709 13:04:24.998754 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:25.079288 29903 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:25.059736 30971 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.943084 30796 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:24.852221 31020 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.193s
I20260709 13:04:25.105690 28971 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 270) took 7013 ms. Trace:
I20260709 13:04:25.045061 29072 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:25.037567 29112 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:25.115775 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 78291.5us
I20260709 13:04:25.116838 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:25.052888 31073 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:25.054836 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.058693 29073 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:25.120280 29910 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:25.097157 30943 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.081018 29148 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:24.946154 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:04:25.081887 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 355034us
I20260709 13:04:25.003170 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:25.163816 29910 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:25.029708 30852 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:25.033553 31027 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.098904 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:25.170567 29143 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:25.097220 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:25.045495 31049 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:24.998786 29146 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:25.103674 29213 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:25.047353 30942 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:24.803393 30854 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:25.056061 30870 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.049870 30947 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:25.110095 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:25.119511 30971 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.112583 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:24.893024 31043 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.118099 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:25.118965 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.119903 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:25.120824 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.121222 29221 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:25.121460 31073 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.096812 30812 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:25.038010 30948 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.169387 29219 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:25.171608 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 74519.7us
W20260709 13:04:25.213209 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 95229.3us
I20260709 13:04:25.107775 31032 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.185014 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:25.127364 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.185602 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:25.060330 30991 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:25.183462 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 306635us
W20260709 13:04:25.187211 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 462013us
I20260709 13:04:25.187750 29216 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:04:25.234480 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.193560 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:25.111128 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:25.193812 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 247726us
I20260709 13:04:25.235888 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:25.194984 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:25.196221 30870 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.098693 30746 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:04:25.103223 30965 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 13:04:25.158484 30984 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:25.203727 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:25.079841 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 289591us
I20260709 13:04:25.268410 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:25.269455 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 226981us
I20260709 13:04:25.159377 31034 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.212294 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.181728 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:25.228814 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.277984 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:25.186615 30676 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:25.113775 30993 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.197788 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:25.241514 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:25.295553 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:25.295674 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.296450 29220 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:25.245007 30947 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:25.245867 31072 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:25.115370 28971 rpcz_store.cc:276] 0709 13:04:18.092260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.092415 (+   155us) service_pool.cc:224] Handling call
0709 13:04:20.255347 (+2162932us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400018954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a77090 00007f7836b15f3a 00007f783c8cc1cc 00007f783c9ca37e 00007f783ae8b2a4 00007f783ae8b232 00007f783ae8b161 00007f783ae8b0dc 00007f783ae8a392 00007f7834e4848c 00007f7834e478c6
0709 13:04:25.105617 (+4850270us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:23.249490 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:23.274979 (+ 25489us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:23.275015 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:04:23.275033 (+    18us) tablet.cc:662] Decoding operations
0709 13:04:23.275497 (+   464us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:23.275564 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:23.275593 (+    29us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:04:23.321450 (+ 45857us) tablet.cc:701] Row locks acquired
0709 13:04:23.321496 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:04:23.332882 (+ 11386us) write_op.cc:270] Start()
0709 13:04:23.337510 (+  4628us) write_op.cc:276] Timestamp: P: 1783602263329153 usec, L: 0
0709 13:04:23.347529 (+ 10019us) op_driver.cc:348] REPLICATION: starting
0709 13:04:23.432005 (+ 84476us) log.cc:844] Serialized 920 byte log entry
0709 13:04:23.810061 (+378056us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b18002207e0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:04:24.116698 (+306637us) op_driver.cc:464] REPLICATION: finished
0709 13:04:24.340246 (+223548us) write_op.cc:301] APPLY: starting
0709 13:04:24.352836 (+ 12590us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:24.378448 (+ 25612us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b5400018954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0a32a 00007f7836a8ff14 00007f7836a8fec3 00007f7836a8fe38 00007f7836a8fdf3 00007f7836a8f0dc 00007f7836ab8e16 00007f7836aa0cbb 00007f7836a8b39d 00007f7836a8ed75
0709 13:04:24.392580 (+ 14132us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:24.398029 (+  5449us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:24.420378 (+ 22349us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:24.422529 (+  2151us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:04:24.422590 (+    61us) write_op.cc:312] APPLY: finished
0709 13:04:24.422784 (+   194us) log.cc:844] Serialized 29 byte log entry
0709 13:04:24.422968 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:24.423077 (+   109us) write_op.cc:454] Released schema lock
0709 13:04:24.423127 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:24.423582 (+   455us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:24.423616 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":870820,"reported_tablets":10,"spinlock_wait_cycles":3516133120,"child_traces":[["op",{"apply.queue_time_us":73666,"apply.run_cpu_time_us":2375,"apply.run_wall_time_us":112116,"num_ops":2,"prepare.queue_time_us":240845,"prepare.run_cpu_time_us":30481,"prepare.run_wall_time_us":321982,"raft.queue_time_us":208323,"raft.run_cpu_time_us":27517,"raft.run_wall_time_us":334873,"replication_time_us":716522,"spinlock_wait_cycles":42944128,"thread_start_us":409473,"threads_started":4,"wal-append.queue_time_us":144449}]]}
I20260709 13:04:25.195497 30852 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:04:25.262748 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 56134.8us
I20260709 13:04:25.269348 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:25.202210 29215 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
W20260709 13:04:25.272069 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.228724 30950 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.228211 30898 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.277978 29142 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:25.193634 31027 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.280611 29140 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:25.315591 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.316521 29219 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:25.280895 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:25.278307 29218 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:25.215996 31073 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.211416 30979 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:25.244238 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:25.335906 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 66654us
I20260709 13:04:25.336872 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:25.297593 29145 raft_consensus.cc:2368] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4.
W20260709 13:04:25.311362 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.249075 31068 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:25.212777 31043 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.357738 29142 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:25.346882 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 102741us
I20260709 13:04:25.330124 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.330214 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.332561 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.349720 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:25.333626 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:25.307278 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 73999.8us
I20260709 13:04:25.303251 30788 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:25.334354 30898 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.345736 29143 raft_consensus.cc:2410] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:25.345800 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:25.233000 30854 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:04:25.232825 30866 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:04:25.348819 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:25.355893 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:25.361889 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:25.383697 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.363790 29144 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:25.369009 29148 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:25.386457 29217 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:25.384143 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.344964 30950 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.371721 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:25.308961 30984 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:25.389848 29220 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:25.373072 29141 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:25.358184 31017 consensus_meta.cc:287] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.754s	user 0.010s	sys 0.000s
W20260709 13:04:25.382757 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.388190 29144 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:25.369735 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:25.382839 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 146920us
I20260709 13:04:25.393962 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:25.384742 31079 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:25.399667 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.364406 30929 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:04:25.400754 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52009.8us
I20260709 13:04:25.401489 29143 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:25.388806 29142 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:25.368985 30788 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:25.368829 30758 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:25.371726 30922 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:25.315662 31034 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.405303 30984 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.383213 31080 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.385877 31070 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.399708 31062 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.399s	user 0.001s	sys 0.000s
I20260709 13:04:25.415314 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:25.433588 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.450922 29218 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 1.
I20260709 13:04:25.348968 30812 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:25.440860 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47024.6us
W20260709 13:04:25.411558 31063 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.398s	user 0.000s	sys 0.001s
I20260709 13:04:25.400125 30963 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:25.404172 31007 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:25.405786 30979 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.441340 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:25.434078 30788 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:25.448936 30947 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:25.454300 30812 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:25.414484 29066 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:25.433378 30951 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:25.440821 31072 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:25.459707 29148 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
W20260709 13:04:25.500859 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 59603.1us
I20260709 13:04:25.451882 31082 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:25.426318 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:25.465051 30967 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:25.509657 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:25.461167 30965 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.471424 29067 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:25.528432 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 178770us
I20260709 13:04:25.411856 31084 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.080s
W20260709 13:04:25.533277 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 177494us
I20260709 13:04:25.434763 31079 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.544366 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:25.537267 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 122041us
I20260709 13:04:25.471239 30929 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:04:25.508845 30951 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.452230 31080 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.546438 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:25.511678 31095 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:25.506110 30866 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.534102 31089 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:25.543889 29904 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:04:25.550521 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 163) took 1306 ms. Trace:
I20260709 13:04:25.576924 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:25.452418 31070 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.515969 30854 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.543165 31082 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:04:25.554234 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 129304us
I20260709 13:04:25.462242 30852 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.553015 31032 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:25.606444 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:25.512280 31007 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:25.533566 30948 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:25.575079 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1552) took 1295 ms. Trace:
I20260709 13:04:25.541911 30967 raft_consensus.cc:2764] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:04:25.622849 31092 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:25.605269 30883 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:25.537235 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:25.626778 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 89622us
I20260709 13:04:25.627679 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:25.564293 31072 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:25.573026 31047 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:25.643550 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:25.605664 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 96095.9us
W20260709 13:04:25.606600 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 152246us
I20260709 13:04:25.584316 30676 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:25.562577 31068 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:25.607738 29214 rpcz_store.cc:276] 0709 13:04:24.284172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.389199 (+105027us) service_pool.cc:224] Handling call
0709 13:04:24.389628 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:24.396221 (+  6593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:24.396270 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:24.420914 (+ 24644us) log.cc:844] Serialized 26 byte log entry
0709 13:04:24.781060 (+360146us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:24.781097 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.781160 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:25.574838 (+793678us) raft_consensus.cc:1685] finished
0709 13:04:25.574860 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.575011 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1764,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":4356,"thread_start_us":308556,"threads_started":1,"wal-append.queue_time_us":412314}
W20260709 13:04:25.621397 31067 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:25.580118 30866 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.659350 30866 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:25.658958 31068 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.537391 30965 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.577117 30993 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:25.580734 31084 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:25.576967 29071 rpcz_store.cc:276] 0709 13:04:24.244168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.383983 (+139815us) service_pool.cc:224] Handling call
0709 13:04:24.384408 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:24.512270 (+127862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:24.512303 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:24.512815 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 13:04:24.780602 (+267787us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:24.792852 (+ 12250us) log.cc:844] Serialized 14 byte log entry
0709 13:04:24.794979 (+  2127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.923873 (+128894us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400186d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:24.923940 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:25.029449 (+105509us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b1800343b40. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20522 00007f7833a227c1
0709 13:04:25.550231 (+520782us) raft_consensus.cc:1685] finished
0709 13:04:25.550268 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.550457 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48822,"prepare.queue_time_us":12026,"prepare.run_wall_time_us":2687,"raft.queue_time_us":36771,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":13752,"spinlock_wait_cycles":431297024,"thread_start_us":257943,"threads_started":1,"wal-append.queue_time_us":340259}
I20260709 13:04:25.645116 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:25.546566 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:25.603412 31095 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.646749 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:25.653663 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:25.664772 29214 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:25.663018 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:25.646859 30967 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.563295 31077 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.109s
I20260709 13:04:25.635545 31072 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:25.665453 29071 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:25.646703 30963 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:25.625169 31007 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:25.667140 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 120677us
W20260709 13:04:25.684186 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.666772 31095 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:25.667542 31086 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:04:25.667661 30993 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:25.666405 31100 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:04:25.670730 31047 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:25.669041 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:25.669116 30929 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:25.666594 30866 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:25.666385 31101 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:25.605995 30854 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.646025 30898 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:25.666725 31074 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 13:04:25.661279 30950 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:25.662407 31084 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.666888 31069 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:25.670313 31099 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:25.668079 31082 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:25.675879 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.673062 31027 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:25.677078 31072 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:25.677917 30963 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.678078 31034 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:25.692701 30993 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.709887 29910 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:25.693486 31047 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:25.676329 30967 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.696768 31079 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:25.666533 30676 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.704506 31084 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:25.706297 30950 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:25.735692 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:25.736740 29068 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:25.704006 31099 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.740748 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:25.719810 30993 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.719350 30963 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.722007 31047 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.710948 31090 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:25.722755 31034 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:25.744527 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:25.748265 29068 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:25.734941 30883 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.734912 31079 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:25.756148 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:25.758282 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:25.759173 29068 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:25.742115 31099 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:25.760375 31070 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:25.774741 31088 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:04:25.762055 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:25.782075 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:25.744597 30929 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:25.785748 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:25.796234 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:25.804265 30967 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:25.784756 31087 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:04:25.802922 29071 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:25.810050 30965 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:25.811147 30854 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:25.774988 31107 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:25.813983 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:25.817618 31095 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:25.760999 31079 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:25.818495 30943 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.002s
I20260709 13:04:25.818624 31073 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.816601 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:25.756857 30950 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:25.818177 31103 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:04:25.825891 29071 raft_consensus.cc:2376] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3. Re-sending same reply.
I20260709 13:04:25.821328 31080 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:25.860242 30788 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.862391 30788 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:25.864319 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:25.864974 29068 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
W20260709 13:04:25.840418 31096 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:04:25.795639 30929 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.834271 31107 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.848948 30943 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.851648 31073 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:25.856705 29221 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:25.762698 30883 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:25.869462 30971 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.658s	user 0.006s	sys 0.005s
I20260709 13:04:25.859293 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:25.899353 29071 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 2: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:25.902289 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:25.874581 29904 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:25.905715 29064 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:25.901945 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:25.856058 30870 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.570s	user 0.016s	sys 0.000s
I20260709 13:04:25.884608 31079 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:25.891342 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:25.924430 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:25.925257 29144 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.895555 29111 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:25.906208 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1554) took 1019 ms. Trace:
I20260709 13:04:25.926139 29064 rpcz_store.cc:276] 0709 13:04:24.886461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.896937 (+1010476us) service_pool.cc:224] Handling call
0709 13:04:25.906142 (+  9205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:25.903090 30821 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:04:25.906610 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:25.930406 29071 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:25.907308 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:25.881081 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:25.907570 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:25.907981 31073 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.933977 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:25.916800 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:25.935720 29143 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:25.917186 30788 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:25.918185 30943 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:25.919068 31089 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:25.898205 30971 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.926831 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:25.932111 29911 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:04:25.938840 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.005s	sys 0.024s
W20260709 13:04:25.947782 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.005s	sys 0.024s
I20260709 13:04:25.948657 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1553) took 1053 ms. Trace:
I20260709 13:04:25.949244 29068 rpcz_store.cc:276] 0709 13:04:24.886153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.937395 (+1051242us) service_pool.cc:224] Handling call
0709 13:04:25.939332 (+  1937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.939550 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.939586 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:25.939642 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:25.939710 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.939780 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:25.939904 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:25.951490 29068 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:25.957396 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1556) took 1070 ms. Trace:
I20260709 13:04:25.958012 29068 rpcz_store.cc:276] 0709 13:04:24.887031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.950696 (+1063665us) service_pool.cc:224] Handling call
0709 13:04:25.951182 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.957060 (+  5878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.957323 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:25.930212 31032 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:25.960832 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:25.961864 30922 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.001s
I20260709 13:04:25.923825 30870 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:25.968794 29069 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:25.970474 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:25.974493 30950 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:25.976672 29068 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:25.979632 29071 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
W20260709 13:04:25.980768 29036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.008s	sys 0.029s
I20260709 13:04:25.982753 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1557) took 1095 ms. Trace:
W20260709 13:04:25.983073 29036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.008s	sys 0.029s
W20260709 13:04:25.989109 29219 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.033s	sys 0.038s
W20260709 13:04:26.008596 29219 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.034s	sys 0.038s
I20260709 13:04:25.983412 29068 rpcz_store.cc:276] 0709 13:04:24.887292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.959168 (+1071876us) service_pool.cc:224] Handling call
0709 13:04:25.976336 (+ 17168us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.982433 (+  6097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:25.982662 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:25.983078 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:26.012918 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:26.014853 29113 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:26.014868 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:26.016359 29219 raft_consensus.cc:2410] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:25.975286 30997 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:26.018333 29214 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:04:26.019271 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:26.017572 30821 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:26.024597 30821 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:26.025605 29064 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:04:25.939424 31073 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:25.980762 30788 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:25.976263 30922 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:26.032618 29071 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:25.980544 31092 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.008687 31032 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:26.031303 29068 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:04:26.035619 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.035808 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1560) took 1146 ms. Trace:
I20260709 13:04:26.033445 31033 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:26.042757 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:26.062026 29071 rpcz_store.cc:276] 0709 13:04:24.888901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.031991 (+1143090us) service_pool.cc:224] Handling call
0709 13:04:26.032379 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.035325 (+  2946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.035730 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.025884 30746 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:26.026865 30852 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.065263 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:26.025794 31036 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:26.067680 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:26.105520 29071 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:26.044450 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1561) took 1006 ms. Trace:
I20260709 13:04:26.108855 29068 rpcz_store.cc:276] 0709 13:04:25.038784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.010845 (+972061us) service_pool.cc:224] Handling call
0709 13:04:26.031011 (+ 20166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.044066 (+ 13055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.044392 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.073449 29219 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
W20260709 13:04:26.040961 30758 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.000s	sys 0.009s
I20260709 13:04:26.044682 31092 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.208328 29144 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
W20260709 13:04:26.572074 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 392488us
I20260709 13:04:26.473026 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1752) took 1686 ms. Trace:
W20260709 13:04:26.356895 29069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1546) took 1553 ms (client timeout 1546 ms). Trace:
I20260709 13:04:26.590396 29140 rpcz_store.cc:276] 0709 13:04:24.786245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.786390 (+   145us) service_pool.cc:224] Handling call
0709 13:04:26.472824 (+1686434us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54003a52d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:26.472942 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106850688}
W20260709 13:04:26.590852 29069 rpcz_store.cc:269] 0709 13:04:24.803747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.854132 (+ 50385us) service_pool.cc:224] Handling call
0709 13:04:26.356585 (+1502453us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b540031e7d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:26.356773 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2161266944}
W20260709 13:04:26.325601 31113 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:04:26.472226 31080 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:26.594971 31080 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:04:26.247583 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 226184us
I20260709 13:04:26.159282 31033 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:26.596455 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:26.596163 31080 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:26.251159 29039 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.014s	sys 0.032s
W20260709 13:04:26.597520 29039 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.014s	sys 0.053s
I20260709 13:04:26.597189 31033 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:26.598337 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 929345us
I20260709 13:04:26.335666 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 228) took 1218 ms. Trace:
W20260709 13:04:26.248333 31083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.003s	sys 0.020s
W20260709 13:04:26.599611 31083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.144s	user 0.006s	sys 0.029s
I20260709 13:04:26.160980 31107 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:26.600102 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 941616us
I20260709 13:04:26.118218 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:26.340961 30985 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:04:26.324729 31124 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:04:26.604203 31094 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.186247 31109 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:26.211617 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.603794 30985 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:26.601162 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:26.601799 30758 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.135s	user 0.005s	sys 0.027s
I20260709 13:04:26.617720 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:26.620154 29071 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:26.355821 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:26.622463 29214 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:26.472860 29903 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:26.357857 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:26.113688 30852 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.224263 31084 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.604804 31094 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.605765 31036 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.605835 31007 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:26.055980 30788 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.558s
I20260709 13:04:26.655311 31094 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:26.606410 31089 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.606503 31112 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:26.224726 30942 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:26.607054 30746 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.149745 30929 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:26.209326 29215 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.029s	sys 0.039s
W20260709 13:04:26.665402 29215 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.320s	user 0.029s	sys 0.057s
W20260709 13:04:26.608613 30676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.013s	sys 0.012s
I20260709 13:04:26.229453 30971 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:26.624925 31083 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.609549 31092 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:26.610188 31109 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.599747 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:26.599354 29220 rpcz_store.cc:276] 0709 13:04:25.117691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.297071 (+179380us) service_pool.cc:224] Handling call
0709 13:04:26.335399 (+1038328us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b54000bd454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:26.335552 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69516800}
W20260709 13:04:26.315776 31115 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:04:26.562202 31073 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:26.662148 31107 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:26.600656 31072 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.867s	user 0.004s	sys 0.012s
W20260709 13:04:26.215345 31030 consensus_meta.cc:287] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.132s	user 0.008s	sys 0.006s
I20260709 13:04:26.104110 30951 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:26.581895 30947 consensus_meta.cc:287] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.081s	user 0.011s	sys 0.000s
W20260709 13:04:26.588307 30963 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.818s	user 0.010s	sys 0.002s
I20260709 13:04:26.621703 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 165) took 1407 ms. Trace:
I20260709 13:04:26.689522 29071 rpcz_store.cc:276] 0709 13:04:25.214626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.619561 (+1404935us) service_pool.cc:224] Handling call
0709 13:04:26.619901 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.621412 (+  1511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.621635 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.045209 31101 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:26.618337 30758 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:26.332798 30940 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:26.105314 30978 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:26.684458 31107 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.611701 31009 raft_consensus.cc:2764] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:04:26.649919 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48876.4us
I20260709 13:04:26.702762 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1549) took 2659 ms. Trace:
I20260709 13:04:26.703190 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:26.653991 30692 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.561385 30883 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:26.640199 30968 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.604559 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:26.703406 29216 rpcz_store.cc:276] 0709 13:04:24.036979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.268901 (+231922us) service_pool.cc:224] Handling call
0709 13:04:24.269247 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.234158 (+964911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.696293 (+1462135us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b54000bde54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:26.696575 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2111678848}
I20260709 13:04:26.705634 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1812) took 1020 ms. Trace:
I20260709 13:04:26.656329 31036 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.660319 31080 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:26.681294 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 81630.1us
I20260709 13:04:26.661943 30942 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:26.667562 30676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.014s	sys 0.013s
I20260709 13:04:26.165414 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:26.714840 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1810) took 1030 ms. Trace:
I20260709 13:04:26.715291 29219 rpcz_store.cc:276] 0709 13:04:25.684746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.150062 (+465316us) service_pool.cc:224] Handling call
0709 13:04:26.714783 (+564721us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:26.668730 29067 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.120s	user 0.007s	sys 0.007s
I20260709 13:04:26.662608 30987 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:26.658638 31094 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:26.667158 30746 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.697218 30940 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:04:26.683022 31099 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.889s	user 0.003s	sys 0.009s
W20260709 13:04:26.669117 30866 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.970s	user 0.012s	sys 0.001s
I20260709 13:04:26.692487 30963 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.693106 31084 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:26.693403 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1564) took 1338 ms. Trace:
I20260709 13:04:26.659022 31007 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:26.588056 30993 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.833s	user 0.009s	sys 0.000s
I20260709 13:04:26.665511 31089 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.698105 29072 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:26.684289 31117 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:26.694120 31129 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 13:04:26.470424 29066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 160) took 2227 ms (client timeout 1958 ms). Trace:
I20260709 13:04:26.635425 30943 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.720113 31094 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:26.704546 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 388) took 1968 ms. Trace:
I20260709 13:04:26.707108 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:26.706128 29218 rpcz_store.cc:276] 0709 13:04:25.685327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.604284 (+918957us) service_pool.cc:224] Handling call
0709 13:04:26.705558 (+101274us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:26.696707 31047 consensus_meta.cc:287] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.938s	user 0.011s	sys 0.001s
I20260709 13:04:26.691233 30947 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:26.711261 30968 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.682814 31083 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.757741 30984 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:04:26.772943 31083 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:26.712409 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:26.773705 31138 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:26.770388 30996 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:26.774235 31138 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.774866 30996 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:26.661943 31112 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.716517 29216 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:26.777760 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1565) took 1422 ms. Trace:
I20260709 13:04:26.777993 31112 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:26.687075 31072 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.778098 29216 rpcz_store.cc:276] 0709 13:04:25.355389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.706426 (+1351037us) service_pool.cc:224] Handling call
0709 13:04:26.716107 (+  9681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.777472 (+ 61365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.777698 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.705439 31107 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.780640 31072 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:26.781446 30996 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.782805 30996 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.782497 28969 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:26.725385 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 129037us
I20260709 13:04:26.752849 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 234) took 1122 ms. Trace:
I20260709 13:04:26.784283 29219 rpcz_store.cc:276] 0709 13:04:25.630412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.719984 (+1089572us) service_pool.cc:224] Handling call
0709 13:04:26.751483 (+ 31499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.751735 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.751771 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.751828 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:26.751889 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.752620 (+   731us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.752774 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.707083 30978 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.787533 30978 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:26.719710 30987 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:26.719545 29067 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:26.711468 30692 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.774904 31117 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:26.761161 29908 rpcz_store.cc:276] 0709 13:04:24.731300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.732257 (+   957us) service_pool.cc:224] Handling call
0709 13:04:24.732560 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.183760 (+451200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.183880 (+   120us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.196105 (+ 12225us) log.cc:844] Serialized 26 byte log entry
0709 13:04:25.416503 (+220398us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:25.416565 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.698773 (+1282208us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b5400682754. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:26.698834 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.698887 (+    53us) raft_consensus.cc:1685] finished
0709 13:04:26.698902 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.699129 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1035,"raft.queue_time_us":297292,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":15130,"spinlock_wait_cycles":102851328,"thread_start_us":565964,"threads_started":2,"wal-append.queue_time_us":340272}
I20260709 13:04:26.731613 30866 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:26.764387 31094 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:26.769807 29902 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:26.770615 30991 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:26.774267 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 61929.8us
I20260709 13:04:26.730405 29220 rpcz_store.cc:276] 0709 13:04:25.355105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.686850 (+1331745us) service_pool.cc:224] Handling call
0709 13:04:26.692862 (+  6012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.693054 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.693083 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.693129 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:26.693169 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.693230 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.693342 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.755467 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1487) took 1537 ms. Trace:
I20260709 13:04:26.788283 29073 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:26.808512 29072 rpcz_store.cc:276] 0709 13:04:25.218428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.697520 (+1479092us) service_pool.cc:224] Handling call
0709 13:04:26.697872 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.755145 (+ 57273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.755397 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.809073 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1488) took 1590 ms. Trace:
I20260709 13:04:26.809405 29073 rpcz_store.cc:276] 0709 13:04:25.218706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.787651 (+1568945us) service_pool.cc:224] Handling call
0709 13:04:26.788022 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.808794 (+ 20772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.809014 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.808521 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:26.812842 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:26.817521 29072 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:04:26.818261 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1489) took 1599 ms. Trace:
I20260709 13:04:26.818562 29072 rpcz_store.cc:276] 0709 13:04:25.218963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.816926 (+1597963us) service_pool.cc:224] Handling call
0709 13:04:26.817286 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.817958 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.818205 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.759132 30993 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.788923 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1814) took 1094 ms. Trace:
I20260709 13:04:26.670058 30985 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:26.832687 29145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 1460) took 2308 ms (client timeout 2176 ms). Trace:
I20260709 13:04:26.728858 31099 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.794114 28969 catalog_manager.cc:5697] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, term changed from 1 to 3, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to <none>, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New cstate: current_term: 3 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:26.798281 30978 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:26.821228 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:26.819732 31117 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.823640 29216 rpcz_store.cc:276] 0709 13:04:25.685865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.779159 (+1093294us) service_pool.cc:224] Handling call
0709 13:04:26.779450 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.779636 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.779660 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.779698 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:26.779730 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.779782 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.779875 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.792273 29139 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
W20260709 13:04:26.763736 29907 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.024s	sys 0.064s
I20260709 13:04:26.767625 31047 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.767271 30943 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.838624 28969 catalog_manager.cc:5697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 3 to 4, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New cstate: current_term: 1 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:26.858035 28969 catalog_manager.cc:5697] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:26.833191 29145 rpcz_store.cc:269] 0709 13:04:24.511855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.516766 (+  4911us) service_pool.cc:224] Handling call
0709 13:04:26.820025 (+2303259us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b54000e9cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:26.820185 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1528493312}
W20260709 13:04:26.757747 29066 rpcz_store.cc:269] 0709 13:04:24.243386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.344408 (+101022us) service_pool.cc:224] Handling call
0709 13:04:26.470068 (+2125660us) spinlock_profiling.cc:243] Waited 969 ms on lock 0x7b54002cbfd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:26.470275 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2228930432}
I20260709 13:04:26.822204 31111 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:26.859764 28969 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:26.838231 29907 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.024s	sys 0.064s
I20260709 13:04:26.863194 29072 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:26.864526 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 153328us
I20260709 13:04:26.856992 29073 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:26.833745 30852 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:26.866232 28969 catalog_manager.cc:5697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:26.791290 31101 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.795022 30940 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.884560 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1490) took 1665 ms. Trace:
I20260709 13:04:26.889829 29073 rpcz_store.cc:276] 0709 13:04:25.219990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.821482 (+1601492us) service_pool.cc:224] Handling call
0709 13:04:26.856757 (+ 35275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.884289 (+ 27532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.884503 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.885658 28969 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 2 to 3, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 3 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:26.769296 30758 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.861474 30866 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:26.856752 31139 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:26.868402 31111 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:26.866820 31114 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:04:26.868908 31109 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:26.867195 30985 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.832566 30898 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:04:26.914137 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:26.884032 30692 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:26.924767 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:26.929543 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:26.927124 29073 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:26.927968 29910 raft_consensus.cc:1813] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Handling vote request from an unknown peer 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:26.931141 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:26.931747 29910 raft_consensus.cc:2410] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 3 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20260709 13:04:26.934070 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 227042us
I20260709 13:04:26.934917 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:26.893350 30852 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:26.849887 30978 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.910487 30940 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.911355 29066 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:26.953428 29139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1755) took 2166 ms (client timeout 1699 ms). Trace:
I20260709 13:04:26.954034 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1567) took 1310 ms. Trace:
I20260709 13:04:26.954136 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:26.954680 30870 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:26.954545 29066 rpcz_store.cc:276] 0709 13:04:25.644156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.860807 (+1216651us) service_pool.cc:224] Handling call
0709 13:04:26.861179 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.953744 (+ 92565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.953975 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:26.955394 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:26.955497 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 405) took 1065 ms. Trace:
I20260709 13:04:26.955966 29142 raft_consensus.cc:2376] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2. Re-sending same reply.
I20260709 13:04:26.955935 29905 rpcz_store.cc:276] 0709 13:04:25.889762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.903005 (+ 13243us) service_pool.cc:224] Handling call
0709 13:04:26.347712 (+444707us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:26.347984 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.796898 (+448914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.799384 (+  2486us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.804720 (+  5336us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:26.806997 (+  2277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.955261 (+148264us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54005a32d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:26.955307 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.955439 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026587776}
I20260709 13:04:26.954938 29143 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:26.958484 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:26.959097 29140 raft_consensus.cc:2376] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3. Re-sending same reply.
I20260709 13:04:26.959550 29217 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:26.909961 31135 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.960207 31135 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.961395 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1808) took 1614 ms. Trace:
I20260709 13:04:26.961747 29217 rpcz_store.cc:276] 0709 13:04:25.347029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.449346 (+102317us) service_pool.cc:224] Handling call
0709 13:04:26.959360 (+1510014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.959942 (+   582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.961214 (+  1272us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b540033b3d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:26.961339 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3471958272}
I20260709 13:04:26.943626 29911 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:26.962963 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:26.891768 30886 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:04:26.946398 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1555) took 2059 ms. Trace:
I20260709 13:04:26.964118 29064 rpcz_store.cc:276] 0709 13:04:24.886772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.939210 (+1052438us) service_pool.cc:224] Handling call
0709 13:04:25.941844 (+  2634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.030877 (+ 89033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.946054 (+915177us) spinlock_profiling.cc:243] Waited 83.5 ms on lock 0x7b54003219d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:26.946319 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191947136}
I20260709 13:04:26.965418 30968 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:26.965618 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 403) took 1771 ms. Trace:
I20260709 13:04:26.966434 31007 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:26.967316 31127 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 13:04:26.968302 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:26.968978 29144 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
W20260709 13:04:26.969314 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1562) took 1930 ms (client timeout 1544 ms). Trace:
I20260709 13:04:26.969758 29904 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:26.969731 29067 rpcz_store.cc:269] 0709 13:04:25.039119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.266760 (+227641us) service_pool.cc:224] Handling call
0709 13:04:26.969126 (+1702366us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b54001d8b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:26.969245 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115752704}
I20260709 13:04:26.971066 29071 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:26.944734 30692 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.972219 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1486) took 1754 ms. Trace:
I20260709 13:04:26.972584 29071 rpcz_store.cc:276] 0709 13:04:25.217431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.690589 (+1473158us) service_pool.cc:224] Handling call
0709 13:04:26.970848 (+280259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.971430 (+   582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.972035 (+   605us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54001d8b50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:26.972163 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643816704}
I20260709 13:04:26.973295 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 232) took 1785 ms. Trace:
I20260709 13:04:26.973676 29215 rpcz_store.cc:276] 0709 13:04:25.188406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.328485 (+140079us) service_pool.cc:224] Handling call
0709 13:04:25.328857 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:25.329085 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:26.665874 (+1336789us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.972937 (+307063us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:26.972991 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:26.973051 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.973123 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.973236 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166578,"prepare.queue_time_us":333754,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":1627,"raft.queue_time_us":167213,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":77,"thread_start_us":1460405,"threads_started":2,"wal-append.queue_time_us":274470}
I20260709 13:04:26.975894 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 408) took 1083 ms. Trace:
W20260709 13:04:26.976133 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 273052us
I20260709 13:04:26.976950 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:26.969347 30943 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:26.937126 31130 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:04:26.978705 29188 leader_election.cc:400] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4.
I20260709 13:04:26.978803 30943 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:26.979065 29188 leader_election.cc:403] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:04:26.979259 30943 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.979696 31049 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:26.979759 30929 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:26.980105 31119 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:26.883229 31117 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:04:26.981091 31029 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:26.981521 31029 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:26.981540 31120 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:26.982069 31120 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:26.982369 31029 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:26.950153 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:26.951540 31099 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:26.952580 29147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1779) took 2024 ms (client timeout 1837 ms). Trace:
I20260709 13:04:26.984687 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:26.962363 31135 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:26.980471 30929 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:26.980623 30922 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:26.983837 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1553) took 1955 ms. Trace:
W20260709 13:04:26.958479 29221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 231) took 1840 ms (client timeout 1949 ms). Trace:
I20260709 13:04:27.007078 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:26.861414 30987 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:26.911674 31138 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:26.914561 31111 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.017467 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:27.000274 31073 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:26.953850 29139 rpcz_store.cc:269] 0709 13:04:24.787385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.792272 (+  4887us) service_pool.cc:224] Handling call
0709 13:04:26.953273 (+2161001us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b5400145354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:26.953365 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4334806272}
I20260709 13:04:26.983404 31029 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.923509 31112 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.988142 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:26.989426 29910 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:26.990774 29215 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:26.991532 29216 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:26.991612 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:26.994099 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.030632 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.995427 29906 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:26.995639 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:26.988881 29147 rpcz_store.cc:269] 0709 13:04:24.928243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.106607 (+178364us) service_pool.cc:224] Handling call
0709 13:04:26.952381 (+1845774us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400192154. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:26.952513 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3515718400}
I20260709 13:04:26.996423 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.004798 30929 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.012710 29220 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:04:27.013394 29148 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:04:27.007326 29221 rpcz_store.cc:269] 0709 13:04:25.118504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.203434 (+ 84930us) service_pool.cc:224] Handling call
0709 13:04:26.958259 (+1754825us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b540033b3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:26.958399 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353002240}
I20260709 13:04:26.986029 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:26.979720 30943 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:26.981099 31119 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:27.006364 29222 rpcz_store.cc:276] 0709 13:04:25.028905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.096728 (+ 67823us) service_pool.cc:224] Handling call
0709 13:04:25.121318 (+ 24590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.169157 (+ 47839us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.169208 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.107496 (+938288us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b2c00127298. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:04:26.108079 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:04:26.530002 (+421923us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:26.621710 (+ 91708us) log.cc:844] Serialized 14 byte log entry
0709 13:04:26.957234 (+335524us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b18003e7100. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:04:26.957425 (+   191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.957502 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:26.983612 (+ 26110us) raft_consensus.cc:1685] finished
0709 13:04:26.983635 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:26.983779 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91081,"prepare.queue_time_us":126651,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":42,"raft.queue_time_us":397115,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":14761,"spinlock_wait_cycles":34427520,"thread_start_us":421434,"threads_started":1,"wal-append.queue_time_us":582013}
I20260709 13:04:27.028630 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:27.024928 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.003113 30796 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:26.913899 31139 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:27.002452 30963 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:26.982849 31120 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.065558 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:27.077481 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.006072 30886 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:26.966085 29903 rpcz_store.cc:276] 0709 13:04:25.193837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.205077 (+ 11240us) service_pool.cc:224] Handling call
0709 13:04:25.307800 (+102723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.644244 (+1336444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.964814 (+320570us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b540063de20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a84dc0 00007f7836b10a02 00007f7836b0cdd6 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:04:26.965388 (+   574us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540063d654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:26.965546 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2643954432}
I20260709 13:04:27.013837 31049 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:26.947398 30852 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.018245 31073 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.008965 31007 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.223s
I20260709 13:04:27.014485 31083 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.260s
I20260709 13:04:27.065615 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:27.086354 29142 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:04:27.078701 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 51862.8us
I20260709 13:04:27.229954 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.086386 29141 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:27.064843 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:27.385242 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 450377us
I20260709 13:04:27.064627 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.346503 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:27.086249 29140 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
W20260709 13:04:27.079901 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 91838.8us
I20260709 13:04:27.314101 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.407158 29037 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:27.354475 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:27.291482 29871 leader_election.cc:336] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.029s (SENT)
I20260709 13:04:27.407757 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:27.409269 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:27.410898 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:27.412019 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 120601us
W20260709 13:04:27.413017 29871 leader_election.cc:336] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.958s (SENT)
I20260709 13:04:27.413697 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:27.406211 29147 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 1.
I20260709 13:04:27.400983 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.424476 29139 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:04:27.424842 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.425639 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:27.031637 31126 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
I20260709 13:04:27.433954 29145 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:04:27.431424 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:27.382705 31142 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:04:27.432463 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:27.470479 29148 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:04:27.294018 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.078260 30870 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:27.031900 30984 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.423s
I20260709 13:04:27.434774 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.078163 31055 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.080145 31099 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:27.437433 29142 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:27.076032 31123 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:27.065083 30692 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:27.229789 31132 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
W20260709 13:04:27.323185 30997 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.277s	user 0.012s	sys 0.002s
I20260709 13:04:27.025812 30746 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:27.028888 31072 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.516588 31072 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.031098 31084 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:27.446679 31033 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.017s	sys 0.065s
W20260709 13:04:27.520205 31033 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.017s	sys 0.066s
I20260709 13:04:27.453287 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:27.076889 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:27.521962 29066 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:27.523157 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:27.524039 29066 raft_consensus.cc:2376] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5. Re-sending same reply.
I20260709 13:04:27.077126 30987 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:27.271661 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:27.454886 29039 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.837s (SENT) [suppressed 27 similar messages]
I20260709 13:04:27.525425 29067 raft_consensus.cc:1813] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Handling vote request from an unknown peer 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:27.526247 29067 raft_consensus.cc:2410] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 3 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:04:27.527318 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:27.456938 29214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.007s	sys 0.041s
W20260709 13:04:27.528191 29214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.008s	sys 0.041s
I20260709 13:04:27.529029 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1563) took 2174 ms. Trace:
I20260709 13:04:27.530148 29147 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:27.384876 29910 raft_consensus.cc:2393] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:27.531981 29065 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:27.386503 30886 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:04:27.533034 30940 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.573s	user 0.004s	sys 0.004s
I20260709 13:04:27.533954 29217 raft_consensus.cc:2410] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:27.439658 29145 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:27.534322 31013 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.484623 30870 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:27.535310 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.535991 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.082125 31089 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:27.081498 30993 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:27.465667 29139 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:27.501185 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.231086 31144 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:27.431722 30898 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:27.427942 31138 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:27.474661 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:27.068608 31125 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:27.557385 31125 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:27.558368 31125 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.428102 31145 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:27.546629 30997 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:27.564931 30997 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:27.568202 29067 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:27.569473 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:27.520190 31084 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:27.570843 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:27.559703 31125 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:27.573982 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:27.440232 30963 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:27.075836 31107 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.083173 31112 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.511945 30991 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:27.502280 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:27.069921 31047 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:27.529414 29214 rpcz_store.cc:276] 0709 13:04:25.354806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.649235 (+1294429us) service_pool.cc:224] Handling call
0709 13:04:26.649589 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.649802 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:26.864658 (+214856us) log.cc:844] Serialized 14 byte log entry
0709 13:04:27.528627 (+663969us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:27.528694 (+    67us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:27.528757 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.528829 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.528957 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7010,"prepare.queue_time_us":46098,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":859622,"threads_started":2}
I20260709 13:04:27.531847 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:27.533347 30886 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.534904 31013 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.071957 30922 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.575019 30898 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.623860 31084 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:27.624559 30898 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:27.624163 29066 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:27.626199 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:27.624377 31017 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.627437 31017 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.630059 31017 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:27.511132 31123 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:27.634526 30870 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.635773 30870 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:27.637172 30997 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:27.270694 31139 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:27.030601 31147 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:26.984436 29902 rpcz_store.cc:276] 0709 13:04:25.892821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.897409 (+  4588us) service_pool.cc:224] Handling call
0709 13:04:26.345533 (+448124us) spinlock_profiling.cc:243] Waited 6.25 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:26.345834 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.815584 (+469750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.975667 (+160083us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b54007e0254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:26.975838 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976223488}
I20260709 13:04:27.546761 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:27.547137 29911 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:27.484139 31120 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:27.552069 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:27.554067 31029 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.002s
I20260709 13:04:27.556950 31138 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:27.575958 29871 leader_election.cc:336] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.168s (SENT)
I20260709 13:04:27.575042 29068 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:27.496037 31150 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:27.556456 30951 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:27.661787 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.581635 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 170806us
I20260709 13:04:27.505286 30984 consensus_queue.cc:1243] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:04:27.546759 30942 consensus_meta.cc:287] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.827s	user 0.011s	sys 0.000s
W20260709 13:04:27.583472 28969 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.004s	sys 0.016s
W20260709 13:04:27.583899 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 81733.3us
W20260709 13:04:27.584774 29188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.011s	sys 0.020s
W20260709 13:04:27.668599 29188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.011s	sys 0.020s
I20260709 13:04:27.510076 31099 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:27.615422 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:27.674124 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:27.615804 29069 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:04:27.676000 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1485) took 2458 ms. Trace:
I20260709 13:04:27.676494 29069 rpcz_store.cc:276] 0709 13:04:25.217122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.581954 (+2364832us) service_pool.cc:224] Handling call
0709 13:04:27.615562 (+ 33608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.675609 (+ 60047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.675907 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:27.065809 30947 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:27.684185 30947 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.684628 31099 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:27.686563 30947 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:27.579733 29142 raft_consensus.cc:2955] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:27.623026 29218 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:04:27.579231 30993 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:04:27.623631 30922 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.660293 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:27.617899 30985 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.688s	user 0.009s	sys 0.001s
I20260709 13:04:27.499228 31055 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:27.578174 31144 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:27.666416 29144 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
W20260709 13:04:27.668076 28969 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.004s	sys 0.016s
I20260709 13:04:27.656769 30886 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:27.711552 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 135657us
I20260709 13:04:27.680861 30852 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:27.682828 31073 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:27.576130 31049 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.696776 29139 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:27.661813 31150 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:04:27.698200 29187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.018s	sys 0.027s
I20260709 13:04:27.658524 31033 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:27.705170 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:27.706753 29219 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.014s	sys 0.032s
W20260709 13:04:27.706912 29071 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.046s	sys 0.086s
W20260709 13:04:27.708400 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 474691us
I20260709 13:04:27.665408 31084 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:27.711468 29146 raft_consensus.cc:2410] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:04:27.735087 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 66378.4us
I20260709 13:04:27.668133 31138 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:27.684664 31080 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.970s	user 0.009s	sys 0.002s
I20260709 13:04:27.705802 31145 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.745205 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:27.747962 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.687808 30893 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:27.690074 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:27.776729 29187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.018s	sys 0.027s
I20260709 13:04:27.674494 31147 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:04:27.785893 29071 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.046s	sys 0.086s
W20260709 13:04:27.787485 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.785624 29214 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
W20260709 13:04:27.785604 29219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.014s	sys 0.032s
I20260709 13:04:27.687656 30942 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:27.819242 31119 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.588s	user 0.007s	sys 0.000s
W20260709 13:04:28.056766 29039 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.049s	sys 0.130s
W20260709 13:04:28.621464 29039 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.096s	user 0.075s	sys 0.160s
W20260709 13:04:27.994091 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.621793 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1571) took 1629 ms. Trace:
W20260709 13:04:28.227121 29874 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.008s	sys 0.038s
W20260709 13:04:28.623214 29874 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.016s	sys 0.054s
W20260709 13:04:28.621805 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 1.16686e+06us
I20260709 13:04:28.624552 29909 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:04:28.623615 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 923929us
I20260709 13:04:28.067101 30886 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.625458 30886 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.626215 30942 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:27.773468 30997 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.854s
I20260709 13:04:27.774658 31123 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:27.704690 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 406) took 1814 ms. Trace:
I20260709 13:04:28.629163 31119 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4.
W20260709 13:04:27.872653 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.630487 29214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1577) took 1635 ms (client timeout 1701 ms). Trace:
W20260709 13:04:28.630986 29214 rpcz_store.cc:269] 0709 13:04:26.994690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.675333 (+680643us) service_pool.cc:224] Handling call
0709 13:04:28.630424 (+955091us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:28.632078 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:28.130852 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.781278 31030 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.857s
W20260709 13:04:28.632745 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.644915 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.842752 31112 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:28.013078 30821 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.017s	sys 0.030s
I20260709 13:04:28.530581 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1566) took 2895 ms. Trace:
I20260709 13:04:28.628665 30886 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:28.694181 30886 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.006642 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1809) took 2322 ms. Trace:
W20260709 13:04:28.637285 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.694815 30886 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:27.745939 30993 consensus_queue.cc:1243] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:04:28.479386 30929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.015s	sys 0.066s
W20260709 13:04:28.690327 30821 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.177s	user 0.038s	sys 0.071s
W20260709 13:04:28.696105 30929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.167s	user 0.017s	sys 0.072s
I20260709 13:04:28.634413 30951 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.853165 31033 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:28.640578 30893 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:28.729240 30893 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.731515 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:28.732342 31112 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.685843 31099 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.000s	sys 0.009s
W20260709 13:04:28.752868 29148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.180s	user 0.038s	sys 0.091s
W20260709 13:04:28.700645 31094 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.884s	user 0.009s	sys 0.004s
W20260709 13:04:28.641019 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.089387 31136 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.172s	user 0.004s	sys 0.008s
I20260709 13:04:28.629627 29907 rpcz_store.cc:276] 0709 13:04:25.890358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.897294 (+  6936us) service_pool.cc:224] Handling call
0709 13:04:25.919214 (+ 21920us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:25.919690 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:25.919732 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:25.920367 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 13:04:26.898739 (+978372us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:26.909712 (+ 10973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.923152 (+ 13440us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.663134 (+739982us) raft_consensus.cc:1685] finished
0709 13:04:27.667737 (+  4603us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.697696 (+ 29959us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2385,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":10110,"thread_start_us":962227,"threads_started":1,"wal-append.queue_time_us":924700}
I20260709 13:04:27.834738 31107 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:28.732856 31112 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.228107 31073 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:04:28.470582 30870 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.834s	user 0.007s	sys 0.003s
W20260709 13:04:28.178139 30987 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.653s	user 0.006s	sys 0.006s
I20260709 13:04:28.645357 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 402) took 1674 ms. Trace:
W20260709 13:04:28.238624 29064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.004s	sys 0.043s
W20260709 13:04:28.793143 29064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.009s	sys 0.061s
I20260709 13:04:27.854941 31043 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:27.748948 31068 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.646752 31145 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.647449 31080 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:04:28.647686 29214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1584) took 1648 ms (client timeout 1818 ms). Trace:
I20260709 13:04:28.648376 29213 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:27.734771 30984 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.734844 30692 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:04:28.650213 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 905052us
I20260709 13:04:28.620821 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:28.617899 29142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.007s	sys 0.002s
W20260709 13:04:28.812575 29142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.007s	sys 0.002s
I20260709 13:04:28.812764 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1581) took 1787 ms. Trace:
I20260709 13:04:28.813113 29069 rpcz_store.cc:276] 0709 13:04:27.025651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.677518 (+651867us) service_pool.cc:224] Handling call
0709 13:04:28.009346 (+331828us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b5800005198. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:28.812701 (+803355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29831552}
I20260709 13:04:28.623376 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1563) took 3584 ms. Trace:
W20260709 13:04:28.652204 29217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1581) took 1655 ms (client timeout 1508 ms). Trace:
I20260709 13:04:28.814260 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:28.814497 29217 rpcz_store.cc:269] 0709 13:04:26.997250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.231875 (+234625us) service_pool.cc:224] Handling call
0709 13:04:28.622778 (+1390903us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b540009f1d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:28.652121 (+ 29343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291267328}
W20260709 13:04:28.655448 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.735903 31144 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:28.622433 29219 rpcz_store.cc:276] 0709 13:04:26.992771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.992877 (+   106us) service_pool.cc:224] Handling call
0709 13:04:27.012792 (+ 19915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.015016 (+  2224us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:04:27.033885 (+ 18869us) log.cc:844] Serialized 14 byte log entry
0709 13:04:28.621408 (+1587523us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:28.621469 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:28.621521 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.621595 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.621733 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49671,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":7827,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"thread_start_us":1587120,"threads_started":1,"wal-append.queue_time_us":1153836}
W20260709 13:04:28.026479 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.827082 30758 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.695250 31155 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:04:28.638867 30922 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:28.695302 29218 rpcz_store.cc:276] 0709 13:04:25.684483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.762948 (+1078465us) service_pool.cc:224] Handling call
0709 13:04:26.763356 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.688928 (+925572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.006215 (+317287us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b540009fbd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:28.006550 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1976399744}
I20260709 13:04:28.693925 29072 rpcz_store.cc:276] 0709 13:04:25.635157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.819276 (+1184119us) service_pool.cc:224] Handling call
0709 13:04:26.819554 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.881403 (+ 61849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.530237 (+1648834us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b540012a554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:28.530494 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99940608}
I20260709 13:04:28.056650 30985 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.838727 30866 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:28.838564 30985 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.839143 30866 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:28.840325 29906 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:27.826974 31055 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:28.476260 31153 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:28.841490 31162 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.841420 31153 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.842033 31162 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.844295 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:27.843079 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:28.845000 29219 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
W20260709 13:04:28.845244 29902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 311) took 1839 ms (client timeout 1910 ms). Trace:
W20260709 13:04:28.845461 29219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1582) took 1848 ms (client timeout 2296 ms). Trace:
W20260709 13:04:28.845783 29219 rpcz_store.cc:269] 0709 13:04:26.997512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.828822 (+1831310us) service_pool.cc:224] Handling call
0709 13:04:28.845408 (+ 16586us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:28.846894 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:27.786273 31150 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:28.847425 29219 raft_consensus.cc:1813] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Handling vote request from an unknown peer 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:28.659621 29067 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
W20260709 13:04:28.848747 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1583) took 1822 ms (client timeout 1818 ms). Trace:
I20260709 13:04:28.716583 30951 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:28.753855 29148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.181s	user 0.038s	sys 0.091s
W20260709 13:04:28.764249 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.849248 29067 rpcz_store.cc:269] 0709 13:04:27.026143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.580245 (+554102us) service_pool.cc:224] Handling call
0709 13:04:28.848677 (+1268432us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:28.850111 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.850455 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:28.851022 29067 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:28.851092 30951 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:28.851534 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1504) took 1067 ms. Trace:
I20260709 13:04:27.852767 30898 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.852319 31007 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:28.852777 31007 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:28.853291 31123 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.853506 30898 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:28.853818 31123 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:28.853703 31007 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.854782 31007 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:28.833678 30758 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:28.755225 31099 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.000s	sys 0.009s
I20260709 13:04:28.856537 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 400) took 1932 ms. Trace:
I20260709 13:04:28.794116 29911 rpcz_store.cc:276] 0709 13:04:26.970540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.970653 (+   113us) service_pool.cc:224] Handling call
0709 13:04:26.970945 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.648224 (+677279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.645141 (+996917us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b54004b50d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:28.645300 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1324612864}
I20260709 13:04:28.856848 31123 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:28.857156 30758 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:28.857457 31123 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.753844 31134 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.013s	sys 0.030s
W20260709 13:04:28.858232 31134 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.013s	sys 0.031s
I20260709 13:04:28.816859 30984 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.849323 31145 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:28.849256 30985 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:28.794377 29871 leader_election.cc:336] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.573s (SENT)
I20260709 13:04:28.858198 31123 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:28.860291 30984 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:28.860431 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 66086.1us
I20260709 13:04:28.861071 31162 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:28.861920 29219 raft_consensus.cc:2410] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 3 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20260709 13:04:28.803812 29214 rpcz_store.cc:269] 0709 13:04:26.999471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.631787 (+1632316us) service_pool.cc:224] Handling call
0709 13:04:28.647624 (+ 15837us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:28.756807 30821 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:04:28.813823 29070 rpcz_store.cc:276] 0709 13:04:25.039423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.302586 (+2263163us) service_pool.cc:224] Handling call
0709 13:04:27.328297 (+ 25711us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.328740 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.328783 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.329401 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.575869 (+246468us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:27.582784 (+  6915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.582875 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:28.623171 (+1040296us) raft_consensus.cc:1685] finished
0709 13:04:28.623192 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:28.623318 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":941,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":38066,"thread_start_us":121582,"threads_started":1,"wal-append.queue_time_us":417737}
W20260709 13:04:28.862392 29219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1576) took 1868 ms (client timeout 2485 ms). Trace:
W20260709 13:04:28.822278 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.822216 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1503) took 1088 ms. Trace:
I20260709 13:04:28.713475 31159 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:28.803846 29066 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:28.828048 31139 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.833081 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:28.833065 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:28.811646 31080 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.829584 31043 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.837040 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:27.824110 31084 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:28.850292 29142 raft_consensus.cc:2955] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:28.773468 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:28.852836 31049 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.857739 30758 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.845698 29902 rpcz_store.cc:269] 0709 13:04:27.005638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.698365 (+692727us) service_pool.cc:224] Handling call
0709 13:04:28.845180 (+1146815us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:28.863415 29219 rpcz_store.cc:269] 0709 13:04:26.994443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.846599 (+1852156us) service_pool.cc:224] Handling call
0709 13:04:28.862331 (+ 15732us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:28.766079 31094 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.786264 31073 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.794121 30893 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:28.803977 31068 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:28.851878 29067 rpcz_store.cc:276] 0709 13:04:27.784497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.850207 (+1065710us) service_pool.cc:224] Handling call
0709 13:04:28.851480 (+  1273us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:28.895905 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 60581us
I20260709 13:04:29.121061 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:29.032140 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 372628us
W20260709 13:04:29.121908 29214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1578) took 2127 ms (client timeout 2611 ms). Trace:
I20260709 13:04:29.122032 29218 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
W20260709 13:04:29.122328 29214 rpcz_store.cc:269] 0709 13:04:26.994907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.862642 (+1867735us) service_pool.cc:224] Handling call
0709 13:04:29.121848 (+259206us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:29.122622 29218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1585) took 2122 ms (client timeout 2416 ms). Trace:
W20260709 13:04:28.897686 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 57521.8us
I20260709 13:04:29.123257 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:28.864586 29069 rpcz_store.cc:276] 0709 13:04:27.733891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.814013 (+1080122us) service_pool.cc:224] Handling call
0709 13:04:28.822156 (+  8143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:28.856976 29909 rpcz_store.cc:276] 0709 13:04:26.924350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.924496 (+   146us) service_pool.cc:224] Handling call
0709 13:04:28.856303 (+1931807us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54005bd1d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:28.856475 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3900079488}
I20260709 13:04:29.123315 29111 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:29.125062 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:28.857687 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:29.130789 29911 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:29.131521 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 405) took 1386 ms. Trace:
I20260709 13:04:29.132014 29911 rpcz_store.cc:276] 0709 13:04:27.744821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.857419 (+1112598us) service_pool.cc:224] Handling call
0709 13:04:29.131447 (+274028us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:28.864851 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:28.955060 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:29.136224 29217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1580) took 2140 ms (client timeout 1885 ms). Trace:
W20260709 13:04:29.137328 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 299196us
W20260709 13:04:29.137630 29217 rpcz_store.cc:269] 0709 13:04:26.996978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.821487 (+1824509us) service_pool.cc:224] Handling call
0709 13:04:29.136150 (+314663us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:28.798699 31147 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.144438 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:28.811317 30886 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.136888 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:29.148797 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:29.139665 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:04:29.141088 29072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1500) took 1483 ms (client timeout 1910 ms). Trace:
W20260709 13:04:29.136278 29871 leader_election.cc:336] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.946s (SENT)
I20260709 13:04:28.835232 30996 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.831018 31107 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:29.123054 29218 rpcz_store.cc:269] 0709 13:04:26.999856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.832685 (+1832829us) service_pool.cc:224] Handling call
0709 13:04:29.122570 (+289885us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:28.830596 31154 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:29.013317 31170 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.059s	user 0.001s	sys 0.001s
I20260709 13:04:29.102733 30951 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.154186 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:29.154924 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.171452 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:29.172412 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.155890 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:29.149773 29072 rpcz_store.cc:269] 0709 13:04:27.657379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.832833 (+1175454us) service_pool.cc:224] Handling call
0709 13:04:29.141029 (+308196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.151345 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:29.174189 29067 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:29.174702 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:28.888516 31055 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.177652 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:29.150736 29871 leader_election.cc:336] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.949s (SENT)
W20260709 13:04:29.145433 30943 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 1.712s	user 0.013s	sys 0.000s
I20260709 13:04:28.863925 31153 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:29.157420 29219 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:29.169972 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.170060 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:29.181128 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1501) took 1501 ms. Trace:
I20260709 13:04:29.181541 29067 rpcz_store.cc:276] 0709 13:04:27.679383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.121495 (+1442112us) service_pool.cc:224] Handling call
0709 13:04:29.181042 (+ 59547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.182091 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:29.178449 29218 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:29.170531 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:29.182986 29871 leader_election.cc:336] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.049s (SENT)
I20260709 13:04:29.183303 29217 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.129559 30922 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.130095 31159 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:28.954634 30898 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:29.146283 31125 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.573s	user 0.008s	sys 0.003s
I20260709 13:04:29.176590 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.191937 29214 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:29.192936 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.193768 29214 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:29.194991 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.172765 29070 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:28.862767 31123 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:29.152093 31138 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.518s	user 0.001s	sys 0.008s
I20260709 13:04:29.144917 31162 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:29.183535 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.140242 31049 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:29.214529 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 75815.1us
I20260709 13:04:29.148545 30929 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.150872 30984 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.144223 29907 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:29.150959 30985 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:29.189234 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.136861 31043 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:29.185350 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.246765 30985 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.202499 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:29.247506 30985 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:29.248768 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46382.6us
I20260709 13:04:29.249550 29039 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:29.250227 30985 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.169603 31154 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.250979 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:29.252707 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.253293 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1492) took 1854 ms. Trace:
I20260709 13:04:29.197144 31030 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:29.205461 29874 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:29.255354 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 49993.2us
I20260709 13:04:29.202143 29218 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:29.180248 30951 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.136708 31139 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.151206 31009 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:29.214715 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:29.262734 29213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1569) took 2271 ms (client timeout 1603 ms). Trace:
I20260709 13:04:28.963135 30866 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:29.216974 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.145833 30758 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:29.226776 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:29.245529 31156 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:04:29.190542 31153 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.162251 30996 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.188162 31183 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:29.190198 31159 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:29.306669 29911 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:29.171579 31107 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:29.239959 29907 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 310) took 2234 ms (client timeout 2870 ms). Trace:
W20260709 13:04:29.243309 31167 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 13:04:29.244290 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.322518 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:29.323196 29221 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.245493 31080 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:29.246466 31043 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:29.256619 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:28.831261 31033 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:29.329690 31181 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:04:29.317757 31125 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
W20260709 13:04:29.331518 31179 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:04:29.258558 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:29.332371 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 73923.7us
I20260709 13:04:29.253650 29148 rpcz_store.cc:276] 0709 13:04:27.398986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.550953 (+151967us) service_pool.cc:224] Handling call
0709 13:04:27.551314 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.572199 (+ 20885us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.572234 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.572794 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.849621 (+1276827us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:28.849668 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.849739 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.253076 (+403337us) raft_consensus.cc:1685] finished
0709 13:04:29.253095 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.253235 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":142,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":118,"thread_start_us":1276418,"threads_started":1,"wal-append.queue_time_us":1668297}
I20260709 13:04:29.334599 31145 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:29.335124 31145 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.335597 31145 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.329677 29218 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:04:29.319855 31164 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.506s	user 0.001s	sys 0.000s
I20260709 13:04:29.264954 29070 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.151312 30987 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:04:29.266222 31030 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:29.264897 29213 rpcz_store.cc:269] 0709 13:04:26.991300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.991408 (+   108us) service_pool.cc:224] Handling call
0709 13:04:29.262520 (+2271112us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54003ccdd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:29.262640 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3679809280}
I20260709 13:04:29.187242 30943 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.262724 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.227558 31134 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:29.229032 30984 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.187108 31055 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.190112 30922 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:29.260207 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 78070.7us
W20260709 13:04:29.269663 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.348996 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.349766 29216 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:29.149457 31174 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 13:04:29.271770 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:29.349231 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:29.351138 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 79441.6us
I20260709 13:04:29.352751 29113 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:29.201526 30898 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.353818 30951 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.354053 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.241420 31112 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:29.355191 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.355363 31139 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.355742 29072 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
W20260709 13:04:29.355989 31007 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.001s
W20260709 13:04:29.260121 29039 connection.cc:256] RPC call timeout handler was delayed by -1.0084739760004595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
I20260709 13:04:29.356276 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.357072 29216 raft_consensus.cc:2410] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:29.300666 31153 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.357754 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.358749 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:29.359359 29216 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:29.359617 31153 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:29.269577 31009 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.360060 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.360620 29216 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:29.360975 30922 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:29.319263 31183 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:29.361346 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:29.361624 31068 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.361963 29216 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.362370 31183 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.363338 31055 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:29.364476 29871 leader_election.cc:336] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.226s (SENT)
W20260709 13:04:29.318866 29907 rpcz_store.cc:269] 0709 13:04:27.005285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.773109 (+1767824us) service_pool.cc:224] Handling call
0709 13:04:29.239852 (+466743us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.366707 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:29.367463 29907 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:29.368717 30985 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:29.369174 30985 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.369390 29216 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
W20260709 13:04:29.289291 29038 connection.cc:256] RPC call timeout handler was delayed by -3.4842033880004237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
I20260709 13:04:29.291102 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:29.263478 30870 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.369794 31030 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.370528 31030 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:29.370343 30870 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.370971 31030 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.371330 31073 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:29.300603 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1490) took 2202 ms. Trace:
I20260709 13:04:29.372503 30870 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.297974 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1569) took 2277 ms. Trace:
I20260709 13:04:29.372074 29142 rpcz_store.cc:276] 0709 13:04:27.097920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.541830 (+443910us) service_pool.cc:224] Handling call
0709 13:04:27.552934 (+ 11104us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:27.579570 (+ 26636us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:04:27.742492 (+162922us) log.cc:844] Serialized 14 byte log entry
0709 13:04:28.812997 (+1070505us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:28.850020 (+ 37023us) log.cc:844] Serialized 527 byte log entry
0709 13:04:28.850179 (+   159us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:29.147033 (+296854us) log.cc:844] Serialized 14 byte log entry
0709 13:04:29.192253 (+ 45220us) log.cc:844] Serialized 14 byte log entry
0709 13:04:29.192433 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.192500 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.300362 (+107862us) raft_consensus.cc:1685] finished
0709 13:04:29.300385 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.300535 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61956,"prepare.queue_time_us":12720,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":100437,"raft.run_cpu_time_us":573,"raft.run_wall_time_us":40587,"thread_start_us":997175,"threads_started":1,"wal-append.queue_time_us":1102521}
I20260709 13:04:29.227957 30929 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.373262 29071 rpcz_store.cc:276] 0709 13:04:27.020615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.023567 (+  2952us) service_pool.cc:224] Handling call
0709 13:04:27.023988 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.078566 (+ 54578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.078599 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.079162 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.633428 (+1554266us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:28.787265 (+153837us) log.cc:844] Serialized 14 byte log entry
0709 13:04:28.787514 (+   249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.787611 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.297721 (+510110us) raft_consensus.cc:1685] finished
0709 13:04:29.297741 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.297886 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54144,"prepare.queue_time_us":19433,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":6320,"raft.queue_time_us":65294,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":8748,"thread_start_us":1707297,"threads_started":2,"wal-append.queue_time_us":2058138}
I20260709 13:04:29.262141 30692 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.263870 30993 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.214390 31138 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.216730 31162 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:29.369971 29038 connection.cc:256] RPC call timeout handler was delayed by -3.45720338800038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260709 13:04:29.325663 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1497) took 1785 ms. Trace:
W20260709 13:04:29.398566 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 45863.4us
I20260709 13:04:29.412535 29067 rpcz_store.cc:276] 0709 13:04:27.540274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.237419 (+1697145us) service_pool.cc:224] Handling call
0709 13:04:29.325588 (+ 88169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.263514 31173 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:29.259899 31176 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:04:29.317463 31168 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:04:29.384788 30692 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.339866 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:29.259037 29217 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:29.344575 29214 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:29.347595 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.324571 31190 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:04:29.346303 31043 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:29.369475 30758 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:29.371426 31030 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.296208 31099 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.263270 30999 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:29.410512 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1495) took 3339 ms. Trace:
I20260709 13:04:29.383896 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:29.477579 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:29.478164 29071 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.376747 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1488) took 2351 ms. Trace:
W20260709 13:04:29.376546 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.413610 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:29.376331 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.481546 29216 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:29.355376 30898 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.320505 29219 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:29.391309 29070 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
W20260709 13:04:29.398885 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 46478.6us
I20260709 13:04:29.338042 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:29.361435 31009 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:29.428309 29221 raft_consensus.cc:2410] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:04:29.440834 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 161255us
I20260709 13:04:29.469352 31068 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:29.462720 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 122946us
I20260709 13:04:29.494760 31049 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:29.495421 31191 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.468967 31043 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:29.368734 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:29.506014 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:29.506572 29146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 226) took 2495 ms (client timeout 1507 ms). Trace:
I20260709 13:04:29.506764 29216 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:29.466322 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:29.507062 29146 rpcz_store.cc:269] 0709 13:04:27.010759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.019783 (+  9024us) service_pool.cc:224] Handling call
0709 13:04:29.506325 (+2486542us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54002e8954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:29.506500 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435624832}
I20260709 13:04:29.507766 29067 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
W20260709 13:04:29.508008 29906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 308) took 2509 ms (client timeout 2353 ms). Trace:
I20260709 13:04:29.369550 30985 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:29.508527 29906 rpcz_store.cc:269] 0709 13:04:26.998996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.731192 (+1732196us) service_pool.cc:224] Handling call
0709 13:04:29.507756 (+776564us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54006ffc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:29.507910 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230480256}
I20260709 13:04:29.509769 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:29.351042 30963 consensus_meta.cc:287] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 1.770s	user 0.009s	sys 0.004s
I20260709 13:04:29.511133 30963 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:29.384370 30929 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:29.511688 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:29.512202 31150 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.665s	user 0.011s	sys 0.000s
I20260709 13:04:29.512322 29909 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:29.476070 30999 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:29.482713 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:29.513151 29217 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.513756 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.506758 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:29.515273 29907 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:29.515688 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.516153 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:29.516551 29068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1498) took 1937 ms (client timeout 1964 ms). Trace:
I20260709 13:04:29.516829 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 411) took 2532 ms. Trace:
I20260709 13:04:29.517988 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.518332 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.518855 29067 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:29.519356 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:29.519625 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:29.519822 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 396) took 2606 ms. Trace:
I20260709 13:04:29.383785 30943 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:29.520884 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.514408 29215 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:29.510728 29143 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:04:29.521893 29143 pending_rounds.cc:85] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 5
I20260709 13:04:29.522164 29143 pending_rounds.cc:107] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:04:29.522418 29143 raft_consensus.cc:2889] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:29.520249 29908 rpcz_store.cc:276] 0709 13:04:26.913425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.913605 (+   180us) service_pool.cc:224] Handling call
0709 13:04:26.913902 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.003344 (+ 89442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.003390 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:27.003444 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:27.003488 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.519598 (+2516110us) spinlock_profiling.cc:243] Waited 89.2 ms on lock 0x7b54005bc554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:29.519646 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.519765 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205233408}
I20260709 13:04:29.491294 30758 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:04:29.434224 29112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.026s	sys 0.026s
W20260709 13:04:29.524259 29112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.028s	sys 0.032s
I20260709 13:04:29.524397 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:29.525141 29066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1576) took 2502 ms (client timeout 2296 ms). Trace:
W20260709 13:04:29.525534 29066 rpcz_store.cc:269] 0709 13:04:27.022914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.625873 (+602959us) service_pool.cc:224] Handling call
0709 13:04:29.524973 (+1899100us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540012a054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:29.525075 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2691762944}
I20260709 13:04:29.526665 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.527448 29066 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:29.469919 31094 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:29.529044 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:29.529779 29072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1582) took 2503 ms (client timeout 3331 ms). Trace:
I20260709 13:04:29.529955 29904 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:04:29.530176 29072 rpcz_store.cc:269] 0709 13:04:27.025894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.174378 (+2148484us) service_pool.cc:224] Handling call
0709 13:04:29.529623 (+355245us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400068bd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:29.529722 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406917888}
I20260709 13:04:29.532117 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:29.532295 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1502) took 2034 ms. Trace:
I20260709 13:04:29.532712 29148 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:29.479028 29140 rpcz_store.cc:276] 0709 13:04:27.025195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.491365 (+466170us) service_pool.cc:224] Handling call
0709 13:04:27.491752 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.541005 (+ 49253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.541045 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.581471 (+ 40426us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.186102 (+604631us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:28.186170 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.186418 (+   248us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.376483 (+1190065us) raft_consensus.cc:1685] finished
0709 13:04:29.376505 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.376669 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48444,"raft.queue_time_us":48728,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":7969,"thread_start_us":473768,"threads_started":1,"wal-append.queue_time_us":1181329}
I20260709 13:04:29.532783 29143 rpcz_store.cc:276] 0709 13:04:27.489253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.560770 (+ 71517us) service_pool.cc:224] Handling call
0709 13:04:29.510378 (+1949608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.522842 (+ 12464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.523745 (+   903us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b54002e8950. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:29.523868 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4482832256}
I20260709 13:04:29.469660 30692 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:29.493399 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.408743 31007 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 13:04:29.463858 29213 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:29.503448 30978 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.384635 30993 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.509938 31036 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.510710 29906 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:29.350104 30984 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:29.512503 29910 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:29.513422 30999 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.517271 29905 rpcz_store.cc:276] 0709 13:04:26.983939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.001272 (+ 17333us) service_pool.cc:224] Handling call
0709 13:04:27.064569 (+ 63297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.311290 (+246721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.311360 (+    70us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:27.311445 (+    85us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:27.311515 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.516549 (+2205034us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54003453d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:29.516601 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.516736 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475199232}
I20260709 13:04:29.520680 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:29.521494 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.534255 29070 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:29.485796 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:29.433027 31073 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.495896 31191 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:29.534962 29071 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:29.470316 31176 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:29.535697 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:29.532408 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:29.476540 29073 rpcz_store.cc:276] 0709 13:04:26.071205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.933506 (+862301us) service_pool.cc:224] Handling call
0709 13:04:26.988704 (+ 55198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.013453 (+ 24749us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.013484 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.017365 (+  3881us) log.cc:844] Serialized 26 byte log entry
0709 13:04:29.163632 (+2146267us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b18003878e0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:04:29.270975 (+107343us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:29.271014 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.271742 (+   728us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.384962 (+113220us) raft_consensus.cc:1685] finished
0709 13:04:29.384992 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.410437 (+ 25445us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24390,"raft.queue_time_us":24752,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":15331,"spinlock_wait_cycles":5399424,"thread_start_us":107132,"threads_started":1,"wal-append.queue_time_us":2344950}
W20260709 13:04:29.516993 29068 rpcz_store.cc:269] 0709 13:04:27.579316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.681586 (+102270us) service_pool.cc:224] Handling call
0709 13:04:29.515522 (+1833936us) spinlock_profiling.cc:243] Waited 9.21 ms on lock 0x7b5800005198. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:29.516479 (+   957us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21193472}
W20260709 13:04:29.486363 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 72856.7us
W20260709 13:04:29.413772 29038 connection.cc:256] RPC call timeout handler was delayed by -3.4284033880003335s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17450000000000002s for a response
I20260709 13:04:29.539192 29217 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
W20260709 13:04:29.539268 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 133867us
I20260709 13:04:29.512691 31150 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.674193 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:29.678099 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:29.720901 29147 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:29.592722 29222 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:29.722195 29222 pending_rounds.cc:85] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 1
I20260709 13:04:29.638968 29143 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:04:29.639335 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.723373 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:29.723881 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:29.724215 29140 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.492830 31174 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:29.651703 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:29.721722 29038 connection.cc:256] RPC call timeout handler was delayed by -3.3456033880001996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
W20260709 13:04:29.644044 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:29.603700 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.719837 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:29.724032 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 174) took 1070 ms. Trace:
I20260709 13:04:29.725553 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1496) took 2226 ms. Trace:
W20260709 13:04:29.722920 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 187281us
I20260709 13:04:29.722711 29222 pending_rounds.cc:107] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:29.701522 31013 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.547060 31107 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:29.547837 31111 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 2.530s	user 0.012s	sys 0.002s
I20260709 13:04:29.757561 30942 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:29.758054 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1573) took 2764 ms. Trace:
I20260709 13:04:29.758586 29214 rpcz_store.cc:276] 0709 13:04:26.993380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.757120 (+2763740us) service_pool.cc:224] Handling call
0709 13:04:29.757483 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.757687 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.757712 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.757758 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:29.757796 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.757856 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.757990 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.759842 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.762579 29214 raft_consensus.cc:2410] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:29.758003 31111 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.760883 29217 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:04:29.769181 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1575) took 2775 ms. Trace:
I20260709 13:04:29.769327 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.769438 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.769726 29217 rpcz_store.cc:276] 0709 13:04:26.994084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.760229 (+2766145us) service_pool.cc:224] Handling call
0709 13:04:29.760600 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.768853 (+  8253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.769056 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:29.761498 29038 connection.cc:256] RPC call timeout handler was delayed by -3.2943033880001167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1894s for a response
W20260709 13:04:29.719708 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 233977us
I20260709 13:04:29.772282 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.772993 29066 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:29.773442 31150 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.757226 31193 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:29.773954 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.774235 31193 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.774716 29069 raft_consensus.cc:2368] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3.
I20260709 13:04:29.775414 29908 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:29.774660 31193 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:29.777570 29144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1798) took 2790 ms (client timeout 1850 ms). Trace:
W20260709 13:04:29.775753 29069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1505) took 1690 ms (client timeout 2160 ms). Trace:
W20260709 13:04:29.778425 29069 rpcz_store.cc:269] 0709 13:04:28.085667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.124847 (+1039180us) service_pool.cc:224] Handling call
0709 13:04:29.775557 (+650710us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b54001d9054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:29.775700 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1491401600}
I20260709 13:04:29.546192 31134 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.779325 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1571) took 2758 ms. Trace:
I20260709 13:04:29.779579 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.763110 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:04:29.771335 29038 connection.cc:256] RPC call timeout handler was delayed by -3.2682033880000745s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
I20260709 13:04:29.780224 29069 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
W20260709 13:04:29.780632 29038 connection.cc:256] RPC call timeout handler was delayed by -2.111137982999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260709 13:04:29.545910 31144 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.712s	user 0.008s	sys 0.001s
I20260709 13:04:29.771278 29070 rpcz_store.cc:276] 0709 13:04:27.498955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.493142 (+1994187us) service_pool.cc:224] Handling call
0709 13:04:29.725492 (+232350us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:29.541061 31120 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.865s	user 0.013s	sys 0.000s
I20260709 13:04:29.778671 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.783933 29214 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:29.784037 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:29.591027 30929 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:29.776278 29222 raft_consensus.cc:2889] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:29.766825 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.779812 29064 rpcz_store.cc:276] 0709 13:04:27.021357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.021490 (+   133us) service_pool.cc:224] Handling call
0709 13:04:27.068349 (+ 46859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.447525 (+379176us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:27.669234 (+221709us) log.cc:844] Serialized 14 byte log entry
0709 13:04:28.793567 (+1124333us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:28.793624 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:28.793669 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.779112 (+985443us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54002080d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:29.779160 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.779275 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79954,"prepare.queue_time_us":822834,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":20288,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":2928,"spinlock_wait_cycles":871291648,"thread_start_us":1319454,"threads_started":2,"wal-append.queue_time_us":1146321}
I20260709 13:04:29.770895 29071 rpcz_store.cc:276] 0709 13:04:28.654059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.477326 (+823267us) service_pool.cc:224] Handling call
0709 13:04:29.723977 (+246651us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:29.778023 29144 rpcz_store.cc:269] 0709 13:04:26.987404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.064301 (+ 76897us) service_pool.cc:224] Handling call
0709 13:04:29.777392 (+2713091us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b54003a57d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:29.777502 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599174528}
I20260709 13:04:29.517380 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:29.780990 29038 connection.cc:256] RPC call timeout handler was delayed by -1.99053798299974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
I20260709 13:04:29.777117 31174 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.789044 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.779691 30999 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.547853 31138 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:29.758208 30942 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:29.762624 31033 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:29.716104 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:29.796170 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.812043 29069 raft_consensus.cc:2376] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5. Re-sending same reply.
I20260709 13:04:29.812637 29064 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:29.813239 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1502) took 2133 ms. Trace:
I20260709 13:04:29.813323 29069 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:04:29.813949 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1507) took 1345 ms. Trace:
I20260709 13:04:29.814015 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:29.815047 31120 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.814513 29069 rpcz_store.cc:276] 0709 13:04:28.469073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.812695 (+1343622us) service_pool.cc:224] Handling call
0709 13:04:29.813012 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.813731 (+   719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.813897 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.794638 30929 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:29.796754 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:29.801712 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 81969.3us
I20260709 13:04:29.816215 29910 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:29.816406 29112 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:29.802138 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:29.813627 29064 rpcz_store.cc:276] 0709 13:04:27.679844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.808384 (+2128540us) service_pool.cc:224] Handling call
0709 13:04:29.812417 (+  4033us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.813023 (+   606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.813190 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.801730 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.818631 29217 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:29.819024 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:29.801841 31009 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.548643 30996 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:29.803730 29038 connection.cc:256] RPC call timeout handler was delayed by -1.8726379829995494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 13:04:29.820932 29038 connection.cc:256] RPC call timeout handler was delayed by -1.7376379829993311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
I20260709 13:04:29.784876 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.726478 31191 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:29.775245 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.543056 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:29.838778 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:29.789136 31020 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.811743 31033 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.761279 31073 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.825201 30978 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.846213 31073 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:29.810132 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1572) took 2817 ms. Trace:
I20260709 13:04:29.846571 30978 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:29.795673 31119 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.847208 30978 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:29.822777 29065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1545) took 5013 ms (client timeout 1699 ms). Trace:
W20260709 13:04:29.848115 29065 rpcz_store.cc:269] 0709 13:04:24.803077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.803345 (+   268us) service_pool.cc:224] Handling call
0709 13:04:29.816548 (+5013203us) spinlock_profiling.cc:243] Waited 2.73 s on lock 0x7b54000686d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:29.816652 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6272404224}
I20260709 13:04:29.849380 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.850634 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.851332 29065 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:29.821641 29070 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:29.851959 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1578) took 2828 ms. Trace:
I20260709 13:04:29.852320 29070 rpcz_store.cc:276] 0709 13:04:27.024813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.795394 (+2770581us) service_pool.cc:224] Handling call
0709 13:04:29.821345 (+ 25951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.851689 (+ 30344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.851884 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.822300 31180 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.807281 31174 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:29.823935 29214 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:29.860808 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.795159 31144 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.698886 31043 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.822448 31107 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:04:29.819585 31120 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.808436 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1509) took 1258 ms. Trace:
I20260709 13:04:29.840034 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 179) took 1041 ms. Trace:
I20260709 13:04:29.594627 31159 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.868824 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.821558 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:29.822539 31173 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:29.846858 29222 rpcz_store.cc:276] 0709 13:04:26.993153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.549659 (+2556506us) service_pool.cc:224] Handling call
0709 13:04:29.592317 (+ 42658us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.805501 (+213184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.810059 (+  4558us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.837447 30963 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:29.822345 31009 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.807670 30984 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.861961 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:29.823408 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:29.863466 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.839051 31189 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.866223 29073 rpcz_store.cc:276] 0709 13:04:28.549813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.770435 (+1220622us) service_pool.cc:224] Handling call
0709 13:04:29.808376 (+ 37941us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.881444 29214 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.866325 29064 rpcz_store.cc:276] 0709 13:04:28.799149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.817911 (+1018762us) service_pool.cc:224] Handling call
0709 13:04:29.839495 (+ 21584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.839666 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.839693 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:29.839732 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:29.839765 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.839836 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.839970 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.809770 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:29.853710 31033 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.847523 31119 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.891721 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:29.847734 31145 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.846199 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:29.857059 30996 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.858317 31180 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.906441 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.907371 29073 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
W20260709 13:04:29.876717 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55234.6us
I20260709 13:04:29.871541 31120 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:29.901827 29910 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:29.902931 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:29.909731 29222 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:29.911185 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.852368 31153 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:04:29.908473 29038 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:29.877995 31068 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:29.869570 31190 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:04:29.919842 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.840890 31020 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.838060 31192 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:04:29.889114 31101 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:29.910647 30996 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.942626 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:29.906903 31162 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:29.955379 31159 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:29.914450 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:29.924311 29064 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:29.982321 29070 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:29.945209 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:30.210719 29215 raft_consensus.cc:2376] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4. Re-sending same reply.
W20260709 13:04:30.211444 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.211845 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:30.212955 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:29.982649 29140 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:29.942333 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:30.213732 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:29.931653 29903 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:30.215516 30984 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:29.890127 31009 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.216183 29070 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:30.216852 30984 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.217356 30984 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.216866 31193 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.218411 29070 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:04:29.902242 29905 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:30.215539 29064 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:30.219445 30984 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:30.219666 31193 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.220417 31193 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.220898 31193 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.221431 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:30.221947 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:30.222234 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 178) took 1454 ms. Trace:
I20260709 13:04:30.025168 29214 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:29.910907 31180 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.222910 29906 raft_consensus.cc:2410] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:30.223286 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:30.223629 31198 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.227s	user 0.002s	sys 0.016s
I20260709 13:04:30.224407 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 317) took 1272 ms. Trace:
I20260709 13:04:30.224758 31101 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:30.224879 29908 rpcz_store.cc:276] 0709 13:04:28.952277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.524172 (+571895us) service_pool.cc:224] Handling call
0709 13:04:30.224210 (+700038us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54005e92d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:30.224351 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575400704}
I20260709 13:04:30.225764 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1499) took 2609 ms. Trace:
W20260709 13:04:30.226321 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 284060us
I20260709 13:04:30.226548 29908 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:30.226649 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.227188 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 404) took 2492 ms. Trace:
I20260709 13:04:29.983644 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:30.227948 31144 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.228221 31073 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:30.228457 29904 raft_consensus.cc:2368] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:30.228698 31073 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.229060 31073 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.229161 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:30.229619 29904 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:30.230279 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 415) took 1594 ms. Trace:
W20260709 13:04:30.230512 29909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 309) took 3225 ms (client timeout 3188 ms). Trace:
I20260709 13:04:30.230588 29904 rpcz_store.cc:276] 0709 13:04:28.635844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.229113 (+1593269us) service_pool.cc:224] Handling call
0709 13:04:30.229442 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.230043 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.230222 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:29.941087 31154 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:29.987289 31190 consensus_queue.cc:1243] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:04:30.232872 29219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 240) took 1280 ms (client timeout 1682 ms). Trace:
I20260709 13:04:30.066751 31111 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:30.233466 29219 rpcz_store.cc:269] 0709 13:04:28.952440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.124660 (+172220us) service_pool.cc:224] Handling call
0709 13:04:30.232649 (+1107989us) spinlock_profiling.cc:243] Waited 8.01 ms on lock 0x7b54003a9dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:30.232795 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18423680}
I20260709 13:04:29.988665 30985 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.930400 31030 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.939519 31120 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:29.840132 31176 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:29.922787 31173 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:29.938258 30922 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:04:30.209206 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.236344 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:29.876083 30942 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.000041 29907 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:29.949810 31135 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.939239 31036 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:29.977795 31183 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:04:30.210608 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 296234us
I20260709 13:04:30.235754 31112 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:29.922389 29217 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:30.254824 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:30.255766 29217 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:29.795450 31134 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.214848 30893 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:29.988041 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:29.890949 31189 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:30.231317 29909 rpcz_store.cc:269] 0709 13:04:27.004984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.136633 (+2131649us) service_pool.cc:224] Handling call
0709 13:04:30.230342 (+1093709us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b5400768c54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:30.230435 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":834851584}
I20260709 13:04:30.235137 29904 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:30.222643 29072 rpcz_store.cc:276] 0709 13:04:28.768166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.221201 (+1453035us) service_pool.cc:224] Handling call
0709 13:04:30.222176 (+   975us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.235682 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1589) took 1137 ms. Trace:
I20260709 13:04:30.227483 29908 rpcz_store.cc:276] 0709 13:04:27.734378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.225939 (+2491561us) service_pool.cc:224] Handling call
0709 13:04:30.226233 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.226966 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.227139 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.226158 29071 rpcz_store.cc:276] 0709 13:04:27.616855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.801758 (+2184903us) service_pool.cc:224] Handling call
0709 13:04:29.802109 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.804290 (+  2181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.804322 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:29.804743 (+   421us) log.cc:844] Serialized 26 byte log entry
0709 13:04:29.874494 (+ 69751us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:29.874543 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.874632 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.225569 (+350937us) raft_consensus.cc:1685] finished
0709 13:04:30.225589 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.225711 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1751,"raft.queue_time_us":1867,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3227,"thread_start_us":49699,"threads_started":1,"wal-append.queue_time_us":88775}
I20260709 13:04:30.230540 29215 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:30.239337 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.240044 29148 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 4. Ops: []
W20260709 13:04:30.245886 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.248557 30866 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:30.248477 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:30.244186 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.237818 30692 consensus_meta.cc:287] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.623s	user 0.012s	sys 0.000s
I20260709 13:04:30.251088 31107 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:30.252763 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:30.213450 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.258672 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.242734 31173 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.266031 31145 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.266937 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.245085 30922 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:30.251085 31036 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:30.397272 31036 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.398305 31099 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:30.252686 31135 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.398903 31099 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.398679 31036 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:30.405678 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 152991us
I20260709 13:04:30.408305 29908 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:30.408989 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 318) took 1354 ms. Trace:
I20260709 13:04:30.409371 29908 rpcz_store.cc:276] 0709 13:04:29.055065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.407625 (+1352560us) service_pool.cc:224] Handling call
0709 13:04:30.407998 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.408724 (+   726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.408929 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.301985 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.357909 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.370563 29146 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:30.402163 29222 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:30.411866 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:30.374331 29112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.009s	sys 0.008s
I20260709 13:04:30.266240 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:30.412577 29112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.010s	sys 0.009s
I20260709 13:04:30.413030 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.413230 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.414548 29219 rpcz_store.cc:276] 0709 13:04:29.098359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.234725 (+1136366us) service_pool.cc:224] Handling call
0709 13:04:30.235073 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.235301 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.235340 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:30.235398 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:30.235444 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.235498 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.235613 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.415354 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:30.415747 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:30.416350 29219 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.415961 29222 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:30.416640 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.414296 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.417887 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:30.418972 31145 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:30.419929 31145 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:30.421177 31145 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:30.423094 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.423686 29219 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:30.411733 29186 leader_election.cc:400] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:30.425460 29186 leader_election.cc:403] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:04:30.424798 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:30.427488 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1591) took 1277 ms. Trace:
I20260709 13:04:30.429333 29219 rpcz_store.cc:276] 0709 13:04:29.149900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.426761 (+1276861us) service_pool.cc:224] Handling call
0709 13:04:30.427032 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.427212 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.427235 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:30.427270 (+    35us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:30.427304 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.427348 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.427443 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.425710 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:30.430789 29222 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:30.431246 31202 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:30.431629 31202 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.432024 31154 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:30.432250 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1818) took 1156 ms. Trace:
I20260709 13:04:30.432803 31202 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:30.403676 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.434600 29142 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:30.435696 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.436826 29219 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 5. Ops: [3.6-3.6]
I20260709 13:04:30.433247 29146 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:30.279040 31183 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.399241 30922 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.436713 29142 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:30.454327 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.454895 29142 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.439684 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.440388 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1588) took 1345 ms. Trace:
I20260709 13:04:30.432618 29222 rpcz_store.cc:276] 0709 13:04:29.275482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.431483 (+1156001us) service_pool.cc:224] Handling call
0709 13:04:30.431762 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.431950 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.431976 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:30.432014 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:30.432046 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.432101 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.432197 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.449468 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.437942 30984 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:04:30.433414 29038 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:30.411609 29141 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:30.456017 29215 rpcz_store.cc:276] 0709 13:04:29.095214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.230127 (+1134913us) service_pool.cc:224] Handling call
0709 13:04:30.230385 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.440147 (+209762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.440311 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.456753 29143 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:30.253723 31112 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:30.267755 31193 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.458464 29216 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:30.513887 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1819) took 1238 ms. Trace:
I20260709 13:04:30.513672 31192 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:30.514689 29216 rpcz_store.cc:276] 0709 13:04:29.275790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.457331 (+1181541us) service_pool.cc:224] Handling call
0709 13:04:30.458265 (+   934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.513645 (+ 55380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.513834 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.516017 31135 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.516207 29216 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:30.516966 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1822) took 1239 ms. Trace:
I20260709 13:04:30.435496 31111 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:30.478788 29142 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:30.588610 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.517267 29216 rpcz_store.cc:276] 0709 13:04:29.278037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.515543 (+1237506us) service_pool.cc:224] Handling call
0709 13:04:30.515873 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.516739 (+   866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.516916 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.481823 29140 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:30.586334 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 245) took 1247 ms. Trace:
I20260709 13:04:30.592154 29222 rpcz_store.cc:276] 0709 13:04:29.339305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.585261 (+1245956us) service_pool.cc:224] Handling call
0709 13:04:30.585662 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.585899 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.585937 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:30.585993 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:30.586056 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.586129 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.586263 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.524653 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.406492 31173 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.590204 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:30.512951 30999 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.701s	user 0.008s	sys 0.004s
I20260709 13:04:30.455996 29066 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:30.596925 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.345747 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 418) took 1169 ms. Trace:
I20260709 13:04:30.570932 31120 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:30.570012 31072 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:30.598606 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.601269 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.569751 31043 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:30.436939 31107 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:30.600485 31191 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.757s	user 0.006s	sys 0.006s
I20260709 13:04:30.604646 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:30.593752 31174 consensus_meta.cc:287] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.718s	user 0.003s	sys 0.010s
I20260709 13:04:30.411564 30692 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.520190 30993 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.590831 30996 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.495183 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 417) took 1702 ms. Trace:
W20260709 13:04:30.601827 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 173276us
I20260709 13:04:30.643621 29216 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:30.644419 29143 raft_consensus.cc:2410] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:30.644809 29142 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:30.669792 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.671077 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.645046 31030 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.646028 31112 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.601670 31183 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:30.601150 30978 consensus_meta.cc:287] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.753s	user 0.016s	sys 0.003s
I20260709 13:04:30.664750 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.665226 31174 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.660068 31107 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:30.673100 29911 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 414) took 2037 ms (client timeout 2148 ms). Trace:
W20260709 13:04:30.680591 29911 rpcz_store.cc:269] 0709 13:04:28.635598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.169731 (+534133us) service_pool.cc:224] Handling call
0709 13:04:30.672904 (+1503173us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400499b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:30.673029 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288585088}
I20260709 13:04:30.651383 31009 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.678715 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:30.605865 31033 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.701s	user 0.013s	sys 0.000s
I20260709 13:04:30.602360 29907 rpcz_store.cc:276] 0709 13:04:29.135975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.845809 (+709834us) service_pool.cc:224] Handling call
0709 13:04:30.304793 (+458984us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.613592 31111 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:30.655884 31099 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.684187 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.687062 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.687546 29905 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:30.688490 31084 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.683147 31112 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.693761 29215 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:04:30.694625 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:30.665282 31191 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.698125 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.699009 29144 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:30.673660 31189 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.637375 31180 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:30.684777 31209 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 13:04:30.706190 29068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 186) took 1512 ms (client timeout 1949 ms). Trace:
I20260709 13:04:30.703596 29141 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
W20260709 13:04:30.706686 29068 rpcz_store.cc:269] 0709 13:04:29.194243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.724689 (+530446us) service_pool.cc:224] Handling call
0709 13:04:30.694449 (+969760us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b5800005198. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:30.706127 (+ 11678us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13903744}
I20260709 13:04:30.703743 29145 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:30.704583 29215 pending_rounds.cc:85] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 2
I20260709 13:04:30.707073 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.707712 29215 pending_rounds.cc:107] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:04:30.708066 29215 raft_consensus.cc:2889] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:30.709025 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:30.709257 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.705451 29142 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:30.712236 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.668454 29904 rpcz_store.cc:276] 0709 13:04:28.793050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.231898 (+1438848us) service_pool.cc:224] Handling call
0709 13:04:30.234921 (+  3023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.434973 (+200052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.482883 (+ 47910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:30.714661 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:30.715552 29143 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:30.602382 31134 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:30.716238 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.716624 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.717186 29143 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:30.723865 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 291845us
I20260709 13:04:30.714572 31084 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:30.681990 30978 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.726514 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:30.727960 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:30.728724 31020 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:30.729178 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:30.733976 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.735097 30978 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:30.728031 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1579) took 3732 ms. Trace:
I20260709 13:04:30.725816 29038 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:30.737622 29220 rpcz_store.cc:276] 0709 13:04:26.995136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.228466 (+3233330us) service_pool.cc:224] Handling call
0709 13:04:30.228774 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.229089 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.229117 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:30.229600 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:04:30.401297 (+171697us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:30.401339 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.401404 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.727747 (+326343us) raft_consensus.cc:1685] finished
0709 13:04:30.727773 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.727960 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7,"raft.queue_time_us":3104,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":171068,"threads_started":1,"wal-append.queue_time_us":320947}
I20260709 13:04:30.739060 29911 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 3: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:30.720544 31189 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:30.703570 31033 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.748396 29143 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:30.749168 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:30.749563 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.749881 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:30.750102 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:30.750682 29143 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:30.751487 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.752244 29143 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:30.722499 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:30.752892 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.729562 31073 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.755105 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:30.753782 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.763214 29146 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:30.747169 31212 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:30.764732 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:30.752168 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:30.767617 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.756145 30978 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:30.763906 30852 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:30.771167 30870 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:30.757073 31024 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:30.771962 29909 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:30.772415 31174 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.772995 30692 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.774273 31017 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:30.774432 31033 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:30.777105 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.754401 31101 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:04:30.794508 29904 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:30.810248 29142 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:30.811892 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:30.813577 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:30.814549 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.814563 29142 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:30.816246 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.790987 31119 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:30.793885 31107 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.756417 31111 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:30.859279 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.757778 29907 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:30.754348 30993 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:04:30.799125 30852 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:04:30.807154 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 65948.7us
W20260709 13:04:30.890280 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.891007 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:30.785130 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1592) took 1608 ms. Trace:
I20260709 13:04:30.817838 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.891871 29219 rpcz_store.cc:276] 0709 13:04:29.176330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.430433 (+1254103us) service_pool.cc:224] Handling call
0709 13:04:30.436602 (+  6169us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.437411 (+   809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.784908 (+347497us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b54000bde50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:30.785065 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13491584}
I20260709 13:04:30.892506 29142 raft_consensus.cc:2376] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2. Re-sending same reply.
W20260709 13:04:30.827190 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.835112 31145 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:30.836068 31017 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:30.836813 31191 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.896523 31017 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.897668 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:30.898494 29907 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:30.873397 29141 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
W20260709 13:04:30.878684 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.899617 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:30.878592 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:30.900897 29907 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.901362 29221 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 3: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:30.869899 29067 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:30.903251 29217 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:04:30.887594 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 122929us
W20260709 13:04:30.888021 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.889424 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:30.893356 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.808727 30978 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:30.837589 31099 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.895361 31145 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:30.896302 30852 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.877573 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:30.779314 31173 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:30.915179 31101 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:30.915784 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:30.916015 30866 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:30.916515 29064 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:30.917133 30866 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:30.917521 30866 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:30.917723 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:30.918314 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:30.919025 29905 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:30.919240 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:30.919459 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.919938 29070 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:30.920110 29146 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:30.885787 31111 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.910920 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:30.912115 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:30.913666 31112 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:30.914115 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:30.916296 31173 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:30.907140 29068 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:30.920306 31219 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:30.920862 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:30.921648 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:30.925408 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:30.809775 31202 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:30.922729 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.908555 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.929332 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.918416 29064 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:30.856284 31180 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:30.926137 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:30.926445 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:30.926934 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:30.905074 31020 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.909686 31191 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:30.929956 29142 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:30.941249 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.942272 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.942071 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:30.943682 31043 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:30.925362 31111 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:30.943858 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:30.944615 31043 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:30.944633 29068 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:30.924376 30993 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:30.945441 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.927202 31119 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:30.948148 29146 raft_consensus.cc:2368] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:30.947947 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:30.912699 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:30.925621 31033 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:30.925508 31049 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:30.948938 28969 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 160) took 9107 ms. Trace:
I20260709 13:04:30.937562 30852 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.937033 31174 raft_consensus.cc:2764] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:04:30.949163 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:30.946261 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.954480 29139 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:30.954402 31030 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.918314 30866 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.929770 31211 raft_consensus.cc:2764] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:04:30.926270 31219 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:30.949059 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:30.953892 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:30.959048 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:30.940090 31101 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:30.930140 31099 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:30.955065 31073 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:30.967468 31030 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.975620 31101 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.975823 30866 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:30.976123 31101 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.977540 31030 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:30.945520 31162 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:04:30.957906 28969 rpcz_store.cc:276] 0709 13:04:21.842080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.330700 (+488620us) service_pool.cc:224] Handling call
0709 13:04:30.948861 (+8618161us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:27.860496 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:27.861183 (+   687us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:27.861226 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:04:27.861248 (+    22us) tablet.cc:662] Decoding operations
0709 13:04:28.001457 (+140209us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:28.001547 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:28.001597 (+    50us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:04:28.002496 (+   899us) tablet.cc:701] Row locks acquired
0709 13:04:28.002526 (+    30us) write_op.cc:260] PREPARE: finished
0709 13:04:28.093419 (+ 90893us) write_op.cc:270] Start()
0709 13:04:28.179045 (+ 85626us) write_op.cc:276] Timestamp: P: 1783602268088240 usec, L: 0
0709 13:04:28.207994 (+ 28949us) op_driver.cc:348] REPLICATION: starting
0709 13:04:28.694498 (+486504us) log.cc:844] Serialized 2801 byte log entry
0709 13:04:29.595281 (+900783us) op_driver.cc:464] REPLICATION: finished
0709 13:04:29.837577 (+242296us) write_op.cc:301] APPLY: starting
0709 13:04:29.837664 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:29.837861 (+   197us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:29.840662 (+  2801us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:29.890876 (+ 50214us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:29.932850 (+ 41974us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:04:29.932899 (+    49us) write_op.cc:312] APPLY: finished
0709 13:04:29.933173 (+   274us) log.cc:844] Serialized 59 byte log entry
0709 13:04:29.933342 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:29.933463 (+   121us) write_op.cc:454] Released schema lock
0709 13:04:29.933506 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:30.399944 (+466438us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b5400018954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b08e28 00007f783a543581 00007f783a542d58 00007f783a5479f1 00007f783a5479a9 00007f783a547939 00007f783a547901 00007f783a546b2d 000000000038f354 000000000038f199
0709 13:04:30.400267 (+   323us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:30.400333 (+    66us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":197098,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":242030,"apply.run_cpu_time_us":3634,"apply.run_wall_time_us":96284,"num_ops":7,"prepare.queue_time_us":830910,"prepare.run_cpu_time_us":30669,"prepare.run_wall_time_us":2539666,"raft.queue_time_us":262471,"raft.run_cpu_time_us":27908,"raft.run_wall_time_us":215421,"replication_time_us":1387124,"spinlock_wait_cycles":93905280,"thread_start_us":1237283,"threads_started":4,"wal-append.queue_time_us":533719,"wal-append.run_cpu_time_us":32719,"wal-append.run_wall_time_us":1699651}]]}
I20260709 13:04:30.968994 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:30.979899 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:30.969094 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.940313 31189 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:04:30.981921 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:30.956175 31119 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.980265 29142 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.957456 31043 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:31.001173 31043 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.001274 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:31.001577 31043 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.002102 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 410) took 1472 ms. Trace:
I20260709 13:04:31.002475 29910 rpcz_store.cc:276] 0709 13:04:29.529852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.920599 (+390747us) service_pool.cc:224] Handling call
0709 13:04:31.001128 (+1080529us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:31.002041 (+   913us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114444544}
I20260709 13:04:31.003502 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:30.986078 30929 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.006271 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:31.006898 29147 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:31.007788 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:31.008456 29147 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.008901 31009 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:31.009366 29910 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:30.981415 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.010525 29071 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.010643 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.011415 29064 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:31.011722 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:30.974185 29068 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:30.945205 31020 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.979226 31099 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:30.982458 31073 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:30.986063 31202 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:30.987238 29221 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:31.019341 29221 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:30.978744 31145 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:30.965227 29066 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:30.992280 29908 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:30.992168 29906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 327) took 1461 ms (client timeout 1910 ms). Trace:
I20260709 13:04:30.993108 29142 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:30.993242 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:30.983791 31049 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:30.997073 31201 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.596s	user 0.003s	sys 0.000s
I20260709 13:04:31.000010 31119 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:31.085790 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:31.086477 29146 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:31.000643 29143 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:31.015717 29188 leader_election.cc:400] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
W20260709 13:04:31.088553 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 72960.8us
W20260709 13:04:31.089090 29903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 322) took 1724 ms (client timeout 2081 ms). Trace:
I20260709 13:04:31.089689 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:31.091044 31036 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.091580 31036 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.015187 29134 tablet_service.cc:1558] Processing DeleteTablet for tablet 348ae3050d97486fa402a5da2bbdce46 with delete_type TABLET_DATA_TOMBSTONED (TS 730186a8c70f43f2a57d80163bbc37d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42040
I20260709 13:04:31.092367 31184 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:31.092753 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:31.005419 30929 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:31.093966 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:31.094348 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.094664 29145 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:31.094993 29066 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:31.095948 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.096544 29066 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
W20260709 13:04:31.096812 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.097352 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.097554 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:31.098364 29148 raft_consensus.cc:2410] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:31.098775 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:31.099439 29148 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:04:31.100670 31020 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:31.100819 29148 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:04:31.101218 29148 pending_rounds.cc:85] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 3
I20260709 13:04:31.101439 29148 pending_rounds.cc:107] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:04:31.101773 29148 raft_consensus.cc:2889] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:31.102272 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 419) took 1915 ms. Trace:
I20260709 13:04:31.011286 31211 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:31.102785 31211 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.102689 29908 rpcz_store.cc:276] 0709 13:04:29.186438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.441154 (+1254716us) service_pool.cc:224] Handling call
0709 13:04:30.992074 (+550920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.074173 (+ 82099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.102085 (+ 27912us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b540051bb50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.102213 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1094310784}
I20260709 13:04:31.103246 31211 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:31.103564 30968 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.103874 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:31.104059 31120 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.104800 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1596) took 1360 ms. Trace:
I20260709 13:04:31.104792 31173 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:31.105401 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:31.106617 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:31.105229 29215 rpcz_store.cc:276] 0709 13:04:29.743981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.594177 (+850196us) service_pool.cc:224] Handling call
0709 13:04:30.601133 (+  6956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.712645 (+111512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.104555 (+391910us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b54003561d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:31.104742 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195093888}
I20260709 13:04:31.107677 29146 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:31.070798 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:30.954551 30978 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.073025 29147 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:31.074339 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.074744 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:31.016979 31123 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:31.075587 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:31.077875 29185 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.577s (SENT)
I20260709 13:04:31.069851 29221 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.078543 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:31.113031 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.104592 31120 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.070060 29070 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:31.113400 29071 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:31.111378 29142 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:31.107082 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:31.080262 29113 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.115159 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.092832 31184 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.113428 29221 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.069542 31073 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:31.117316 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:31.115396 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:31.117870 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.117911 29113 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.119061 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:31.121089 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:31.120741 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.121925 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:31.123064 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:31.119411 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:31.044795 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.124938 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.101715 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:31.127561 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:31.127892 29113 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:31.186561 29185 leader_election.cc:336] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.697s (SENT)
I20260709 13:04:31.167249 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 175) took 2509 ms. Trace:
I20260709 13:04:31.219336 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.220125 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.167249 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.178244 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:31.220885 29071 raft_consensus.cc:2376] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4. Re-sending same reply.
W20260709 13:04:31.172844 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.179795 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:31.220579 29911 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:31.223766 29908 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:31.219744 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 91947.3us
I20260709 13:04:31.225705 29910 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
W20260709 13:04:31.227633 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.219663 29069 rpcz_store.cc:276] 0709 13:04:28.657666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.815874 (+1158208us) service_pool.cc:224] Handling call
0709 13:04:29.825805 (+  9931us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.826316 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.826348 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:29.826933 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:04:30.211042 (+384109us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:30.691105 (+480063us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b18003f6220. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f783a5053c7 00007f7836b179a7 00007f7836b270bf
0709 13:04:30.691250 (+   145us) log.cc:844] Serialized 14 byte log entry
0709 13:04:30.691451 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.691522 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.996523 (+305001us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:31.167039 (+170516us) raft_consensus.cc:1685] finished
0709 13:04:31.167062 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.167186 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14048,"prepare.queue_time_us":189084,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":2054,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":13220,"spinlock_wait_cycles":29886720,"thread_start_us":410978,"threads_started":2,"wal-append.queue_time_us":425650}
I20260709 13:04:31.230268 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.230939 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:31.179180 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56177.5us
I20260709 13:04:31.232812 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:31.232977 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:31.128567 30852 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:31.234330 29905 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:31.234627 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.235268 29064 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:31.235499 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:31.236156 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.236375 29905 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:31.236732 29064 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:31.237555 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.238198 29214 raft_consensus.cc:2410] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:31.239423 29112 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:31.226943 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.233908 29907 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:31.113597 31173 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.240970 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.112705 31123 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.118675 31017 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.244102 29221 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:31.245095 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:31.069319 31099 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.288975 29911 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:04:31.289050 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44046.6us
I20260709 13:04:31.289915 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 420) took 1925 ms. Trace:
W20260709 13:04:31.290115 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 184) took 2148 ms (client timeout 1573 ms). Trace:
I20260709 13:04:31.290421 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:31.290547 29067 rpcz_store.cc:269] 0709 13:04:29.141735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.519413 (+377678us) service_pool.cc:224] Handling call
0709 13:04:31.289877 (+1770464us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54003edad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.290046 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3102698752}
I20260709 13:04:31.127992 30942 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:31.291396 31215 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:31.226924 31030 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.125823 31154 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:31.070082 31219 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:31.089529 29903 rpcz_store.cc:269] 0709 13:04:29.364796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.520399 (+155603us) service_pool.cc:224] Handling call
0709 13:04:31.088745 (+1568346us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54007c01d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.088996 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376695680}
I20260709 13:04:31.237479 30929 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:31.218590 30866 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:04:31.254309 29871 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.930s (SENT)
I20260709 13:04:31.104060 30968 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.252846 31180 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:31.259686 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.278558 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:31.287997 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.291126 29073 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:31.253161 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:31.251804 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:31.292758 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 1508) took 1029 ms. Trace:
I20260709 13:04:31.316119 29144 rpcz_store.cc:276] 0709 13:04:30.263366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.413954 (+150588us) service_pool.cc:224] Handling call
0709 13:04:31.292567 (+878613us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54003a57d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.292699 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642955008}
I20260709 13:04:31.250339 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.128824 30893 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:31.128950 31009 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:31.166467 31084 raft_consensus.cc:2764] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:04:31.120782 31184 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.229760 30758 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.126595 31162 raft_consensus.cc:2764] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:04:31.295488 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:31.116456 31202 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.297361 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.291848 31215 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.298416 29112 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:31.238059 31147 raft_consensus.cc:2764] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:04:31.074128 29906 rpcz_store.cc:269] 0709 13:04:29.530742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.542775 (+ 12033us) service_pool.cc:224] Handling call
0709 13:04:30.991945 (+1449170us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540051bb54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:30.992105 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395944704}
I20260709 13:04:31.229434 30692 raft_consensus.cc:2764] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:04:31.166774 30898 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.159s
I20260709 13:04:31.249274 31033 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:31.249099 31191 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:31.252568 30852 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:31.178860 31111 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.290302 29911 rpcz_store.cc:276] 0709 13:04:29.364196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.288328 (+1924132us) service_pool.cc:224] Handling call
0709 13:04:31.288729 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.289625 (+   896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.289846 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:31.313627 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.278915 31123 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:31.312073 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 57804us
W20260709 13:04:31.363101 29871 leader_election.cc:336] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.028s (SENT)
I20260709 13:04:31.348037 31162 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:31.244869 31174 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:04:31.288996 31107 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:31.300503 30997 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:31.368361 30997 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.349668 31219 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.368607 31223 tablet_replica.cc:333] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:04:31.368793 30997 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.296139 30985 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:31.241832 29904 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:31.326143 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:31.315405 31201 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:04:31.320493 31009 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.342027 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:31.344158 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:31.344623 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 223593us
I20260709 13:04:31.378839 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:31.354404 29218 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:04:31.321606 30929 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:31.344612 31215 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:31.324098 29905 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:31.363767 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.317953 31030 raft_consensus.cc:2764] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:04:31.327461 31119 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.347071 31043 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.362964 29907 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:31.360869 31033 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:31.321199 31073 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.386716 29911 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:31.373185 31162 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:31.375707 31049 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.365358 31191 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:31.388641 31009 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.373867 31225 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:31.404435 31119 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:31.405666 31111 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:31.406620 31101 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.407111 31020 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:31.407317 31043 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:31.407534 31020 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:31.408062 31036 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:31.416464 31191 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.422616 31228 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:31.423728 31228 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:31.424124 31228 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.411342 31225 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:31.425714 31225 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.422748 31162 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.428606 31184 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.430620 31184 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:31.466305 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.469024 29067 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:31.469468 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.470753 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.471561 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.472204 29067 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:31.472759 29065 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:31.473470 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.474001 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.474655 29067 raft_consensus.cc:2368] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3.
I20260709 13:04:31.312973 29069 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:31.347901 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.314227 29071 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
W20260709 13:04:31.341886 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 43532.6us
W20260709 13:04:31.319762 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 68035us
W20260709 13:04:31.324559 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.475862 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:31.511089 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.505445 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 240) took 1068 ms. Trace:
I20260709 13:04:31.477070 29066 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:31.485381 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.492159 29072 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:31.512471 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.513523 29070 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:31.514183 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:31.517961 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.511947 29139 rpcz_store.cc:276] 0709 13:04:30.436460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.439407 (+  2947us) service_pool.cc:224] Handling call
0709 13:04:31.504738 (+1065331us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b54002e8954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.505126 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141142784}
I20260709 13:04:31.519289 29066 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:31.520314 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:31.521680 29139 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:31.520807 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.342645 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:31.529783 29144 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:31.530382 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:31.531651 29143 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:31.345247 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:31.532743 29145 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.369815 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1610) took 1104 ms. Trace:
I20260709 13:04:31.534020 29217 rpcz_store.cc:276] 0709 13:04:30.265691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.265944 (+   253us) service_pool.cc:224] Handling call
0709 13:04:31.369571 (+1103627us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b5400355cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.369739 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126698880}
I20260709 13:04:31.527592 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.537587 29069 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:31.538846 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.511202 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.374439 29221 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:31.390815 30942 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:31.393355 30893 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:31.374955 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 144086us
I20260709 13:04:31.492710 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 180) took 2684 ms. Trace:
I20260709 13:04:31.474033 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.523307 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:31.515964 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.478565 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:31.525794 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.539875 29069 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:31.579571 29070 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:31.475936 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.566573 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:31.568490 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:31.570922 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:31.564610 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:31.552575 29144 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:31.556615 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:31.543962 29903 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:04:31.587245 31024 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:31.544363 29906 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:31.587798 31024 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:31.588462 30991 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:31.589154 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 422) took 1766 ms. Trace:
I20260709 13:04:31.589782 29906 rpcz_store.cc:276] 0709 13:04:29.822786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.353962 (+1531176us) service_pool.cc:224] Handling call
0709 13:04:31.377658 (+ 23696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.588578 (+210920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.589016 (+   438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:31.589986 31024 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.564642 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:31.541429 29875 leader_election.cc:336] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.507s (SENT)
I20260709 13:04:31.597038 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:31.575117 29073 rpcz_store.cc:276] 0709 13:04:28.807835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.908348 (+1100513us) service_pool.cc:224] Handling call
0709 13:04:31.290780 (+1382432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.313476 (+ 22696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.492124 (+178648us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54003edad0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.492410 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3144859008}
I20260709 13:04:31.600605 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1509) took 1336 ms. Trace:
I20260709 13:04:31.601325 29144 rpcz_store.cc:276] 0709 13:04:30.263657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.530909 (+1267252us) service_pool.cc:224] Handling call
0709 13:04:31.531589 (+   680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.600151 (+ 68562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.600484 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:31.604756 29143 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.608871 29144 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:31.572160 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1808) took 1318 ms. Trace:
I20260709 13:04:31.609934 29140 rpcz_store.cc:276] 0709 13:04:30.253501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.253617 (+   116us) service_pool.cc:224] Handling call
0709 13:04:30.253902 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.591461 (+337559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.571584 (+980123us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54003b4054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:31.571960 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523265920}
W20260709 13:04:31.612558 29185 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.849s (SENT)
I20260709 13:04:31.551213 30942 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.613163 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.613191 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 241) took 1163 ms. Trace:
I20260709 13:04:31.563941 29219 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:04:31.547367 29071 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:31.615759 29140 rpcz_store.cc:276] 0709 13:04:30.450162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.611727 (+1161565us) service_pool.cc:224] Handling call
0709 13:04:31.612347 (+   620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.612770 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.612807 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.612879 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:31.612929 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.612997 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.613111 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:31.619465 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1510) took 1351 ms. Trace:
I20260709 13:04:31.587616 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:31.620064 29144 rpcz_store.cc:276] 0709 13:04:30.263944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.602990 (+1339046us) service_pool.cc:224] Handling call
0709 13:04:31.608129 (+  5139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.614541 (+  6412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.614818 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:31.629532 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.551414 29214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1608) took 1608 ms (client timeout 1885 ms). Trace:
I20260709 13:04:31.569125 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:31.634935 29064 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:31.635286 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 406) took 2106 ms. Trace:
I20260709 13:04:31.635725 29903 rpcz_store.cc:276] 0709 13:04:29.528567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.349243 (+1820676us) service_pool.cc:224] Handling call
0709 13:04:31.375315 (+ 26072us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.587384 (+212069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.635000 (+ 47616us) spinlock_profiling.cc:243] Waited 8.46 ms on lock 0x7b5400767fd0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.635212 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19450752}
I20260709 13:04:31.636327 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.637246 29064 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:31.637408 29139 raft_consensus.cc:2410] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:31.638216 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.639070 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 246) took 1098 ms. Trace:
I20260709 13:04:31.639420 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.543529 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 313) took 4537 ms. Trace:
W20260709 13:04:31.645099 31237 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:04:31.647135 31233 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:31.633263 29214 rpcz_store.cc:269] 0709 13:04:29.971193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.223063 (+251870us) service_pool.cc:224] Handling call
0709 13:04:31.550863 (+1327800us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003a93d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.551158 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2332038016}
I20260709 13:04:31.650602 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 315) took 2780 ms. Trace:
I20260709 13:04:31.651156 29909 rpcz_store.cc:276] 0709 13:04:28.870278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.438304 (+1568026us) service_pool.cc:224] Handling call
0709 13:04:30.438623 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.804347 (+365724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.650094 (+845747us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54005bd1d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:31.650384 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766029184}
I20260709 13:04:31.408423 31020 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:31.657364 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 294138us
I20260709 13:04:31.657858 31020 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.658836 29065 raft_consensus.cc:2410] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:31.419585 31211 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.660611 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.661952 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.662345 31211 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:31.662496 29065 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.573818 30893 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.663966 29215 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:31.562556 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:31.621049 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 46873.4us
I20260709 13:04:31.639436 29139 rpcz_store.cc:276] 0709 13:04:30.540298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.638045 (+1097747us) service_pool.cc:224] Handling call
0709 13:04:31.638407 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.638764 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.638788 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.638846 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:31.638873 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.638942 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.639025 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:31.554373 30968 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:31.677080 31235 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 13:04:31.554250 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 175398us
I20260709 13:04:31.568572 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 323) took 2203 ms. Trace:
I20260709 13:04:31.409550 31120 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:31.678320 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:31.676837 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.679745 29145 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:31.681788 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.680176 29066 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:31.646760 30922 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.687439 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.688675 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.686750 31024 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:31.691533 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.692456 29214 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:04:31.623517 31033 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.693949 29146 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:31.695585 29146 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:31.696287 29147 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:31.697242 29142 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:31.697324 31101 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:31.699064 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 220866us
I20260709 13:04:31.679232 29907 rpcz_store.cc:276] 0709 13:04:29.365089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.302531 (+1937442us) service_pool.cc:224] Handling call
0709 13:04:31.332160 (+ 29629us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.567835 (+235675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.568337 (+   502us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:31.550302 30929 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.644248 29902 rpcz_store.cc:276] 0709 13:04:27.006181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.926916 (+2920735us) service_pool.cc:224] Handling call
0709 13:04:31.092584 (+1165668us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:31.092816 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.093082 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:31.293140 (+200058us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.542368 (+249228us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.542529 (+   161us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:31.542731 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.542911 (+   180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.543172 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":219862,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":2931,"raft.queue_time_us":107665,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":214,"spinlock_wait_cycles":67467008,"thread_start_us":385620,"threads_started":2,"wal-append.queue_time_us":106542}
W20260709 13:04:31.690160 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 57124.6us
I20260709 13:04:31.591655 30991 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.095s
I20260709 13:04:31.707358 29139 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:31.711452 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.712721 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.713676 29069 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.715512 29874 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:31.621516 31192 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.722517 29147 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:31.724591 29143 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:31.726058 30968 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:31.726613 30991 raft_consensus.cc:3055] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:31.727105 30991 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:31.726799 30968 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.729578 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.730262 31162 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:31.731474 31191 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:31.732836 31111 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:31.733314 31111 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.733772 31111 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:31.736334 29140 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:04:31.738915 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1539) took 1016 ms. Trace:
I20260709 13:04:31.740298 31049 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.740935 31049 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.743342 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.743947 31049 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:31.750424 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.752441 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:31.751693 29071 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 2.
I20260709 13:04:31.740017 29140 rpcz_store.cc:276] 0709 13:04:30.722701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.673953 (+951252us) service_pool.cc:224] Handling call
0709 13:04:31.718621 (+ 44668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.737092 (+ 18471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.738780 (+  1688us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:31.752094 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.759707 29069 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
W20260709 13:04:31.747735 29908 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.523s	user 0.005s	sys 0.009s
W20260709 13:04:31.761754 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 149414us
I20260709 13:04:31.735102 30968 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:31.766155 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.771539 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.766768 29908 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:31.775803 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.777947 29071 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:31.777935 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:31.779312 29071 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:31.774186 29185 leader_election.cc:336] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.910s (SENT)
I20260709 13:04:31.781004 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.782423 31240 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:31.782864 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:31.783047 31099 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.767855 30991 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.790310 30991 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.792088 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:31.791960 31009 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.792866 29902 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:31.793119 29907 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:31.794986 31009 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:31.795483 31228 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.796438 30692 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:31.796841 30997 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.797149 30968 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:31.804260 29067 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:31.804968 30852 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:31.839527 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:31.840487 29216 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:31.844642 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:31.845510 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.814672 29039 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.850s (SENT)
I20260709 13:04:31.815095 29871 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:31.848044 31240 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.817782 29141 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:31.849009 29064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:31.849846 29064 raft_consensus.cc:606] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:31.850138 31228 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:31.818482 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 103135us
I20260709 13:04:31.820132 29214 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:31.821128 29065 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:31.852440 29067 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:04:31.821338 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:31.853360 29218 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:31.853188 31240 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:31.853643 29911 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:31.822487 31202 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.824545 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:31.822655 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:31.823148 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:31.835242 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:31.836515 30968 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:31.837657 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:31.845463 29222 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:31.861191 29071 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:31.862007 29065 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:31.809149 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.816548 31101 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:31.854727 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:31.858857 29066 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:31.850291 29064 raft_consensus.cc:614] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:31.871372 29219 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:31.871548 29221 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:31.872014 29070 raft_consensus.cc:2410] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:31.879878 29073 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:31.818326 31123 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:31.880954 29067 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:31.883881 29216 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:31.885329 29220 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:31.861138 30968 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:31.890321 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:31.889590 29215 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:31.891618 29148 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:31.893187 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:31.895570 29217 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:31.897897 29218 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:31.899855 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:31.900589 29220 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.904066 30758 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.906080 30968 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.909688 29072 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:31.915158 29142 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:31.915566 31099 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:31.915891 29904 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:04:31.920794 29219 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:04:31.907954 31215 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:31.936707 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.948499 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.949234 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.953366 29148 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:31.954505 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:31.954751 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.955241 29146 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:31.957562 29068 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:31.958537 31215 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:31.959789 29222 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
W20260709 13:04:31.960394 29871 leader_election.cc:336] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.908s (SENT)
I20260709 13:04:31.961097 30997 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:31.961444 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 332) took 1186 ms. Trace:
I20260709 13:04:31.962685 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1540) took 1008 ms. Trace:
I20260709 13:04:31.969585 29072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:31.971066 29072 raft_consensus.cc:606] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:31.971458 29072 raft_consensus.cc:614] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:31.970484 29065 rpcz_store.cc:276] 0709 13:04:30.953957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.852411 (+898454us) service_pool.cc:224] Handling call
0709 13:04:31.861838 (+  9427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.862400 (+   562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.962449 (+100049us) spinlock_profiling.cc:243] Waited 9.03 ms on lock 0x7b54000695d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.962606 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20761856}
W20260709 13:04:31.976055 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 201883us
W20260709 13:04:31.976796 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.965889 31020 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:31.979883 31020 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:31.980129 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:31.980489 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:31.982631 29215 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:04:31.988761 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1519) took 1073 ms. Trace:
I20260709 13:04:31.989300 29073 rpcz_store.cc:276] 0709 13:04:30.907865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.780594 (+872729us) service_pool.cc:224] Handling call
0709 13:04:31.809766 (+ 29172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.883516 (+ 73750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.981503 (+ 97987us) spinlock_profiling.cc:243] Waited 65.2 ms on lock 0x7b54000686d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:31.981667 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149938560}
I20260709 13:04:31.963558 31033 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.997083 31033 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:31.999418 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1600) took 1066 ms. Trace:
I20260709 13:04:31.999820 29066 rpcz_store.cc:276] 0709 13:04:30.933050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.718396 (+785346us) service_pool.cc:224] Handling call
0709 13:04:31.999255 (+280859us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b54002d3f54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.999357 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182002048}
I20260709 13:04:32.008553 31033 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:32.009644 31241 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:04:32.012755 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 243) took 1528 ms. Trace:
I20260709 13:04:32.013181 29144 rpcz_store.cc:276] 0709 13:04:30.484655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.632087 (+1147432us) service_pool.cc:224] Handling call
0709 13:04:31.685672 (+ 53585us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:31.720502 (+ 34830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.720530 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:31.738175 (+ 17645us) log.cc:844] Serialized 52 byte log entry
0709 13:04:31.777590 (+ 39415us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:31.777653 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.777730 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.012528 (+234798us) raft_consensus.cc:1685] finished
0709 13:04:32.012553 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.012692 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22417,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":179,"thread_start_us":37068,"threads_started":1,"wal-append.queue_time_us":134394}
I20260709 13:04:32.014077 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:32.015115 31009 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:32.015460 30968 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:32.015722 29185 leader_election.cc:336] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.964s (SENT)
I20260709 13:04:32.016254 29185 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:32.016734 31009 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:32.017177 31009 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.017542 31009 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:32.015275 29113 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:32.018568 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:32.016628 29065 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:32.025293 29146 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
W20260709 13:04:32.017668 29185 leader_election.cc:336] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.933s (SENT)
I20260709 13:04:31.991210 31240 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
W20260709 13:04:32.032507 29188 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.047s (SENT)
W20260709 13:04:32.052425 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48063.4us
I20260709 13:04:32.049100 29215 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:32.049001 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:32.041366 29220 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:32.042426 29059 tablet_service.cc:1558] Processing DeleteTablet for tablet 344aa5f73f0f4124b1512832a0bca2c9 with delete_type TABLET_DATA_TOMBSTONED (TS a936878f59c840a6bd176a1d2d7347b1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47946
W20260709 13:04:32.055994 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 514893us
I20260709 13:04:32.028455 30758 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:32.057935 29039 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.061s (SENT)
I20260709 13:04:32.063287 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:32.041312 30929 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:32.060602 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:32.067080 31017 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.099331 31017 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.042232 31033 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:32.072710 30922 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.970466 29904 rpcz_store.cc:276] 0709 13:04:30.774951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.554598 (+779647us) service_pool.cc:224] Handling call
0709 13:04:31.781479 (+226881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.916460 (+134981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.960165 (+ 43705us) spinlock_profiling.cc:243] Waited 73.9 ms on lock 0x7b54002a08d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:31.961181 (+  1016us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54002a08d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:31.961366 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577053312}
W20260709 13:04:32.080230 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 65038.1us
I20260709 13:04:32.082104 31020 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.088517 29112 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:32.089079 29186 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:32.096081 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.097270 29217 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:04:32.099776 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 50844.7us
W20260709 13:04:32.076822 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59206us
I20260709 13:04:32.078580 29214 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:04:32.104720 29188 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.158s (SENT)
I20260709 13:04:32.108166 29038 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:32.144853 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 55959.2us
I20260709 13:04:32.107509 31024 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:32.147982 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 59774.7us
I20260709 13:04:32.149645 31020 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:32.150282 31080 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:32.113533 30929 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:32.164074 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:04:32.163105 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 152632us
W20260709 13:04:32.168352 29185 leader_election.cc:336] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.650s (SENT)
I20260709 13:04:32.169001 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:32.172283 29871 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.042s (SENT)
I20260709 13:04:32.172948 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:32.175073 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 69058.5us
I20260709 13:04:32.177752 30991 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:32.178558 30991 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:32.181973 30997 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:32.183079 31036 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:32.184335 31036 consensus_queue.cc:1243] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:32.185638 29146 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:32.186131 29146 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:32.188429 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:32.188742 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.190706 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1619) took 1110 ms. Trace:
I20260709 13:04:32.191751 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:32.190737 29148 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:32.194233 29113 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:32.195822 31030 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:32.196377 31030 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.196835 31030 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:32.197171 31017 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:32.192768 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 49548.5us
I20260709 13:04:32.199764 31049 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.200567 31049 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:32.203279 31049 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:32.208151 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:32.179874 30991 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:32.211190 31192 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:32.191190 29067 rpcz_store.cc:276] 0709 13:04:31.080449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.891077 (+810628us) service_pool.cc:224] Handling call
0709 13:04:31.913697 (+ 22620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.919749 (+  6052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.919775 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:31.920519 (+   744us) log.cc:844] Serialized 26 byte log entry
0709 13:04:32.102880 (+182361us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:32.102922 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.102996 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.190468 (+ 87472us) raft_consensus.cc:1685] finished
0709 13:04:32.190489 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.190623 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":132,"thread_start_us":179573,"threads_started":1,"wal-append.queue_time_us":267810}
I20260709 13:04:32.187162 29146 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.213665 29146 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:32.216305 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 176866us
I20260709 13:04:32.222450 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:32.216943 31228 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.174391 30922 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:32.226733 31024 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:32.226276 31228 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.227217 31024 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:32.227705 31024 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:32.200268 31073 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:04:32.231101 31223 raft_consensus.cc:2243] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:32.217765 30692 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:32.164788 31123 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:32.233594 31192 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:32.236220 31123 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:32.236750 31123 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:32.237856 31123 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:32.239295 31123 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:32.217926 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 45703.8us
W20260709 13:04:32.240756 29871 leader_election.cc:336] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.080s (SENT)
W20260709 13:04:32.219101 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 49181.8us
I20260709 13:04:32.220013 30978 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:32.162793 31033 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:32.243782 31033 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.243826 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:32.244340 31033 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.244598 29908 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:32.220427 30929 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.165371 31080 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.248188 31080 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:32.248091 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:32.250983 30942 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:32.252637 30942 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:32.253111 31257 tablet_replica.cc:333] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:04:32.220978 31049 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:32.242522 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:32.254751 29038 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:32.262841 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 71219.1us
I20260709 13:04:32.229264 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 419) took 1716 ms. Trace:
I20260709 13:04:32.264497 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:32.231312 31073 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:04:32.267841 29908 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:32.270722 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:32.281677 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 114732us
I20260709 13:04:32.264770 29910 rpcz_store.cc:276] 0709 13:04:30.512984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.550471 (+1037487us) service_pool.cc:224] Handling call
0709 13:04:31.551342 (+   871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.679689 (+128347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.679725 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:31.763619 (+ 83894us) log.cc:844] Serialized 26 byte log entry
0709 13:04:32.167532 (+403913us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:32.167578 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.167683 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.228810 (+ 61127us) raft_consensus.cc:1685] finished
0709 13:04:32.228830 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.229006 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95786,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":128,"thread_start_us":498279,"threads_started":2,"wal-append.queue_time_us":443256}
I20260709 13:04:32.272009 29067 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
W20260709 13:04:32.288223 29871 leader_election.cc:336] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.163s (SENT)
W20260709 13:04:32.267333 31111 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.533s	user 0.007s	sys 0.006s
I20260709 13:04:32.268172 30893 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:32.296468 30893 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:32.290598 30692 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:32.300125 31223 raft_consensus.cc:2272] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.301019 31249 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:32.301596 29144 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:32.302209 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:32.302543 31202 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.303345 29065 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:32.303105 31202 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:32.303786 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1620) took 1186 ms. Trace:
I20260709 13:04:32.304286 30968 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:32.303167 29902 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:04:32.310299 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 86199.6us
I20260709 13:04:32.310532 31228 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:32.297598 30893 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:32.317828 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.321959 29145 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:32.318038 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 122705us
I20260709 13:04:32.315665 29146 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:04:32.304334 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:32.311439 29064 rpcz_store.cc:276] 0709 13:04:31.117039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.896983 (+779944us) service_pool.cc:224] Handling call
0709 13:04:31.965025 (+ 68042us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.965542 (+   517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.965575 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:31.966034 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:04:32.114653 (+148619us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:32.114722 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.114815 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.303552 (+188737us) raft_consensus.cc:1685] finished
0709 13:04:32.303572 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.303716 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3249,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":5983,"thread_start_us":148053,"threads_started":1,"wal-append.queue_time_us":288217}
I20260709 13:04:32.328809 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:32.329550 29139 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:32.330646 29148 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:32.332614 29216 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:32.325618 31192 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:04:32.334110 31080 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.335129 31192 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:32.336970 30922 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:32.336030 28969 catalog_manager.cc:5697] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 2 to 3, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 3 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.337343 30922 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:32.338577 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:32.328284 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:32.340777 29910 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:32.340961 29140 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:04:32.345288 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70727.7us
I20260709 13:04:32.349730 29145 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:32.362568 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:32.360549 29213 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
W20260709 13:04:32.364907 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.365231 29139 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:32.365866 31228 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:32.368883 31080 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:32.371744 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:32.372812 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:32.373289 31202 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:32.373538 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:32.373920 31202 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:32.374253 29910 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:32.365886 31094 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:32.374843 31094 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:32.375260 31094 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:32.376682 31009 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:32.378821 31223 ts_tablet_manager.cc:1916] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:32.379727 29904 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:32.374321 31202 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.295614 31111 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:32.381296 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 232794us
W20260709 13:04:32.383529 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.399461 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.364243 31036 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:04:32.385511 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:32.382107 28969 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 3 to 4, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 4 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:04:32.394011 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:32.401810 31239 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
W20260709 13:04:32.402537 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 53216.7us
I20260709 13:04:32.405396 31017 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:32.395097 30978 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:04:32.406180 31017 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:32.401803 31147 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:32.409581 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.420513 29069 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:32.414587 28969 catalog_manager.cc:5697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 3 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.398698 31202 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.423745 29215 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:32.395193 31228 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:04:32.409098 31147 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.458671 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:32.450114 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:32.423883 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 212621us
W20260709 13:04:32.485677 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.483861 28969 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 5 to 6, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 6 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.407435 31017 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:32.476526 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 251655us
I20260709 13:04:32.457533 31036 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:32.417397 30997 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:04:32.486729 31147 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:32.505221 28969 catalog_manager.cc:5697] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:32.514585 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 63762.7us
W20260709 13:04:32.516891 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 144160us
I20260709 13:04:32.512491 28969 catalog_manager.cc:5697] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 3 to 4, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 4 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.482414 31033 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.521451 28969 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.419736 30922 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:32.523113 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 160726us
W20260709 13:04:32.530300 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.514276 30758 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:32.526762 28969 catalog_manager.cc:5697] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 2, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.543107 29910 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:32.551873 28969 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:32.563619 28969 catalog_manager.cc:5697] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 3, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:04:32.567847 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:32.586548 29038 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:32.591336 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:04:32.592759 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 142758us
I20260709 13:04:32.602481 31134 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:04:32.565088 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:32.592125 28969 catalog_manager.cc:5697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 2 to 3, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 3 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:32.622437 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 202473us
I20260709 13:04:32.581228 31009 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:32.608012 31036 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:32.635804 28969 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 4, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 4 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:32.638253 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47009us
I20260709 13:04:32.622287 30922 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:32.633544 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 115417us
W20260709 13:04:32.643803 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 82298.3us
I20260709 13:04:32.645033 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:32.621351 31215 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:32.627948 30997 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:04:32.656703 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.639624 31262 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:04:32.637050 31257 raft_consensus.cc:2243] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:32.663757 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:32.665364 31263 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:04:32.661433 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:32.634220 31111 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:32.644519 29904 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:32.677907 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:32.648994 31009 raft_consensus.cc:2764] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:04:32.642163 31276 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:32.649084 31215 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.676898 31257 raft_consensus.cc:2272] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:32.654737 31273 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:32.686779 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:32.679560 31123 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:32.670403 30997 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:04:32.690179 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 290596us
W20260709 13:04:32.699530 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45419.7us
I20260709 13:04:32.690634 31264 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:32.686523 31215 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:32.684227 31276 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:32.691670 31273 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.692371 31111 raft_consensus.cc:2764] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:04:32.682711 31030 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:32.722998 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 45374.4us
I20260709 13:04:32.715788 31282 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:32.716014 31123 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:32.724102 31263 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:32.724349 31285 raft_consensus.cc:2764] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:04:32.725028 31256 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
W20260709 13:04:32.729748 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 170979us
I20260709 13:04:32.735064 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 194) took 1627 ms. Trace:
I20260709 13:04:32.796286 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:32.796770 31017 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:32.798770 31030 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:32.790835 31274 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 13:04:32.775327 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:32.802832 31277 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 13:04:32.803529 31266 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 13:04:32.803894 31020 consensus_meta.cc:287] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.637s	user 0.009s	sys 0.001s
W20260709 13:04:32.801409 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 89231.1us
I20260709 13:04:32.791771 31111 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:32.712589 31033 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:32.805014 31283 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:32.719676 31273 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.818800 29069 rpcz_store.cc:276] 0709 13:04:31.107470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.760590 (+653120us) service_pool.cc:224] Handling call
0709 13:04:32.734867 (+974277us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54002ace54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:32.735005 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1475474304}
I20260709 13:04:32.712749 30893 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.093s
W20260709 13:04:32.856763 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 54624.1us
I20260709 13:04:32.826583 31263 consensus_queue.cc:1243] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:04:32.848359 29871 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.093s (SENT) [suppressed 79 similar messages]
I20260709 13:04:32.824972 31017 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:32.850430 31024 consensus_meta.cc:287] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.616s	user 0.012s	sys 0.000s
I20260709 13:04:32.742125 31264 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.872978 29038 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:32.858255 29902 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.875196 31257 ts_tablet_manager.cc:1916] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:32.878865 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:32.882537 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1618) took 1806 ms. Trace:
I20260709 13:04:32.831398 29904 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:32.891108 29071 rpcz_store.cc:276] 0709 13:04:31.076091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.892354 (+816263us) service_pool.cc:224] Handling call
0709 13:04:31.961778 (+ 69424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:31.962178 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.962216 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:31.964754 (+  2538us) log.cc:844] Serialized 26 byte log entry
0709 13:04:32.265713 (+300959us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:32.265761 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.265831 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.882314 (+616483us) raft_consensus.cc:1685] finished
0709 13:04:32.882334 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.882456 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":645,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":4587,"thread_start_us":269390,"threads_started":1,"wal-append.queue_time_us":498964}
I20260709 13:04:32.873775 31017 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.895073 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:32.821944 31147 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.871281 31020 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.919755 31123 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:32.817382 31036 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:32.924916 31094 consensus_meta.cc:287] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.002s
W20260709 13:04:32.944406 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 65652.1us
I20260709 13:04:32.825359 31276 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.909605 31283 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:32.912804 30893 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:32.947990 31264 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:32.933943 31080 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:32.884799 30997 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:32.958962 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70466.5us
I20260709 13:04:32.888072 31024 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:32.958103 31094 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.916052 31020 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:32.962277 31263 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.973032 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:32.972096 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 111415us
I20260709 13:04:32.954784 31290 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.968626 31276 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:32.982007 29185 connection.cc:256] RPC call timeout handler was delayed by -1.3187211539998316s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2405s for a response
I20260709 13:04:32.992643 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:32.989778 29185 connection.cc:256] RPC call timeout handler was delayed by -1.2965447810001933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23820000000000002s for a response
I20260709 13:04:32.978256 31030 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.999619 31094 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:32.975014 30991 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.984902 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:33.023512 29185 connection.cc:256] RPC call timeout handler was delayed by -1.2407447810001031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2444s for a response
I20260709 13:04:32.999871 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:04:33.030368 29185 connection.cc:256] RPC call timeout handler was delayed by -1.189121153999622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2549s for a response
I20260709 13:04:33.001302 31297 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:33.031819 31080 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.964804 31283 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:33.035079 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.027495 31030 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.039186 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:33.001502 31286 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:33.033953 31202 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.003s
I20260709 13:04:33.001381 31296 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.040154 29907 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:33.001204 31024 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:33.007396 31147 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:33.015530 31020 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:33.046443 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 57951.5us
I20260709 13:04:33.037637 31111 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.987289 31290 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:33.054443 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 54644.7us
I20260709 13:04:33.012035 31215 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:33.032720 31228 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:04:33.006024 31272 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:04:33.049948 29904 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:33.032527 31080 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:32.987959 31263 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:33.033129 30991 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:33.075507 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 46620.8us
I20260709 13:04:33.048075 31202 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.081956 29146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:04:33.068475 30997 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.094269 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.093776 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:33.097680 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:33.098063 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:33.098320 29071 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:33.098479 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:33.099037 29064 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:33.099051 29069 raft_consensus.cc:2368] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:33.098124 31024 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:33.104095 31294 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:33.049803 31296 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:33.105129 29871 leader_election.cc:336] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.056s (SENT)
I20260709 13:04:33.105579 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:33.112562 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:33.113777 31030 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:33.113731 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:33.114481 29213 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:33.114784 29064 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:33.115039 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:33.115643 29067 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:33.080776 31017 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:33.117242 31080 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:33.117758 31017 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:33.118161 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:33.120456 31273 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.123045 29067 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:33.079767 31286 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:04:33.128615 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:33.129458 29214 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:33.133522 31306 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.134186 31306 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.136858 31307 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:04:33.137524 31309 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:04:33.104868 31301 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.094689 30997 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:33.263505 29220 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 13:04:33.252378 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 208429us
I20260709 13:04:33.292160 29220 pending_rounds.cc:85] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 5
I20260709 13:04:33.157835 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:33.266516 29218 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:04:33.152504 29146 raft_consensus.cc:606] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:33.122687 31239 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:33.301303 29185 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.158s (SENT)
I20260709 13:04:33.294708 29146 raft_consensus.cc:614] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:33.302114 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:33.300755 29213 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:33.303422 29215 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:04:33.240676 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:04:33.269248 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.292797 29220 pending_rounds.cc:107] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:04:33.304924 29220 raft_consensus.cc:2889] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:33.252641 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:33.127141 31303 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:04:33.145610 31283 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:33.149917 31272 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:33.144956 30991 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:33.258069 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:33.305755 29217 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:33.488054 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:33.578945 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.635483 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 334170us
I20260709 13:04:33.532343 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:33.128163 31030 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.142437 29902 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
W20260709 13:04:33.341040 31310 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 13:04:33.133658 31202 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:33.140362 31296 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:33.649248 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 408656us
I20260709 13:04:33.139313 31215 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:33.650075 29215 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
W20260709 13:04:33.356870 31312 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:04:33.265451 31263 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:33.141119 31290 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:33.305413 30893 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:33.331184 31314 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 13:04:33.340157 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:33.452064 31281 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:33.306444 31306 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:33.333259 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:33.331329 29019 placement_policy_util.cc:248] tablet 344aa5f73f0f4124b1512832a0bca2c9: not considering replicas for movement since the tablet's status is 'UNAVAILABLE'
I20260709 13:04:33.453145 30929 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:04:33.660889 31111 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.010s
I20260709 13:04:33.364362 31319 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:33.331980 30997 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:33.693420 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:33.473443 31297 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:33.365206 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:33.648221 31273 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:33.720256 29185 leader_election.cc:336] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.160s (SENT)
I20260709 13:04:33.665519 31036 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.702841 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:33.726635 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:33.704854 31283 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.758721 29213 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:33.697788 31030 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.717486 29218 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:04:33.687606 30942 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.716905 31290 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.336046 31049 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:33.661283 29904 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:33.300969 31301 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.697806 31305 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.336247 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:33.706677 31279 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:33.711028 30929 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:33.696584 31223 ts_tablet_manager.cc:1929] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:33.811313 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:33.815867 29220 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
W20260709 13:04:33.797930 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 105502us
I20260709 13:04:33.717458 31111 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:33.662672 31272 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:33.734408 31294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.004s	sys 0.024s
I20260709 13:04:33.763865 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:33.821022 31294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.004s	sys 0.026s
I20260709 13:04:33.821211 29139 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:33.821446 31294 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:33.822192 31272 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:33.737120 31036 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.736970 31297 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.770354 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:33.772248 29217 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:33.772974 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:33.793147 30942 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:33.841475 31297 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:33.753506 30893 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:04:33.845083 29147 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:33.845486 30942 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:33.718062 31281 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:33.779582 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.712690 31263 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:04:33.727406 31319 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:33.815443 29906 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:33.747805 31306 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:33.825778 29142 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:33.854015 29908 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:33.844194 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:33.773248 31283 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.840090 29141 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:33.847986 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 145876us
I20260709 13:04:33.715633 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:33.808331 31257 ts_tablet_manager.cc:1929] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:04:33.851243 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:33.864907 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:33.865164 29147 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:33.863482 29186 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:33.793025 31290 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:33.823513 29185 connection.cc:256] RPC call timeout handler was delayed by -1.1374792500000694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21989999999999998s for a response
I20260709 13:04:33.794746 31009 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:33.854797 29144 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:33.849673 31223 log.cc:1199] T 348ae3050d97486fa402a5da2bbdce46 P 730186a8c70f43f2a57d80163bbc37d2: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/wals/348ae3050d97486fa402a5da2bbdce46
I20260709 13:04:33.825220 31279 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.849263 31272 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.858530 31024 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.753s	user 0.002s	sys 0.009s
I20260709 13:04:33.875136 31257 log.cc:1199] T 344aa5f73f0f4124b1512832a0bca2c9 P a936878f59c840a6bd176a1d2d7347b1: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/wals/344aa5f73f0f4124b1512832a0bca2c9
I20260709 13:04:33.875648 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:33.876463 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:33.876806 31320 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.774s	user 0.009s	sys 0.000s
I20260709 13:04:33.877441 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:33.878187 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:33.878893 31330 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.879380 29064 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:33.822811 31294 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.882843 31290 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:33.884500 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:33.887071 31319 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.887660 31030 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:33.890125 29141 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:04:33.900055 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:33.919888 29215 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:33.920217 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:33.921146 29066 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 2: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:33.922631 31279 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:33.901880 29185 leader_election.cc:336] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.609s (SENT)
I20260709 13:04:33.924237 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:33.924984 29185 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.624s (SENT)
I20260709 13:04:33.864672 29905 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:33.903398 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:33.927150 29073 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:33.904405 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:33.905117 29070 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:33.840739 30929 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.925475 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:33.905514 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:33.929003 29068 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:33.797371 31305 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:33.892731 31009 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:33.908390 31330 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:33.917364 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:33.918172 31223 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42040 (request call id 77) took 2958 ms. Trace:
W20260709 13:04:33.932192 29214 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.668s	user 0.000s	sys 0.010s
I20260709 13:04:33.907513 29112 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:33.933075 29215 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:33.900507 31024 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:33.936046 29218 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:33.955060 29214 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:33.954152 31257 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47946 (request call id 78) took 1987 ms. Trace:
I20260709 13:04:33.957172 31257 rpcz_store.cc:276] 0709 13:04:31.994453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.042060 (+ 47607us) service_pool.cc:224] Handling call
0709 13:04:32.252508 (+210448us) ts_tablet_manager.cc:1151] Deleting tablet 344aa5f73f0f4124b1512832a0bca2c9
0709 13:04:32.252635 (+   127us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:04:33.248433 (+995798us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:33.795226 (+546793us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:33.954036 (+158810us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":243135,"tablet-delete.queue_time_us":196988,"thread_start_us":77060,"threads_started":1}
I20260709 13:04:33.932708 31301 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:33.963866 31306 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:04:33.964607 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 57194.7us
I20260709 13:04:33.965335 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:33.964200 31301 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:33.951381 29139 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:33.940479 29073 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:33.901067 31272 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:33.977257 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:33.979629 29071 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:33.954856 31279 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:33.905141 31297 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:04:33.967686 30929 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.985327 29065 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:04:33.989235 29065 pending_rounds.cc:85] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 4
I20260709 13:04:33.989576 29065 pending_rounds.cc:107] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:04:33.989876 29065 raft_consensus.cc:2889] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:33.936295 31030 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:04:33.978045 31333 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:34.012954 29070 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:34.005062 31263 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.023214 31263 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.025545 31263 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:34.036202 29068 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
W20260709 13:04:34.055806 29216 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.000s	sys 0.016s
W20260709 13:04:34.056363 29216 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.000s	sys 0.016s
I20260709 13:04:34.063314 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:34.065131 29073 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:33.996940 31279 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:34.084911 31033 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:34.085520 31033 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:34.086591 31033 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.092470 31036 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:34.092774 31281 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:34.094255 31030 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:34.105068 31030 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.105592 31030 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:34.111063 31330 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:33.969178 31223 rpcz_store.cc:276] 0709 13:04:31.012028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.014865 (+  2837us) service_pool.cc:224] Handling call
0709 13:04:31.368204 (+353339us) ts_tablet_manager.cc:1151] Deleting tablet 348ae3050d97486fa402a5da2bbdce46
0709 13:04:31.368316 (+   112us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:04:33.042694 (+1674378us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:33.681888 (+639194us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:33.918059 (+236171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":870800,"tablet-delete.queue_time_us":274486,"thread_start_us":156212,"threads_started":1}
I20260709 13:04:34.027902 31283 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:34.095858 31009 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:34.101109 30922 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:34.064980 31333 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:04:34.138546 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 212237us
I20260709 13:04:34.124790 30922 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:34.162624 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:04:34.173971 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 279744us
I20260709 13:04:34.176518 29113 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:34.220131 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:34.176612 31330 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:34.188493 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:34.169596 31009 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:04:34.176349 31094 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.192s
I20260709 13:04:34.175426 29906 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
W20260709 13:04:34.355404 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 253033us
W20260709 13:04:34.494586 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 569635us
W20260709 13:04:34.277999 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 312606us
W20260709 13:04:34.495327 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 275301us
W20260709 13:04:34.504678 29185 leader_election.cc:336] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.654s (SENT)
I20260709 13:04:34.505113 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:34.505828 31360 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:34.267968 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:34.496224 31263 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:34.496732 31202 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:34.301113 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:34.493958 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:34.527269 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 226295us
I20260709 13:04:34.373468 31305 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:34.213552 30978 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.293156 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:34.502147 31330 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:34.523486 30929 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.004s
I20260709 13:04:34.492930 31024 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:34.227658 31340 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:34.437987 31272 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:34.251365 31341 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:34.450629 31017 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:34.320291 31239 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:34.252288 31338 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.292s
I20260709 13:04:34.400946 31049 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.564261 31334 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:04:34.564499 31365 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:34.405905 31333 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:34.526147 31263 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:34.534098 29906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 438) took 1225 ms (client timeout 1609 ms). Trace:
W20260709 13:04:34.546015 29185 leader_election.cc:336] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.883s (SENT)
I20260709 13:04:34.223244 31301 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:34.406399 31036 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.137s
I20260709 13:04:34.368189 31335 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.243s
I20260709 13:04:34.541983 31360 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:34.582266 29906 rpcz_store.cc:269] 0709 13:04:33.302801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.305339 (+  2538us) service_pool.cc:224] Handling call
0709 13:04:34.523318 (+1217979us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.593120 31360 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.574869 31017 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:34.578749 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:34.595651 31017 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:34.595959 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 64926us
W20260709 13:04:34.596735 29185 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.047s (SENT)
I20260709 13:04:34.597100 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:34.585397 31030 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:34.598464 29185 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.382s (SENT)
I20260709 13:04:34.575960 31239 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:34.600096 31030 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:34.557761 30978 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:34.580163 31340 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:34.603662 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:34.219002 30922 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.214061 31279 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.564134 31024 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.573487 31350 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:34.575304 31341 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.575583 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:34.582360 31263 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:34.544687 31330 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:34.602682 31340 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:34.622879 31281 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:34.623149 29903 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:34.624903 30978 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:34.626871 31239 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:34.637676 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 449240us
I20260709 13:04:34.579947 31049 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:34.595734 31360 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:34.647552 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 119489us
I20260709 13:04:34.598871 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:34.630210 31343 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:34.650825 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1640) took 1524 ms. Trace:
I20260709 13:04:34.656397 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:34.646281 31341 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:34.658175 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 153479us
I20260709 13:04:34.648904 31350 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:34.648880 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:34.671247 29905 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:34.654465 30922 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:34.660478 30978 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.664033 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 65584.8us
I20260709 13:04:34.675980 30978 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:34.662362 29216 rpcz_store.cc:276] 0709 13:04:33.123535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.123964 (+   429us) service_pool.cc:224] Handling call
0709 13:04:33.147371 (+ 23407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.147740 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.147766 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:33.288153 (+140387us) log.cc:844] Serialized 26 byte log entry
0709 13:04:34.056870 (+768717us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:34.056935 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.057040 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:34.648214 (+591174us) raft_consensus.cc:1685] finished
0709 13:04:34.648229 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.648341 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1460,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":580294,"threads_started":1,"wal-append.queue_time_us":881110}
I20260709 13:04:34.677482 31263 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.678473 30978 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:34.678740 31317 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 1.576s	user 0.014s	sys 0.023s
W20260709 13:04:34.676445 29185 leader_election.cc:336] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.405s (SENT)
I20260709 13:04:34.680553 31263 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:34.681167 29185 leader_election.cc:336] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.444s (SENT)
I20260709 13:04:34.681617 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:34.675985 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.655823 31281 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:34.686892 29113 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:34.660960 31338 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:34.662207 31365 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:34.661477 31301 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:34.710330 29871 leader_election.cc:336] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.027s (SENT)
I20260709 13:04:34.721437 31049 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:34.708719 31350 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.723109 31338 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.723883 31338 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:34.726195 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39218 (request call id 9) took 1672 ms. Trace:
I20260709 13:04:34.687839 31340 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.726754 29146 rpcz_store.cc:276] 0709 13:04:33.061459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.075255 (+ 13796us) service_pool.cc:224] Handling call
0709 13:04:34.725908 (+1650653us) spinlock_profiling.cc:243] Waited 69.9 ms on lock 0x7b54000429d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b07853 00007f783bad560a 00007f783505b7d4 00007f783505b762 00007f783505b691 00007f783505b60c 00007f783505a8c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:34.726060 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160747392}
I20260709 13:04:34.727577 31340 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:34.729789 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:34.731063 29871 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.102s (SENT)
I20260709 13:04:34.731549 31330 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.732362 31281 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:34.733853 31330 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:34.740607 29871 leader_election.cc:336] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.105s (SENT)
W20260709 13:04:34.742527 29185 leader_election.cc:336] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.974s (SENT)
I20260709 13:04:34.743182 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:34.744246 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 57409us
I20260709 13:04:34.744503 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:34.745157 29113 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:04:34.745893 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 58309.1us
I20260709 13:04:34.746456 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:34.694200 31294 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:34.751748 29214 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:34.752395 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:34.755343 29220 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:34.693132 31263 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:34.746737 31017 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.765462 31294 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:34.766388 31372 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:34.749595 31240 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:34.769934 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:34.805734 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:34.821637 29220 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:34.806545 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:34.822373 29147 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:34.809659 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:34.810583 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:34.803818 29871 leader_election.cc:336] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.168s (SENT)
I20260709 13:04:34.823709 29216 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:34.752948 31363 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:34.818477 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:34.827173 29214 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:34.825969 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:34.827836 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 127108us
I20260709 13:04:34.823853 29213 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:34.753154 31094 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:04:34.840342 31240 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:34.817235 31341 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.844223 29064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:34.853061 29064 raft_consensus.cc:606] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:34.845043 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:34.826066 31049 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:34.817576 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:34.858968 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:34.861274 29217 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:34.808303 31338 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:34.802757 31017 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.828717 31372 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:34.873126 29185 leader_election.cc:336] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 3.188s (SENT)
I20260709 13:04:34.864194 31330 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.891703 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.890614 29217 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:34.856398 31363 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:34.815399 31365 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:34.872354 31372 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:34.940274 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 137025us
I20260709 13:04:34.952284 31276 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:34.952912 31276 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:34.953783 31276 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.955029 31276 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:34.956610 31372 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.958971 31017 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:34.959297 31372 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:34.959769 31294 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:34.903236 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.944675 31363 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.965734 29064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:60678
I20260709 13:04:34.966373 29064 raft_consensus.cc:606] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:34.966686 29064 raft_consensus.cc:614] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:34.967126 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.966868 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:34.931229 31330 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:34.979431 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1637) took 1869 ms. Trace:
I20260709 13:04:34.979900 29222 rpcz_store.cc:276] 0709 13:04:33.110099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.110315 (+   216us) service_pool.cc:224] Handling call
0709 13:04:34.978089 (+1867774us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:34.978300 (+   211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:34.979029 (+   729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:34.979065 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:34.979110 (+    45us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:34.979160 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.979229 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.979364 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6545792}
I20260709 13:04:34.982668 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:34.983285 29217 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:34.989588 29185 leader_election.cc:336] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.936s (SENT)
I20260709 13:04:34.988473 31240 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:34.995287 31240 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:34.995769 31240 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:34.996632 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.002305 31033 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:35.010875 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:35.011116 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:35.012010 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1558) took 1770 ms (client timeout 1675 ms). Trace:
I20260709 13:04:35.012365 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:35.012411 29067 rpcz_store.cc:269] 0709 13:04:33.241449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.011819 (+1770370us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:35.011937 (+   118us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:04:35.011009 29069 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:35.013664 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:35.014305 31338 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:35.014827 31338 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.016899 31338 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:35.020579 29071 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:35.033277 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:35.048460 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.048513 29217 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:35.086786 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:35.088495 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 139938us
I20260709 13:04:35.109586 29071 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:04:35.035005 31371 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.044301 31374 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:35.044536 31378 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:04:35.128103 31374 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:35.128726 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.110370 29066 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:35.115247 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:35.131505 29065 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.131032 31371 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:35.136554 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:35.117267 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 127690us
I20260709 13:04:35.138823 29065 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
W20260709 13:04:35.139469 29185 leader_election.cc:336] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.599s (SENT)
I20260709 13:04:35.140594 29185 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:35.122380 29222 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:35.141573 29073 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:04:35.146250 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 109773us
I20260709 13:04:35.124898 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:35.108604 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:35.040122 31330 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:35.210610 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.210696 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:35.211201 29871 leader_election.cc:336] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.093s (SENT)
I20260709 13:04:35.211328 29070 raft_consensus.cc:2376] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6. Re-sending same reply.
I20260709 13:04:35.211715 29871 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:35.212078 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:35.164552 31341 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:35.212705 29072 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:35.216286 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.216912 29064 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:35.067294 31017 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.048029 31049 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:35.209625 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:35.120843 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:35.208150 29065 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:35.220117 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.227519 29068 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.223906 29072 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:35.208314 31273 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.236408 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.237285 29064 raft_consensus.cc:2376] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6. Re-sending same reply.
I20260709 13:04:35.241345 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.236604 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:35.237826 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:35.242952 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 103557us
I20260709 13:04:35.243156 29073 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:35.244438 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:35.245147 29068 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:35.224190 31343 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:35.226979 31371 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:35.252688 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.252049 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:35.266268 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:35.247332 31365 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:35.240595 31273 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.225371 31374 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:35.276731 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.247465 29906 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:35.243577 31017 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.290750 29910 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:35.286844 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:35.301681 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:35.308472 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:35.316764 29213 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:35.313061 31272 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:35.345530 31272 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.345984 31272 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:35.346417 31394 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:04:35.346778 31273 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:35.349193 31017 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:35.351111 31340 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.623s	user 0.010s	sys 0.003s
I20260709 13:04:35.316076 31049 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:04:35.354494 29902 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:35.376250 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 266) took 1515 ms. Trace:
I20260709 13:04:35.381021 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:35.315263 31346 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:04:35.380916 29143 rpcz_store.cc:276] 0709 13:04:33.861106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.861575 (+   469us) service_pool.cc:224] Handling call
0709 13:04:33.888813 (+ 27238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.971624 (+ 82811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.971674 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:33.971726 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:33.971762 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.356583 (+1384821us) spinlock_profiling.cc:243] Waited 82.6 ms on lock 0x7b54001bb7d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:35.376041 (+ 19458us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:35.376189 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190032768}
I20260709 13:04:35.385030 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:35.386386 31346 consensus_queue.cc:1243] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:35.408277 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:35.394269 31330 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:35.406862 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet c59eab5922dc4c07a3a86e915e3cab7f: Illegal state: Replica 6ad9faa1c14444b29c9db8e71f46e2d6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:04:35.418116 29213 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:35.408190 31390 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:04:35.421777 29185 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:35.423888 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:35.425051 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:35.426115 29147 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:04:35.426409 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:35.427152 29139 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:35.427417 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:35.402400 31228 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:04:35.414573 31350 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.449211 29141 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:35.431448 29139 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:35.436932 31017 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:35.437052 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:35.428478 31348 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.431582 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:35.446732 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:35.446866 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:35.438334 29143 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:04:35.399760 31400 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:04:35.407500 31272 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:35.452158 31348 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.459573 31348 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:35.428664 31294 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.429113 31333 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:35.431102 31279 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:35.481964 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:35.484022 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:35.484730 29139 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:35.485873 29911 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:35.488123 31333 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:35.489089 31333 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.428946 31343 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:04:35.493745 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:35.493937 31343 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:35.494298 29139 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:04:35.422883 31330 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.457541 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:35.497076 31400 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:35.497203 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:35.497828 31400 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:35.498343 29214 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:35.453430 31350 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.461166 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:35.499832 29147 raft_consensus.cc:2376] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3. Re-sending same reply.
I20260709 13:04:35.500433 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:35.501405 31350 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:35.501667 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:35.502614 29144 raft_consensus.cc:2410] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:35.502897 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:35.503271 31330 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:35.503499 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:35.461618 31272 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:35.462967 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:35.463807 31240 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:35.464985 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.466210 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:35.467661 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.429689 31335 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:04:35.468811 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:35.537632 29073 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:35.468979 31406 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.470718 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:35.542307 29215 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.471276 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:35.472607 29217 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:04:35.474045 29220 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:35.472653 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:35.451563 31017 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:35.481383 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.481906 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.494347 29216 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.437762 31340 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:35.504199 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:35.505465 31033 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.007s
I20260709 13:04:35.506752 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.547376 29068 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:35.511215 31350 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:04:35.512064 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 87072.8us
I20260709 13:04:35.519548 29146 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:35.520275 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:35.554173 29910 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.521584 31400 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:35.523108 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:35.494443 31343 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:35.556355 31343 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:35.557703 31343 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.456286 29213 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:35.560283 31335 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:35.560842 31335 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:35.561965 31335 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.563517 31335 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.450192 31228 consensus_queue.cc:1243] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:04:35.532940 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 70097us
I20260709 13:04:35.540493 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:35.548708 29039 leader_election.cc:336] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.105s (SENT)
I20260709 13:04:35.565647 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:35.565865 29214 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:35.566690 31017 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:35.567239 31017 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:35.526120 31290 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:35.568255 31290 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:35.569605 31290 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.533994 31263 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:35.570761 31290 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.570952 31263 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:35.571389 31263 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.574766 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:35.575706 31339 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:35.576162 31339 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.576567 31339 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:35.579624 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1646) took 2320 ms. Trace:
I20260709 13:04:35.581506 29219 rpcz_store.cc:276] 0709 13:04:33.258840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.259064 (+   224us) service_pool.cc:224] Handling call
0709 13:04:33.268023 (+  8959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.268315 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:04:33.657650 (+389335us) log.cc:844] Serialized 14 byte log entry
0709 13:04:35.579044 (+1921394us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b18003e64a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a
0709 13:04:35.579240 (+   196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:35.579287 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:35.579342 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.579409 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:35.579537 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":493730,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2527,"raft.queue_time_us":2003,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":88,"spinlock_wait_cycles":7267200,"thread_start_us":523683,"threads_started":2,"wal-append.queue_time_us":250952,"wal-append.run_cpu_time_us":1205,"wal-append.run_wall_time_us":1034643}
I20260709 13:04:35.576573 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:35.585510 31340 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:35.585986 31340 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:35.586880 31340 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.588289 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.588958 29064 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:35.589860 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:35.590281 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:35.592244 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:35.592666 29066 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.592782 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:35.552443 31406 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:35.595510 31406 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:35.597632 31024 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:35.591667 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:35.600360 29069 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.604986 29066 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
W20260709 13:04:35.606051 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 89144.9us
W20260709 13:04:35.606571 29039 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.758s (SENT)
I20260709 13:04:35.606997 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:35.609380 29216 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:35.485426 28958 catalog_manager.cc:5028] TS 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): tablet 348ae3050d97486fa402a5da2bbdce46 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:35.541100 31294 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.622608 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:35.842201 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.866343 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:35.732896 29215 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:35.728434 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:35.786643 29910 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:35.728739 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:35.416939 28955 catalog_manager.cc:5028] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): tablet 344aa5f73f0f4124b1512832a0bca2c9 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:04:36.008735 29065 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:04:35.753180 29069 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
W20260709 13:04:35.655519 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.000435 29066 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:35.812151 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:35.834867 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.012540 29214 raft_consensus.cc:2410] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:04:36.012681 29219 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
W20260709 13:04:35.653600 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.729125 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:35.835156 29221 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:35.792621 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 172481us
I20260709 13:04:36.004458 29072 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:36.011263 29067 raft_consensus.cc:2376] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2. Re-sending same reply.
I20260709 13:04:35.765990 29073 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:04:36.014730 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 407483us
W20260709 13:04:35.650728 28958 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 165426us
I20260709 13:04:36.012182 29066 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:35.995775 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:36.015983 29066 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:35.620617 31406 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.011827 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:35.683584 31281 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.011s	sys 0.011s
I20260709 13:04:35.734726 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:36.020864 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.027453 29068 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:36.027757 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.018954 29066 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.022605 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:36.023348 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.623188 31290 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:35.623302 31346 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.653095 31343 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.032895 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:36.043777 29070 raft_consensus.cc:2376] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6. Re-sending same reply.
W20260709 13:04:36.025310 31281 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.151s	user 0.021s	sys 0.016s
I20260709 13:04:36.035135 29140 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:36.035260 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:36.040701 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.041085 29219 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:35.764779 30922 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.024742 31406 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.017534 31279 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:36.030380 31134 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:36.028517 31369 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.358s	user 0.001s	sys 0.019s
I20260709 13:04:36.020311 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:36.051688 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:36.058069 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:36.019662 28955 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 602871us
I20260709 13:04:36.047869 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:36.060386 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.053324 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:36.064090 29064 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:36.032991 31330 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.067286 29147 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:36.067888 29069 raft_consensus.cc:2376] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4. Re-sending same reply.
I20260709 13:04:36.040042 30978 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:04:36.036983 31240 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:36.059141 31406 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.045652 31024 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.045869 31272 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.073720 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:36.074544 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:36.076309 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:36.070470 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.052043 31385 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.084565 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:36.085350 29070 raft_consensus.cc:2376] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5. Re-sending same reply.
W20260709 13:04:36.089213 31339 consensus_meta.cc:287] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.005s
I20260709 13:04:36.065064 31279 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:36.089704 31339 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.086243 29068 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:36.095095 29218 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:36.049006 31400 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:04:36.076102 31330 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.104017 31340 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:36.059654 31290 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:04:36.061810 30922 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.104530 31370 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:36.050527 31346 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.077081 29907 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:36.049253 31343 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.092566 31385 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.071480 31281 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:04:36.125118 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.132500 29065 raft_consensus.cc:2410] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:36.072431 31134 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.106834 31339 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:36.094862 31272 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.127616 31400 consensus_queue.cc:1243] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:04:36.095000 31024 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.137503 30922 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:36.138334 29216 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:36.138962 29066 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:04:36.138070 31281 consensus_queue.cc:1243] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:04:36.140942 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.149187 29065 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:36.144883 29066 pending_rounds.cc:85] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 1
W20260709 13:04:36.145653 29188 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.639s (SENT)
I20260709 13:04:36.149940 29066 pending_rounds.cc:107] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:04:36.150314 29066 raft_consensus.cc:2889] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:36.157603 29067 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:36.157249 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.158913 29066 raft_consensus.cc:2410] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:36.156519 31272 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:36.159807 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:36.149034 31263 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.000s
I20260709 13:04:36.160447 29066 raft_consensus.cc:2376] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5. Re-sending same reply.
I20260709 13:04:36.161003 31343 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:36.161378 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.161083 31263 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.162132 29067 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:36.163333 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.163558 31263 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:36.163890 29067 raft_consensus.cc:2376] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5. Re-sending same reply.
I20260709 13:04:36.138267 31385 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.139539 31370 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:36.174213 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.182153 31346 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:36.192991 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.193800 29068 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:36.194589 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.195120 29068 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:36.208621 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.210458 29068 pending_rounds.cc:85] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 4
I20260709 13:04:36.210842 29068 pending_rounds.cc:107] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:04:36.211169 29068 raft_consensus.cc:2889] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:36.208217 29147 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:36.211455 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:36.214745 31335 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.002s
I20260709 13:04:36.215684 29073 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:36.215552 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.219288 29070 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:36.224169 29072 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 5. Current term is 6. Ops: [3.5-5.6]
W20260709 13:04:36.226177 31413 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:04:36.231078 29069 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:36.234165 29213 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:36.234704 29213 pending_rounds.cc:85] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 1
I20260709 13:04:36.234982 29213 pending_rounds.cc:107] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:36.235261 29213 raft_consensus.cc:2889] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:36.237984 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.239373 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.239843 29064 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:36.240196 29221 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:36.243314 29070 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:36.236981 29037 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:36.245760 29069 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:36.246042 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.246740 29219 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:36.246204 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.247895 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:36.248124 31290 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:36.248525 31290 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.248848 31290 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:36.251206 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 197936us
I20260709 13:04:36.266706 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:36.272974 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:36.273834 29215 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:36.275807 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:36.276582 29221 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:36.278676 29219 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:36.279070 29219 pending_rounds.cc:85] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 1
I20260709 13:04:36.279510 29219 pending_rounds.cc:107] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:36.279772 29219 raft_consensus.cc:2889] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:36.280814 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.288373 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:36.278993 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:36.281381 29219 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:36.291976 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:36.292448 29219 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.294466 29215 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:36.295658 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.275080 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:36.297112 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:36.297137 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 254626us
I20260709 13:04:36.296706 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:36.297911 29215 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:36.298277 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:36.298525 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:36.298856 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:36.251796 31024 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:36.299716 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:36.289063 29140 raft_consensus.cc:2376] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6. Re-sending same reply.
I20260709 13:04:36.304852 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:36.305573 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:36.305972 29140 raft_consensus.cc:2376] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6. Re-sending same reply.
I20260709 13:04:36.299731 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:36.298892 29910 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:36.301941 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:36.306722 29908 raft_consensus.cc:2368] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:36.307008 29139 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:36.302532 29144 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:04:36.304122 29217 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.304478 29214 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:04:36.312371 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 71803.2us
I20260709 13:04:36.313112 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:36.313545 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:36.265786 31294 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:36.316658 29186 leader_election.cc:400] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:36.317065 29186 leader_election.cc:403] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:04:36.318343 29148 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:36.319708 29215 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:36.285182 29147 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:36.323096 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:36.288686 31335 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.324354 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:36.314322 29142 raft_consensus.cc:2376] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5. Re-sending same reply.
W20260709 13:04:36.326869 29188 leader_election.cc:336] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.820s (SENT)
I20260709 13:04:36.307615 31365 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:36.324728 30978 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:36.331969 30978 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.332368 30978 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:36.332599 29214 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:36.333698 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.342458 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:36.317708 31371 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:36.351610 29217 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:36.359196 29139 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:36.325583 31233 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:36.332819 31335 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.325392 31330 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:36.364423 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 53807.3us
I20260709 13:04:36.368579 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:36.369433 29145 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:04:36.369493 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 346957us
I20260709 13:04:36.368963 29214 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:04:36.375147 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55549.3us
I20260709 13:04:36.378757 31371 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:36.379890 31371 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.385596 31406 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.386883 29067 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:36.388945 31279 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.390992 31279 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:36.364096 31233 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:36.391975 31233 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:36.393214 29139 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:36.393652 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:36.394227 29143 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:36.394526 29140 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:36.395843 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:36.398748 31290 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.400861 31290 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:36.404254 31425 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:36.405508 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:36.412300 29188 leader_election.cc:336] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.892s (SENT)
I20260709 13:04:36.412977 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:36.413723 29148 raft_consensus.cc:2368] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:36.414575 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:36.414667 30978 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.415366 29144 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:36.417973 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:36.418463 31228 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:36.419476 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:36.420249 31420 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:36.421250 31281 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.421142 31239 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:36.421875 31239 consensus_queue.cc:1243] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:36.421777 31281 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.422397 31239 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:36.423116 31239 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:36.423434 31239 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:36.267292 31370 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.424408 31239 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.419796 31228 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:04:36.425833 31239 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:36.428550 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50396.9us
I20260709 13:04:36.429301 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:36.430316 31408 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:36.430429 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:36.430874 31408 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:36.431738 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:36.431704 31408 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.432946 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:36.433293 31343 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:36.433830 31343 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:36.434310 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:36.434468 30922 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:36.435257 31406 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:36.435359 31339 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:36.437834 31343 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.439908 29219 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:36.432964 31408 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:36.444098 31330 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:36.430348 31281 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:36.444825 31330 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:04:36.445360 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.445916 31330 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:36.446312 31330 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:36.446365 29221 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:36.447052 31263 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:36.412451 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:36.430684 31421 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:36.451501 31134 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:36.451702 30978 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:36.453418 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.455895 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:36.447242 31330 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:36.460249 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 141864us
W20260709 13:04:36.443758 31416 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.218s	user 0.000s	sys 0.003s
I20260709 13:04:36.466756 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:36.470142 29221 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:04:36.464659 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:36.470695 29221 pending_rounds.cc:85] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 2
I20260709 13:04:36.471344 29221 pending_rounds.cc:107] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:04:36.471778 29221 raft_consensus.cc:2889] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:36.466794 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:36.476557 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:36.479358 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:36.480331 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.479749 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.490878 29219 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:36.490154 29216 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:36.498068 31406 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:36.499468 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:36.500484 31017 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:36.501020 31017 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:36.502188 31017 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.503336 31017 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:36.497139 29219 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:36.509852 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:36.441043 31233 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.508720 29113 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:36.475422 31263 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:36.487082 31350 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.523684 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:36.486447 31421 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:04:36.514045 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 47361.7us
I20260709 13:04:36.513319 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.476029 31281 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:36.514652 31408 raft_consensus.cc:2764] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:04:36.514796 31339 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.486732 31424 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:36.515478 31240 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:36.464253 31335 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:36.517066 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.511946 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:36.463793 31134 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:04:36.497327 31330 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.528849 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:36.533793 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.517124 31340 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.519714 31420 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:36.526095 31350 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.545714 29219 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:36.552968 31343 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:36.553745 31343 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.556653 31343 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:36.564924 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:36.581456 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.540580 31339 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:36.597605 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 88940us
I20260709 13:04:36.597887 31370 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.600642 31370 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.600754 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:36.601040 31370 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.602583 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:36.549225 31333 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:36.528167 31263 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.550568 31335 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:36.611792 31335 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.613014 31263 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:36.563905 31350 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.613917 31335 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:36.586858 31406 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:36.544713 31281 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.599467 31339 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:36.628798 31294 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:36.629369 31294 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:36.630298 31294 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.564133 31340 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.619402 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 349) took 1237 ms. Trace:
I20260709 13:04:36.618523 31408 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:36.642983 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 78165us
W20260709 13:04:36.645967 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 136187us
W20260709 13:04:36.651865 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 193654us
I20260709 13:04:36.652915 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:36.657917 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:36.635551 31425 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:36.666779 29218 raft_consensus.cc:2410] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:36.672129 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:36.674346 31263 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:36.641038 29902 rpcz_store.cc:276] 0709 13:04:35.377617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.383072 (+  5455us) service_pool.cc:224] Handling call
0709 13:04:35.416770 (+ 33698us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:35.603312 (+186542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:35.603343 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:35.603851 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:04:35.817115 (+213264us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:35.822944 (+  5829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.841562 (+ 18618us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:36.443204 (+601642us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:36.564604 (+121400us) raft_consensus.cc:1685] finished
0709 13:04:36.583776 (+ 19172us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:36.615437 (+ 31661us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":186071,"raft.queue_time_us":186263,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":34303872,"thread_start_us":189559,"threads_started":1,"wal-append.queue_time_us":570541}
I20260709 13:04:36.686048 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:36.634096 31421 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.691895 31350 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.696177 31425 raft_consensus.cc:2764] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:04:36.698829 31279 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:36.696522 31350 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:36.675125 31333 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:36.679694 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:36.676218 31335 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:04:36.670105 29112 consensus_peers.cc:597] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 79cdfc6a725145348b70034422ef477b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:36.675325 31340 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:36.698980 31339 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.708631 31408 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:36.747377 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:36.722352 31233 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:36.669353 31290 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:36.705832 31385 consensus_meta.cc:287] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.001s
I20260709 13:04:36.680027 31406 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:04:36.703400 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:36.733827 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 94246.2us
I20260709 13:04:36.754761 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:36.735414 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 129661us
I20260709 13:04:36.721498 31281 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:36.746922 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:36.756446 29113 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:36.757269 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:36.774330 29911 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:36.755686 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 83867.1us
I20260709 13:04:36.774835 29113 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:36.747319 31279 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.778040 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:36.779007 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 99418.4us
W20260709 13:04:36.777753 29039 leader_election.cc:336] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.554s (SENT)
I20260709 13:04:36.783605 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:36.755137 31263 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:36.778301 31443 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:36.786353 31424 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.783181 31343 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:36.805843 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:36.809891 31339 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:36.812602 31350 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:36.812320 31343 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.811846 31424 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.816847 28969 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 261) took 5116 ms. Trace:
I20260709 13:04:36.817171 31370 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.756122 31385 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.818667 31424 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:36.819382 31370 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:36.820382 31385 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:36.803803 31449 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:36.821935 31449 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.822321 31449 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:36.821465 31339 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:36.825814 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:36.786115 31279 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.811235 31333 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.813544 31443 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:36.827718 31263 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:36.908970 31435 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.231s	user 0.000s	sys 0.002s
I20260709 13:04:36.908789 31263 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.829030 31134 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:36.916394 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 90764.7us
I20260709 13:04:36.817312 28969 rpcz_store.cc:276] 0709 13:04:31.700620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.702881 (+  2261us) service_pool.cc:224] Handling call
0709 13:04:36.816754 (+5113873us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:32.947585 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:32.948293 (+   708us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:32.948349 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:04:32.948381 (+    32us) tablet.cc:662] Decoding operations
0709 13:04:32.949573 (+  1192us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:32.949652 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:32.949678 (+    26us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:04:33.008712 (+ 59034us) tablet.cc:701] Row locks acquired
0709 13:04:33.008759 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:04:33.008936 (+   177us) write_op.cc:270] Start()
0709 13:04:33.009078 (+   142us) write_op.cc:276] Timestamp: P: 1783602273008867 usec, L: 0
0709 13:04:33.009101 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:04:33.009792 (+   691us) log.cc:844] Serialized 4800 byte log entry
0709 13:04:33.349244 (+339452us) op_driver.cc:464] REPLICATION: finished
0709 13:04:33.811342 (+462098us) write_op.cc:301] APPLY: starting
0709 13:04:33.811445 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:33.811733 (+   288us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:33.811868 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:33.840403 (+ 28535us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:33.840694 (+   291us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:04:33.840745 (+    51us) write_op.cc:312] APPLY: finished
0709 13:04:33.841158 (+   413us) log.cc:844] Serialized 89 byte log entry
0709 13:04:33.841356 (+   198us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:33.841521 (+   165us) write_op.cc:454] Released schema lock
0709 13:04:33.841610 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:33.841999 (+   389us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:33.842029 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14668,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":461768,"apply.run_cpu_time_us":4240,"apply.run_wall_time_us":30636,"num_ops":12,"prepare.queue_time_us":149593,"prepare.run_cpu_time_us":9389,"prepare.run_wall_time_us":103191,"raft.queue_time_us":227595,"raft.run_cpu_time_us":20778,"raft.run_wall_time_us":349119,"replication_time_us":339934,"thread_start_us":633496,"threads_started":4,"wal-append.queue_time_us":68739,"wal-append.run_cpu_time_us":27368,"wal-append.run_wall_time_us":1848169}]]}
I20260709 13:04:36.919261 31343 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:36.920715 31443 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.926436 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:36.921393 31279 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:36.828364 31447 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:36.908231 31424 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:36.915525 31411 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:04:36.828822 31339 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.932857 31459 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:36.932963 31263 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:36.914361 31448 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:36.923137 31272 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:36.937086 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 121311us
I20260709 13:04:36.928417 29911 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:36.907402 31333 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:36.958343 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:36.960268 31333 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:36.931608 31447 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.970813 31447 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:36.947156 31442 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.266s	user 0.000s	sys 0.009s
I20260709 13:04:36.949460 31339 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:36.952132 31448 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.935619 31424 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.932163 31444 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:36.928443 31134 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.956743 31459 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:36.986948 29113 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:36.991899 31448 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:36.992321 31432 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.480s	user 0.001s	sys 0.003s
I20260709 13:04:36.993654 31456 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:36.993969 31459 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:36.994124 31456 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.994498 31456 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:36.998801 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 72449.2us
I20260709 13:04:37.001279 31444 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:37.006978 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:37.007683 29221 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:37.001729 31444 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.006762 31439 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:37.008762 31444 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:37.010792 31424 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:37.015214 29221 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:37.017040 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:37.017606 29221 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:37.018523 31272 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:04:37.026551 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:37.038329 29911 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:37.042939 31438 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:37.040591 31439 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:37.049044 31439 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:37.049672 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:37.049903 31460 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:37.051347 31370 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:37.053050 31370 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:04:37.056855 29188 leader_election.cc:336] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.948s (SENT)
I20260709 13:04:37.060449 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:37.060596 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.017796 31294 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:04:37.066239 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:37.043953 31438 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:37.066931 31228 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:37.045971 31444 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:37.094139 29902 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:37.094480 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:37.088105 31460 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:04:37.097105 31408 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:04:37.097699 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 394377us
I20260709 13:04:37.133697 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:37.134603 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.098330 31446 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:37.099160 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 321421us
I20260709 13:04:37.135658 29038 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:37.138950 29039 leader_election.cc:336] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.627s (SENT)
I20260709 13:04:37.139889 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.099500 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 230) took 1022 ms. Trace:
I20260709 13:04:37.099676 31024 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:04:37.100719 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 109055us
W20260709 13:04:37.101928 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.064826 31458 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:37.109040 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.118278 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.067619 31461 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:37.141829 29071 rpcz_store.cc:276] 0709 13:04:36.076702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.232788 (+156086us) service_pool.cc:224] Handling call
0709 13:04:36.252607 (+ 19819us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.252840 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.252856 (+    16us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:36.289929 (+ 37073us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.519731 (+229802us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:36.589212 (+ 69481us) log.cc:844] Serialized 14 byte log entry
0709 13:04:36.737290 (+148078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.737669 (+   379us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.097463 (+359794us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b18004728a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:04:37.099191 (+  1728us) raft_consensus.cc:1685] finished
0709 13:04:37.099294 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.099441 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19287,"prepare.queue_time_us":21595,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":44,"raft.queue_time_us":92872,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":21151,"spinlock_wait_cycles":13029760,"thread_start_us":381744,"threads_started":3,"wal-append.queue_time_us":293651}
I20260709 13:04:37.068511 31239 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:04:37.068890 31449 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:37.125048 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:37.068631 31272 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:04:37.150936 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:37.155031 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:37.161307 29069 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:04:37.161736 29069 pending_rounds.cc:85] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 3
I20260709 13:04:37.161962 29069 pending_rounds.cc:107] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:04:37.162245 29069 raft_consensus.cc:2889] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:37.130574 31443 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:37.165031 29188 leader_election.cc:400] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:37.143532 31228 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:37.137761 31446 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.180684 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:37.146927 29906 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:37.138314 31408 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:37.150496 31400 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:37.150585 31017 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:37.182123 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:37.201726 29064 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:37.099515 31456 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:37.124205 31335 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:04:37.165563 31024 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.189836 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:37.190106 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:37.190869 31134 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:37.191241 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:37.192301 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:37.191879 31228 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:37.192979 31446 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:37.193897 29068 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:37.195428 31450 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:37.170744 31443 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.156627 31423 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:37.167835 30922 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 13:04:37.187471 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:37.157334 31459 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:37.164197 31330 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.168033 31339 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:37.167291 31279 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.152408 31445 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:37.201427 31017 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:37.150259 31464 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.123s
I20260709 13:04:37.280299 29148 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 5. Ops: []
I20260709 13:04:37.157645 31290 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:04:37.253333 29071 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:37.253859 29065 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:37.280227 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.184619 31458 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:37.150507 31424 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:37.260105 31423 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:04:37.261592 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 74329.7us
I20260709 13:04:37.167662 31281 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 13:04:37.184846 31447 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 13:04:37.257838 31437 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 13:04:37.254686 31466 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:04:37.269536 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.269986 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.204459 31343 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.273047 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.205277 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:37.252794 29066 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:37.257961 31443 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:37.388027 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.252944 29064 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:04:37.253345 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 46419.2us
I20260709 13:04:37.389285 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.389827 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.280141 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.254717 31450 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:04:37.268576 31428 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:04:37.281545 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.282496 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:37.282711 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.286231 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:37.404500 29070 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.282116 31017 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:37.275743 31279 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:37.272372 31339 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.295079 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:37.292595 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:37.278352 31400 raft_consensus.cc:2764] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:04:37.299911 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 114751us
I20260709 13:04:37.257030 31024 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:37.265810 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 156830us
I20260709 13:04:37.302605 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.348681 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:37.300640 31456 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:37.301312 31449 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:37.272300 31330 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:37.288396 31438 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:37.366732 31416 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:04:37.273350 31408 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:37.302165 31448 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:04:37.349681 31423 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:37.428066 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:37.380401 29072 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:37.380336 29143 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 2.
I20260709 13:04:37.384140 29903 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:37.435505 29067 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:37.384873 31343 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:37.386355 29066 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:37.386922 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:37.390733 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.408442 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.409566 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.438035 29140 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:37.410153 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.410336 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:37.404808 31473 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.110s	user 0.001s	sys 0.001s
I20260709 13:04:37.412387 31444 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:37.428135 31339 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.412918 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.403069 31450 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:37.406001 31290 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.418912 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:37.419281 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.429231 29113 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:37.448491 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.449121 29141 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:37.451139 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:37.365509 31458 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:37.434294 29038 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:37.437422 29073 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.440375 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.439419 29064 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:37.412700 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:37.453611 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.458580 29146 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:37.452626 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.455298 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:37.459332 29216 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:37.459661 29222 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.460284 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:37.437510 31406 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:37.456627 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.456902 29217 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:37.461871 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:37.462118 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.462921 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:37.457252 29219 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:37.457818 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:37.464249 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.454402 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:37.465102 29072 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.462543 29217 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:37.467163 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.465943 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:37.469298 29188 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:37.471740 29070 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:37.473869 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:37.418550 31476 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:04:37.476241 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.463455 29144 raft_consensus.cc:2376] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6. Re-sending same reply.
I20260709 13:04:37.452745 31444 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:37.483464 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.484321 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.480444 29188 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:37.485594 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:37.490284 29066 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:37.491389 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.452441 31290 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:37.461910 31406 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:37.493636 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.461683 31281 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:37.494189 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.445499 31445 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:04:37.466334 31456 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:37.477185 31400 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:37.487725 31444 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.466746 31350 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:04:37.501255 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:37.522805 29220 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:37.498764 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.499639 31464 raft_consensus.cc:2764] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:04:37.504415 28971 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 3 to 5, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 5 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.480876 31448 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:37.499330 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.507155 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:37.487214 31272 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:37.525336 28971 catalog_manager.cc:5697] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 3 to 4, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 4 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:37.542712 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44485.3us
I20260709 13:04:37.543519 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.543413 28971 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 2 to 5, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 5 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.545794 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.546787 29066 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:37.487491 31439 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:04:37.515161 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.483963 31481 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:37.515894 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:37.499207 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.493834 31425 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:04:37.476958 31449 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:04:37.484021 31294 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:04:37.532312 29146 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:37.511089 29072 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:37.506395 31406 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.533015 31448 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:37.509359 31456 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.497741 29222 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
W20260709 13:04:37.554505 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 96754.6us
I20260709 13:04:37.611739 29220 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:37.520294 31444 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:37.554036 29144 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:37.542083 31281 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:37.544617 28971 catalog_manager.cc:5697] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 2 to 3, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.550626 31272 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:37.511761 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45903.4us
I20260709 13:04:37.489434 31463 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:04:37.553498 31335 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:37.553894 31447 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:37.640681 29148 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:37.498471 31478 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 13:04:37.611163 29906 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:37.643108 29220 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:04:37.641021 31456 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:37.556468 31263 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:37.633518 31448 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:37.653270 28971 catalog_manager.cc:5697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 2, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:37.641212 31024 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:37.649693 31272 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:37.660502 31279 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:37.703161 31488 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:37.703447 31487 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:04:37.703148 31436 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:04:37.705732 31483 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:04:37.666714 31330 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:37.673669 29220 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:37.719796 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.680436 28971 catalog_manager.cc:5697] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 2 to 3, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.783196 28971 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 3, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.785432 28971 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 2 to 3, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:37.786851 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 67173.3us
I20260709 13:04:37.787653 28971 catalog_manager.cc:5697] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.788197 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:37.687868 31481 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.686710 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 453) took 1149 ms. Trace:
I20260709 13:04:37.668232 31281 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.122s
I20260709 13:04:37.805155 29220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:37.793437 28971 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 4 to 5, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 5 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.794776 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:37.805974 29220 raft_consensus.cc:606] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:37.806607 29220 raft_consensus.cc:614] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:37.810566 28971 catalog_manager.cc:5697] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.814182 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:37.797245 31400 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:37.847841 29220 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:37.849620 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:37.850833 29221 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:37.855688 29142 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:04:37.865274 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.801498 31481 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:37.853670 31290 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:37.893055 29213 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:37.854538 31423 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:37.829659 31343 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:37.907861 29073 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:37.916941 29073 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:37.925369 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.801692 29906 rpcz_store.cc:276] 0709 13:04:36.522710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.525072 (+  2362us) service_pool.cc:224] Handling call
0709 13:04:37.094345 (+569273us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8eee 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:37.671593 (+577248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4791168}
I20260709 13:04:37.895121 31486 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:04:37.919556 31017 consensus_meta.cc:287] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.001s
W20260709 13:04:37.952544 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 142491us
I20260709 13:04:37.955206 29073 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:37.957787 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:37.960667 29142 raft_consensus.cc:2410] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:37.951967 31279 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:37.954317 31239 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.984642 31500 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:37.992848 31461 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:37.993448 31461 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:37.994598 31461 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:37.949231 31017 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.014099 29065 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:38.015870 31446 consensus_meta.cc:287] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.751s	user 0.008s	sys 0.003s
I20260709 13:04:38.016347 31446 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.018767 31446 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:38.023818 29069 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:04:38.024698 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.028093 31481 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:38.043334 31239 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:38.015542 31017 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:38.045889 31430 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:38.080147 31430 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:38.081359 31430 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:38.083150 31430 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:38.024716 31443 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.627s	user 0.008s	sys 0.000s
I20260709 13:04:38.048184 31461 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:38.100963 29142 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:04:38.093480 31339 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.003s
W20260709 13:04:38.121050 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 159457us
W20260709 13:04:38.122964 29215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.008s	sys 0.016s
W20260709 13:04:38.123366 29215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.008s	sys 0.016s
I20260709 13:04:38.084297 31443 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.126251 29113 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:38.128347 29113 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:38.129475 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:38.079363 31444 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:38.133831 29112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.005s	sys 0.003s
W20260709 13:04:38.134161 29112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.005s	sys 0.003s
I20260709 13:04:38.137368 29147 raft_consensus.cc:2368] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:04:38.122025 31272 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.124142 31339 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.139186 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:38.524214 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:38.524797 31339 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:38.525285 29148 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:38.491478 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1694) took 1034 ms. Trace:
I20260709 13:04:38.135488 31406 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.619967 29216 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:04:38.164724 29038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.007s	sys 0.009s
W20260709 13:04:38.626777 29038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.031s	sys 0.019s
W20260709 13:04:38.627264 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 982046us
I20260709 13:04:38.628186 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:38.199970 29036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.007s	sys 0.017s
W20260709 13:04:38.629204 29036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.018s	sys 0.028s
W20260709 13:04:38.448575 29185 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.536s (SENT) [suppressed 43 similar messages]
I20260709 13:04:38.132802 31496 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:38.118366 31448 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.527274 29215 rpcz_store.cc:276] 0709 13:04:37.456805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.456920 (+   115us) service_pool.cc:224] Handling call
0709 13:04:37.457236 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:37.457562 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.457589 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.472567 (+ 14978us) log.cc:844] Serialized 26 byte log entry
0709 13:04:38.123867 (+651300us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:38.131989 (+  8122us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.132221 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.132321 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:38.491127 (+358806us) raft_consensus.cc:1685] finished
0709 13:04:38.491159 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.491389 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7816,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":493,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":18440,"thread_start_us":614433,"threads_started":1,"wal-append.queue_time_us":961718}
I20260709 13:04:38.132582 31510 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:04:38.670073 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.202553 31503 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.670763 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:38.134796 31437 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.534s
I20260709 13:04:38.630570 31239 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:38.136495 31476 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.539s
I20260709 13:04:38.147776 31439 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:04:38.675510 31456 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.986s	user 0.013s	sys 0.000s
I20260709 13:04:38.698400 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1673) took 1311 ms. Trace:
I20260709 13:04:38.731894 29067 rpcz_store.cc:276] 0709 13:04:37.387318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.514887 (+127569us) service_pool.cc:224] Handling call
0709 13:04:37.546925 (+ 32038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:37.547375 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.547416 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.552374 (+  4958us) log.cc:844] Serialized 26 byte log entry
0709 13:04:38.130407 (+578033us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:38.130448 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.130504 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:38.698110 (+567606us) raft_consensus.cc:1685] finished
0709 13:04:38.698139 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.698319 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5154,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":41540,"thread_start_us":497400,"threads_started":1,"wal-append.queue_time_us":1089238}
I20260709 13:04:38.699330 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1604) took 1217 ms. Trace:
W20260709 13:04:38.700650 31516 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 13:04:38.707418 31339 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.708662 31505 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.546s	user 0.000s	sys 0.000s
I20260709 13:04:38.745844 31443 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:38.746718 31443 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.747164 31443 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:38.748625 29219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:38.749264 29219 raft_consensus.cc:606] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:38.717654 31406 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:38.721444 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:38.723984 29147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1876) took 1431 ms (client timeout 1897 ms). Trace:
I20260709 13:04:38.724550 31017 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:38.727880 29215 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:38.734059 29144 rpcz_store.cc:276] 0709 13:04:37.481774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.644709 (+162935us) service_pool.cc:224] Handling call
0709 13:04:37.708891 (+ 64182us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:37.916470 (+207579us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.916505 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:37.921256 (+  4751us) log.cc:844] Serialized 52 byte log entry
0709 13:04:38.136155 (+214899us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:38.136202 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.136272 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:38.699063 (+562791us) raft_consensus.cc:1685] finished
0709 13:04:38.699101 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.699261 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":681,"raft.queue_time_us":169971,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":27454,"thread_start_us":367131,"threads_started":2,"wal-append.queue_time_us":762687}
I20260709 13:04:38.681133 31503 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:38.744547 31450 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.288s	user 0.009s	sys 0.004s
I20260709 13:04:38.731197 31239 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.786782 31339 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.820386 31438 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:38.787786 31476 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:38.820809 31438 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:38.821031 31476 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:38.824623 29213 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:38.821810 31339 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:38.819854 31239 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:38.793452 31491 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.843s	user 0.009s	sys 0.012s
I20260709 13:04:38.828406 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1866) took 1554 ms. Trace:
I20260709 13:04:38.828876 29139 rpcz_store.cc:276] 0709 13:04:37.273723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.274205 (+   482us) service_pool.cc:224] Handling call
0709 13:04:37.274526 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:37.274910 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.274953 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.417834 (+142881us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b2c00010da8. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:04:37.418262 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:04:37.686560 (+268298us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:37.686599 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.686671 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:38.792738 (+1106067us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:38.828154 (+ 35416us) raft_consensus.cc:1685] finished
0709 13:04:38.828183 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.828335 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":494,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":6560,"spinlock_wait_cycles":10396288,"thread_start_us":267902,"threads_started":1,"wal-append.queue_time_us":451354}
W20260709 13:04:38.831399 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 110088us
I20260709 13:04:38.797714 31408 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:38.740033 31507 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.604s	user 0.001s	sys 0.000s
I20260709 13:04:38.814801 31480 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.857313 29147 rpcz_store.cc:269] 0709 13:04:37.292870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.389614 (+ 96744us) service_pool.cc:224] Handling call
0709 13:04:38.723687 (+1334073us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b54001bbcd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:38.723882 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1717223168}
I20260709 13:04:38.750401 31456 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.824769 31444 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:38.897817 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1675) took 1479 ms. Trace:
I20260709 13:04:38.828510 31239 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:38.898304 29066 rpcz_store.cc:276] 0709 13:04:37.418213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.547662 (+129449us) service_pool.cc:224] Handling call
0709 13:04:37.549281 (+  1619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:37.626140 (+ 76859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.626166 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.631257 (+  5091us) log.cc:844] Serialized 26 byte log entry
0709 13:04:38.127813 (+496556us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:38.127852 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.127942 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:38.897417 (+769475us) raft_consensus.cc:1685] finished
0709 13:04:38.897443 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.897754 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2896,"raft.queue_time_us":56355,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":13867,"thread_start_us":455990,"threads_started":1,"wal-append.queue_time_us":630168}
I20260709 13:04:38.875445 31339 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:38.907790 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.818101 31450 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:38.788575 31439 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:38.821715 31438 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:38.909451 29215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "813e480732fd4061837e784d4d768b1f"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:38.910540 29141 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:04:38.875890 31294 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:38.920643 29215 raft_consensus.cc:606] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:38.928794 29215 raft_consensus.cc:614] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:38.912016 31430 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.828s	user 0.016s	sys 0.000s
I20260709 13:04:38.902439 31515 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:38.938696 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1881) took 1477 ms. Trace:
I20260709 13:04:38.911764 31339 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.910969 31239 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:38.939318 29145 rpcz_store.cc:276] 0709 13:04:37.460946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.645642 (+184696us) service_pool.cc:224] Handling call
0709 13:04:37.645963 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:37.646283 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.646320 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.659840 (+ 13520us) log.cc:844] Serialized 26 byte log entry
0709 13:04:37.949774 (+289934us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:37.949821 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.949895 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:38.938434 (+988539us) raft_consensus.cc:1685] finished
0709 13:04:38.938471 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.938634 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":182053,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"thread_start_us":150151,"threads_started":1,"wal-append.queue_time_us":339487}
I20260709 13:04:38.896821 31480 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.944495 31339 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:38.821846 31476 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.937294 31438 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:38.956158 31443 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:38.939004 31294 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:38.958323 31272 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:38.958809 31294 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:38.943622 31515 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.962117 31515 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:38.967392 31530 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.967943 31530 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:38.970062 31530 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:38.971119 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:38.972262 29215 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:38.974428 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:38.974968 29220 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:38.974861 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:38.975509 29221 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:38.976776 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:38.977444 29219 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:38.970536 31530 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:38.984344 29221 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:38.981899 29220 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:38.986181 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:38.986238 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:38.964056 31448 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:38.990895 29219 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:38.992697 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:38.950584 31239 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:38.993845 31437 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:38.994692 31437 consensus_queue.cc:1243] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:38.995026 31239 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:38.995265 31294 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:38.995755 31294 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:38.996416 31520 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 13:04:38.997025 31461 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.899s	user 0.008s	sys 0.004s
I20260709 13:04:38.996794 31294 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:38.997536 31461 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.998348 31461 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.000118 29064 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:39.000669 31461 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:39.001685 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1676) took 1583 ms. Trace:
I20260709 13:04:39.003026 31450 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.002051 29072 rpcz_store.cc:276] 0709 13:04:37.418473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.629356 (+210883us) service_pool.cc:224] Handling call
0709 13:04:37.629737 (+   381us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:37.643873 (+ 14136us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.643897 (+    24us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:37.708443 (+ 64546us) log.cc:844] Serialized 52 byte log entry
0709 13:04:38.193834 (+485391us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:38.675148 (+481314us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.974618 (+299470us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.974688 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.001467 (+ 26779us) raft_consensus.cc:1685] finished
0709 13:04:39.001486 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.001625 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8196,"prepare.queue_time_us":704571,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":1464,"raft.queue_time_us":166878,"raft.run_cpu_time_us":409,"raft.run_wall_time_us":74782,"spinlock_wait_cycles":4222208,"thread_start_us":1096625,"threads_started":3,"wal-append.queue_time_us":1015997}
I20260709 13:04:39.003958 31488 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:39.004407 31488 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:38.980672 31279 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:38.966701 31439 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.011842 31488 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.013226 31480 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:39.013815 31503 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:39.013978 31501 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.103s	user 0.009s	sys 0.011s
I20260709 13:04:39.014703 31456 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:39.014689 31488 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:39.016330 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1665) took 1761 ms. Trace:
I20260709 13:04:39.016330 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:39.016841 29068 rpcz_store.cc:276] 0709 13:04:37.255114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.255284 (+   170us) service_pool.cc:224] Handling call
0709 13:04:37.266639 (+ 11355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:37.378719 (+112080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.378740 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.403453 (+ 24713us) log.cc:844] Serialized 26 byte log entry
0709 13:04:37.801783 (+398330us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:37.801831 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.801947 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.013446 (+1211499us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:39.016047 (+  2601us) raft_consensus.cc:1685] finished
0709 13:04:39.016068 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.016266 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7944,"raft.queue_time_us":29677,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":2421,"spinlock_wait_cycles":13133568,"thread_start_us":465477,"threads_started":2,"wal-append.queue_time_us":405587}
I20260709 13:04:39.020442 31339 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.027674 29064 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:39.029363 31339 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.036648 29147 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:39.037606 29221 raft_consensus.cc:2410] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:04:39.038623 29140 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:04:39.039238 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1872) took 1753 ms. Trace:
I20260709 13:04:39.039521 29140 rpcz_store.cc:276] 0709 13:04:37.286057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.498454 (+212397us) service_pool.cc:224] Handling call
0709 13:04:37.498832 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.033146 (+1534314us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400296e20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a84dc0 00007f7836b0f1c0 00007f7836b10231 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c
0709 13:04:39.039020 (+  5874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.039188 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241225600}
W20260709 13:04:39.040632 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 592069us
I20260709 13:04:39.043665 29221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
W20260709 13:04:39.044085 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.044340 29221 raft_consensus.cc:606] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:39.048698 31478 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.050019 31478 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.052829 31478 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.053485 29213 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:39.055828 31425 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:39.056172 31339 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.056413 31518 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:39.056849 31339 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.057134 31518 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.056500 31425 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.058974 31294 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:39.059273 31425 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:38.987433 31530 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.060515 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:39.040846 31456 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:39.040640 31436 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:39.042639 31503 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:04:39.065308 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:39.069216 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.045542 31439 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:04:39.078999 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1607) took 1594 ms. Trace:
I20260709 13:04:39.063954 31456 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:39.062115 31510 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:39.080637 29141 rpcz_store.cc:276] 0709 13:04:37.485028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.856642 (+371614us) service_pool.cc:224] Handling call
0709 13:04:37.873394 (+ 16752us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:38.930338 (+1056944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.078690 (+148352us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400145854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:39.078920 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2380084608}
I20260709 13:04:39.063180 31339 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:39.085789 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 415078us
I20260709 13:04:39.086835 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:39.089792 29145 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:39.088609 29113 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:39.078127 31239 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:39.097981 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:39.087586 31456 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:39.079658 31518 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.071873 31436 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:39.110086 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 307820us
I20260709 13:04:39.075487 31503 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:39.126358 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.126904 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:39.132012 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:39.145015 31510 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.091876 31370 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:04:39.148327 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:39.150856 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.152390 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:39.172474 31339 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:39.174645 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.174712 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.175318 29068 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:39.121834 31443 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:39.125591 31518 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:39.177328 31518 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.125754 31134 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:39.180073 29218 raft_consensus.cc:2368] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:04:39.184183 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:39.109611 31239 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.188602 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:39.180850 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 209535us
I20260709 13:04:39.200467 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.203208 29066 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:04:39.209187 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 76820.6us
I20260709 13:04:39.214339 31535 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:39.215770 31535 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.218421 31535 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:39.229518 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.231088 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.233834 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.228904 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.235688 29070 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:04:39.236908 29185 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.928s (SENT)
I20260709 13:04:39.237370 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:39.228853 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 185290us
I20260709 13:04:39.240965 29213 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:04:39.242902 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 74184.7us
I20260709 13:04:39.245628 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:39.246634 29218 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:39.247430 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.248297 29069 raft_consensus.cc:2410] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:04:39.187803 31134 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:39.251713 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:39.251739 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.202577 31443 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:39.253623 29218 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:39.253903 29071 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:04:39.212342 31496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.011s	sys 0.004s
I20260709 13:04:39.282997 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:39.284157 29070 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:39.285058 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.286010 29069 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:39.286959 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.288224 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.289620 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.290374 29067 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:39.241492 31319 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.287726 29069 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:39.293931 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.300983 29066 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:39.301149 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.301874 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.302430 29069 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:39.303193 29066 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:39.295105 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.304291 29213 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:39.304394 29068 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:04:39.305351 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.305938 29213 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:39.306763 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.295905 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.308252 29071 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:39.186841 31339 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.313452 29071 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:39.307785 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:39.311061 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.314939 29067 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:39.242462 31532 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:39.312448 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.315510 29218 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:39.316120 29070 raft_consensus.cc:2410] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:39.316896 29213 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:04:39.284425 31496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.011s	sys 0.004s
I20260709 13:04:39.317050 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:39.319988 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 70052.2us
I20260709 13:04:39.320521 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.320757 29218 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:39.321178 29072 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:39.321878 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.281096 31530 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:39.337409 31530 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:39.337754 31532 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:39.338009 31134 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.338560 31532 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.339785 31503 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.325093 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.325310 29071 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:04:39.327253 31518 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.336372 31339 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.344883 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:39.335482 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:39.344882 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:39.347802 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:39.348495 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.348871 29144 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:04:39.348932 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.299423 31535 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:39.353572 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:39.365221 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:39.356164 31532 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:39.361056 29216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1702) took 1902 ms (client timeout 1593 ms). Trace:
I20260709 13:04:39.372547 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.319555 31496 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:39.363417 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:39.310602 31540 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.363502 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.380716 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.381454 29071 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:39.337808 31239 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:39.345556 31449 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:39.369423 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:39.373100 29216 rpcz_store.cc:269] 0709 13:04:37.458709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.460084 (+  1375us) service_pool.cc:224] Handling call
0709 13:04:39.360869 (+1900785us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400283cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:39.360991 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2449714048}
I20260709 13:04:39.361438 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.382962 29068 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:39.373646 29070 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:39.384398 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.385133 29070 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:39.373126 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.386171 29069 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:39.376858 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.374248 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.387877 29066 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:39.388283 29067 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:39.380138 31464 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:04:39.396171 31515 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.396761 31515 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.399302 31515 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:39.374315 31535 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.363548 31448 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:39.310765 31319 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.405510 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:39.406394 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.407248 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.403404 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.369592 31339 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:39.434182 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.435560 29067 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:39.436111 29066 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:39.412905 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.438138 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:39.439415 31537 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:04:39.404093 31540 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.442600 31487 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:39.444222 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:39.448086 29217 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:04:39.444283 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.412413 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.380399 31461 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:39.451778 29068 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 3: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:39.456449 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.457417 29070 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:39.458168 29069 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:39.403339 31450 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:39.452390 29071 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:39.389776 31449 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:39.467180 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 224340us
I20260709 13:04:39.474426 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.475592 29070 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:39.476702 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.410836 31535 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:39.446679 31532 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:39.447533 31531 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:39.414573 31517 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:39.498001 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:39.488078 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 166647us
I20260709 13:04:39.498919 29071 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
W20260709 13:04:39.491865 29037 connection.cc:256] RPC call timeout handler was delayed by -1.4599941110000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
I20260709 13:04:39.493185 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:39.454226 31461 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.499679 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.500815 29069 raft_consensus.cc:2410] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:39.501809 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.464756 31406 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.445698 31503 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 13:04:39.514926 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.434379 31495 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:04:39.468786 31436 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.503341 29037 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:39.525303 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56892.2us
I20260709 13:04:39.527585 31480 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:39.528184 31480 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:39.529067 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.449393 31339 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.402531 31443 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.532455 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.521106 31504 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:39.579418 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.580029 29070 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:39.484843 31450 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:39.532823 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.584206 29071 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:39.533587 29069 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:39.517839 29874 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:39.586076 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.586660 29069 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
W20260709 13:04:39.523646 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 150976us
I20260709 13:04:39.529033 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:39.538580 29067 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:39.513552 31408 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:39.576637 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 57379.6us
I20260709 13:04:39.513257 31488 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:39.493723 31532 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:39.503726 31461 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:39.457844 31487 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.589919 29222 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
W20260709 13:04:39.576506 31491 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:04:39.587687 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.597437 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.598090 29069 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:39.598258 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.598870 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.599486 29069 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:39.599587 29067 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:04:39.600828 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 71872.5us
I20260709 13:04:39.603652 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:39.604285 29911 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:39.600243 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 82528.8us
I20260709 13:04:39.611023 29071 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:39.582564 31370 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:39.522702 31406 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:39.641469 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50925.7us
I20260709 13:04:39.636778 29069 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:39.643573 29070 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:39.645025 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.645936 29069 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:39.647083 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.651584 31480 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.652446 31450 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.516336 31496 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.510279 31531 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.657964 31531 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:39.600944 31350 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:04:39.661198 31458 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:39.591157 31533 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 13:04:39.601696 31408 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.662763 31458 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:04:39.678740 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:39.632236 31294 consensus_meta.cc:287] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.008s
I20260709 13:04:39.631016 31541 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:39.580444 31550 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:04:39.637457 31239 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:39.602053 31448 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.094s
W20260709 13:04:39.723276 29064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1679) took 2297 ms (client timeout 2192 ms). Trace:
I20260709 13:04:39.697758 31541 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:39.723721 29064 rpcz_store.cc:269] 0709 13:04:37.425689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.488196 (+ 62507us) service_pool.cc:224] Handling call
0709 13:04:39.723047 (+2234851us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400186354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:39.723180 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3373415808}
I20260709 13:04:39.724418 31541 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:39.511842 31517 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.726183 31524 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:04:39.726627 31456 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.605s	user 0.004s	sys 0.004s
I20260709 13:04:39.727171 31456 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.726598 31524 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:39.708271 31239 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.728549 31239 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:39.728778 31524 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.730139 31524 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:39.731734 31542 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.198s	user 0.000s	sys 0.000s
I20260709 13:04:39.721639 31339 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.733037 31339 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:04:39.693799 31497 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:04:39.693054 31458 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:39.735071 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.777477 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:39.778512 29070 raft_consensus.cc:2410] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:04:39.740348 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:39.739646 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:39.779649 29064 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:39.779889 29218 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:39.745287 29911 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:39.605753 31488 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:39.774519 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 110710us
I20260709 13:04:39.689157 31370 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:39.798213 29145 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:39.750944 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.799855 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.783265 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.789388 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.802606 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:39.803421 29218 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:39.797652 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:39.805135 29071 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 2.
I20260709 13:04:39.796455 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.806206 29070 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:39.807041 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.807742 29070 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:39.809882 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:39.579322 31504 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.814477 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:39.815426 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:39.816591 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.733533 31339 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.733242 31449 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.819893 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:39.829770 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:39.820176 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.830760 29908 raft_consensus.cc:2410] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:04:39.820873 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:39.797308 29906 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:39.734047 31540 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:39.851292 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:39.798451 31515 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:39.830884 29069 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:39.841895 29064 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:39.819141 29038 leader_election.cc:400] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:04:39.819327 31279 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.860379 29038 leader_election.cc:403] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:39.853266 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.861653 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:39.863315 29908 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:39.866178 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:39.867590 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:39.872040 31554 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:04:39.871660 31279 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.886690 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:39.704303 31532 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.893203 29908 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:39.884217 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 82999.1us
I20260709 13:04:39.895610 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:39.877552 31449 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:39.857081 31319 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:39.857618 31408 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:39.851632 31568 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:39.860301 31551 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:39.878924 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:39.921108 29139 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:39.890302 31487 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.867520 31515 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:39.966266 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:39.967659 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 367526us
I20260709 13:04:39.910775 31517 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:39.913751 31551 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:39.918646 31408 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:04:39.865574 31458 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:39.988140 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:39.935943 31528 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:39.944917 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:39.979159 31551 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:39.910073 31370 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:40.045076 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68948us
W20260709 13:04:40.050552 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 152866us
I20260709 13:04:40.062883 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:40.065363 29143 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:40.060142 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:39.918812 31568 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:40.068763 31504 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:40.104305 31559 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.293s	user 0.002s	sys 0.000s
W20260709 13:04:40.114482 31565 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
W20260709 13:04:40.117544 31566 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:04:40.117318 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:40.220353 29143 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:40.055235 31572 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:04:40.238530 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:40.239590 29218 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:40.213771 31568 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:40.214689 31574 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.223s	user 0.007s	sys 0.009s
I20260709 13:04:40.258160 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:40.259117 29143 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
W20260709 13:04:40.284343 31571 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
W20260709 13:04:40.286291 31579 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:04:40.267309 31239 consensus_meta.cc:287] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.533s	user 0.007s	sys 0.005s
W20260709 13:04:40.288277 31575 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 13:04:40.288815 31239 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:40.291306 31588 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
I20260709 13:04:40.293980 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 310) took 1227 ms. Trace:
I20260709 13:04:40.294499 29147 rpcz_store.cc:276] 0709 13:04:39.066708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.066864 (+   156us) service_pool.cc:224] Handling call
0709 13:04:39.067234 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.131181 (+ 63947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.131207 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:39.131251 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:39.131283 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.293681 (+1162398us) spinlock_profiling.cc:243] Waited 63.7 ms on lock 0x7b5400191754. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:40.293739 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.293910 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146624384}
I20260709 13:04:40.301954 31551 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:40.302666 31562 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.577s	user 0.000s	sys 0.002s
I20260709 13:04:40.304315 29905 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:04:40.306711 31541 consensus_meta.cc:287] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.000s
I20260709 13:04:40.306803 31476 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:40.307499 31541 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:40.347165 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 288760us
W20260709 13:04:40.388777 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.405395 31551 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:40.406502 29185 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:40.408497 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:04:40.410980 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:04:40.433293 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.469074 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:40.472417 31524 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.742s	user 0.019s	sys 0.001s
I20260709 13:04:40.324263 31568 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:40.542987 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:40.588201 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:40.588552 29143 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:40.589430 31429 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:40.589829 31429 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:40.590174 31429 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:40.549741 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 688133us
I20260709 13:04:40.592836 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:40.593197 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:40.593818 31582 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 13:04:40.592556 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1602) took 1345 ms. Trace:
W20260709 13:04:40.527881 31449 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.618s	user 0.020s	sys 0.003s
I20260709 13:04:40.594759 29065 rpcz_store.cc:276] 0709 13:04:39.247426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.248844 (+  1418us) service_pool.cc:224] Handling call
0709 13:04:39.249234 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:39.438268 (+189034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.438315 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:39.439240 (+   925us) log.cc:844] Serialized 52 byte log entry
0709 13:04:39.510383 (+ 71143us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:39.510424 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.510488 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.302017 (+791529us) spinlock_profiling.cc:243] Waited 8.48 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:40.567615 (+265598us) raft_consensus.cc:1685] finished
0709 13:04:40.567639 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.592484 (+ 24845us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":189796,"raft.queue_time_us":194842,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":15231,"spinlock_wait_cycles":19498880,"thread_start_us":70665,"threads_started":1,"wal-append.queue_time_us":286268}
I20260709 13:04:40.568833 31339 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:40.579329 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1729) took 1222 ms. Trace:
I20260709 13:04:40.617132 29219 rpcz_store.cc:276] 0709 13:04:39.356717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.453469 (+ 96752us) service_pool.cc:224] Handling call
0709 13:04:39.453797 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:39.458399 (+  4602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.458430 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:39.458984 (+   554us) log.cc:844] Serialized 52 byte log entry
0709 13:04:39.747654 (+288670us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:39.747712 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.747804 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.579016 (+831212us) raft_consensus.cc:1685] finished
0709 13:04:40.579048 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.579240 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4222,"raft.queue_time_us":5030,"spinlock_wait_cycles":13321984,"thread_start_us":124543,"threads_started":1,"wal-append.queue_time_us":362978}
I20260709 13:04:40.594403 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:40.620680 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1736) took 1100 ms. Trace:
I20260709 13:04:40.620893 29069 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
W20260709 13:04:40.585831 31556 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.004s	sys 0.009s
I20260709 13:04:40.586220 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1684) took 1060 ms. Trace:
I20260709 13:04:40.594254 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:40.595016 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:40.545354 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:40.592897 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:40.591616 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:40.633131 29038 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:40.630673 29070 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:40.634716 29038 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:40.595955 29064 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:04:40.631054 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 85842.9us
I20260709 13:04:40.600322 31515 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:40.563014 31574 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
W20260709 13:04:40.613572 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 207162us
I20260709 13:04:40.621237 29219 rpcz_store.cc:276] 0709 13:04:39.520029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.619258 (+1099229us) service_pool.cc:224] Handling call
0709 13:04:40.619642 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.619872 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.619909 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.620331 (+   422us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:40.620387 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.620462 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.620606 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:40.588251 31568 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:40.631847 31572 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:04:40.644649 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:40.646083 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:40.663605 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 255773us
I20260709 13:04:40.665087 31568 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:40.645156 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:40.695901 31555 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.968s	user 0.005s	sys 0.000s
I20260709 13:04:40.675302 31583 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:40.683305 31591 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:04:40.682704 31339 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:04:40.670204 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:40.683650 31586 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:04:40.700690 31583 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:40.701150 31583 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.701181 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:40.701699 31583 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:40.702574 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:40.703594 31515 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:40.704392 31515 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:40.704591 31456 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:40.704964 31515 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:40.705988 31515 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:40.707518 29903 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:40.708709 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 365) took 1107 ms. Trace:
I20260709 13:04:40.709062 29903 rpcz_store.cc:276] 0709 13:04:39.601536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.604089 (+  2553us) service_pool.cc:224] Handling call
0709 13:04:40.707037 (+1102948us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:40.707284 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.708378 (+  1094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.708648 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34560768}
I20260709 13:04:40.709595 31541 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:40.710510 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1728) took 1387 ms. Trace:
I20260709 13:04:40.710839 31406 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:40.710958 29217 rpcz_store.cc:276] 0709 13:04:39.323626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.448882 (+125256us) service_pool.cc:224] Handling call
0709 13:04:39.449264 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.465524 (+ 16260us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:39.581760 (+116236us) log.cc:844] Serialized 14 byte log entry
0709 13:04:40.047277 (+465517us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:40.102263 (+ 54986us) log.cc:844] Serialized 26 byte log entry
0709 13:04:40.541035 (+438772us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:40.541077 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.541159 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.710299 (+169140us) raft_consensus.cc:1685] finished
0709 13:04:40.710319 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.710453 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12642,"prepare.queue_time_us":259414,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":442790,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":26717,"thread_start_us":538389,"threads_started":3,"wal-append.queue_time_us":321894}
I20260709 13:04:40.711948 31583 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:40.713605 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 322) took 1104 ms. Trace:
I20260709 13:04:40.713946 31279 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:40.714349 31583 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:40.631767 29066 raft_consensus.cc:2376] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4. Re-sending same reply.
I20260709 13:04:40.644055 31529 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:40.651721 29070 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
W20260709 13:04:40.715951 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.667932 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:40.629614 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1735) took 1149 ms. Trace:
W20260709 13:04:40.626891 31556 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.004s	sys 0.009s
I20260709 13:04:40.602073 31551 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:40.715564 31449 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.720314 31449 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:40.592618 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:40.710637 31461 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.090s	user 0.015s	sys 0.008s
I20260709 13:04:40.720261 31551 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:40.713953 29217 rpcz_store.cc:276] 0709 13:04:39.609136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.712820 (+1103684us) service_pool.cc:224] Handling call
0709 13:04:40.713130 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.713299 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.713324 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.713365 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:40.713395 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.713458 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.713552 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:40.721437 29072 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:40.582755 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1734) took 1214 ms. Trace:
W20260709 13:04:40.670689 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.672703 29219 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:40.734104 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 319) took 1125 ms. Trace:
I20260709 13:04:40.734541 29219 rpcz_store.cc:276] 0709 13:04:39.608464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.647627 (+1039163us) service_pool.cc:224] Handling call
0709 13:04:40.666982 (+ 19355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.733775 (+ 66793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.734038 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:40.625353 29911 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
W20260709 13:04:40.721664 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 77123.8us
I20260709 13:04:40.738157 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:40.717905 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:40.739298 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.743382 31488 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:40.743551 31339 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:40.718436 29218 rpcz_store.cc:276] 0709 13:04:39.480493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.560991 (+1080498us) service_pool.cc:224] Handling call
0709 13:04:40.628874 (+ 67883us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.629221 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.629255 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.629302 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:40.629349 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.629422 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.629551 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:40.684949 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 139237us
I20260709 13:04:40.736559 29221 rpcz_store.cc:276] 0709 13:04:39.368689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.464313 (+1095624us) service_pool.cc:224] Handling call
0709 13:04:40.581624 (+117311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.582009 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.582072 (+    63us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:40.582200 (+   128us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:40.582271 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.582398 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.582623 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:40.669696 31239 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:40.769922 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:40.770296 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:40.770591 29144 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:40.755960 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:40.756861 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:40.688730 31517 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.750185 31456 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.777756 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:40.779851 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:40.780695 29069 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
W20260709 13:04:40.780216 29214 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.024s	sys 0.041s
I20260709 13:04:40.630097 29068 rpcz_store.cc:276] 0709 13:04:39.525299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.529157 (+  3858us) service_pool.cc:224] Handling call
0709 13:04:39.580276 (+ 51119us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.592228 (+ 11952us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.592275 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.592851 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.823789 (+230938us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:40.212102 (+388313us) log.cc:844] Serialized 14 byte log entry
0709 13:04:40.585080 (+372978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.585335 (+   255us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.585420 (+    85us) raft_consensus.cc:1685] finished
0709 13:04:40.585477 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.585840 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39915,"prepare.queue_time_us":210823,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":383290,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":26657,"thread_start_us":483473,"threads_started":2,"wal-append.queue_time_us":295184}
W20260709 13:04:40.783053 29214 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.025s	sys 0.041s
I20260709 13:04:40.796142 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:40.797336 29064 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:40.772897 31572 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:40.808920 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.786943 31443 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:40.748916 31461 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:40.784698 31504 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:04:40.821986 29218 raft_consensus.cc:2955] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:40.787138 31439 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:04:40.814096 31406 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.823307 31572 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:40.793735 31550 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:04:40.788522 31456 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:40.824211 31443 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:40.843647 29216 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:04:40.801441 31488 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:04:40.826913 31586 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:40.788687 31517 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:40.805620 31529 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:04:40.832183 31406 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:40.838570 31339 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:40.860787 31443 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:40.859844 31591 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:04:40.904985 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 133839us
I20260709 13:04:40.843516 31598 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:40.906002 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:40.880215 31279 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:40.930826 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 228340us
W20260709 13:04:40.941150 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.850747 31568 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:40.945143 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:40.873847 31602 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:40.904171 31578 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:40.899885 31587 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:40.951941 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:40.899941 31586 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:40.881994 31592 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:40.967422 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:40.930325 31577 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:40.904523 31591 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:40.972419 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:40.930012 31551 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:40.977139 29221 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:40.964636 31476 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.008s
I20260709 13:04:40.964730 31602 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:40.978297 31583 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:40.968047 31480 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:41.005309 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1739) took 1142 ms. Trace:
I20260709 13:04:40.969579 31504 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:41.005776 29219 rpcz_store.cc:276] 0709 13:04:39.862701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.735409 (+872708us) service_pool.cc:224] Handling call
0709 13:04:40.735830 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.736209 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:04:40.744615 (+  8406us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.004924 (+260309us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.004988 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:41.005038 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.005111 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.005247 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17409,"prepare.queue_time_us":7916,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":242314,"threads_started":1}
I20260709 13:04:40.970798 31604 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:41.009521 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 223462us
I20260709 13:04:41.015323 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1740) took 1152 ms. Trace:
I20260709 13:04:41.015861 29218 rpcz_store.cc:276] 0709 13:04:39.862955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.745543 (+882588us) service_pool.cc:224] Handling call
0709 13:04:40.787285 (+ 41742us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.787529 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:40.899600 (+112071us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.014858 (+115258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.014919 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:41.014976 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.015071 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.015226 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20821,"thread_start_us":93487,"threads_started":1}
W20260709 13:04:41.047302 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60265.8us
I20260709 13:04:40.985188 31590 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:41.050150 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:40.966843 31279 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:40.995541 31449 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.006004 31583 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:41.067167 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:41.055521 31597 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 13:04:40.988168 31592 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:41.103951 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 131616us
I20260709 13:04:41.020221 31611 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:41.055346 31504 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:41.105886 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:40.995493 31618 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:04:41.055423 31515 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:40.985241 31587 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:41.068535 31589 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:04:41.066409 31583 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:41.130049 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 76414.8us
I20260709 13:04:41.131470 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:41.132776 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:41.072021 31461 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:41.147789 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.154615 29219 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
W20260709 13:04:41.161305 31606 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:04:41.163166 31587 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.163559 31625 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
W20260709 13:04:41.166254 31585 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 13:04:41.167994 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1696) took 1573 ms. Trace:
I20260709 13:04:41.169070 31279 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.169432 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:41.169811 31406 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.118209 31504 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:41.170492 31504 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:41.121606 31618 consensus_queue.cc:1243] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:04:41.168510 29071 rpcz_store.cc:276] 0709 13:04:39.594245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.816594 (+222349us) service_pool.cc:224] Handling call
0709 13:04:39.817439 (+   845us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.818060 (+   621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.818107 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.823381 (+  5274us) log.cc:844] Serialized 506 byte log entry
0709 13:04:39.975735 (+152354us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:39.975780 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.975863 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.167676 (+1191813us) raft_consensus.cc:1685] finished
0709 13:04:41.167706 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.167883 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":143715,"threads_started":1,"wal-append.queue_time_us":990586}
I20260709 13:04:41.054214 31604 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.182698 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:41.131440 31456 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.132335 31488 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:41.187948 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:41.192886 29219 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:41.132577 31572 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.194033 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:41.056548 31590 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.060281 31591 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.197343 31591 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.198899 31590 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:41.200237 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:41.201124 31568 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.190829 31625 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.201817 31625 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:41.203441 31613 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:04:41.204718 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1886) took 1967 ms. Trace:
I20260709 13:04:41.202219 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1895) took 1907 ms. Trace:
I20260709 13:04:41.147511 31551 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.205091 29213 rpcz_store.cc:276] 0709 13:04:39.237010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.689043 (+1452033us) service_pool.cc:224] Handling call
0709 13:04:40.689514 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.720127 (+ 30613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.720159 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:40.720662 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:04:40.894865 (+174203us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:40.894906 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.894972 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.204484 (+309512us) raft_consensus.cc:1685] finished
0709 13:04:41.204513 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.204660 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49971,"raft.queue_time_us":142806,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":19901,"thread_start_us":173914,"threads_started":1,"wal-append.queue_time_us":272429}
W20260709 13:04:41.185982 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 128060us
I20260709 13:04:41.205322 29216 rpcz_store.cc:276] 0709 13:04:39.294697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.444206 (+149509us) service_pool.cc:224] Handling call
0709 13:04:39.444701 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.860201 (+1415500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.201981 (+341780us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400339fd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:41.202161 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3216686464}
I20260709 13:04:41.209587 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:41.172593 31611 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:41.211685 29216 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 3.
I20260709 13:04:41.212471 29140 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:41.130455 31515 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.215412 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:41.216324 29213 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:41.220678 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:41.223223 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:41.194751 31517 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.069494 31586 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:41.247288 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 114563us
W20260709 13:04:41.266592 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 528503us
I20260709 13:04:41.268100 29185 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:41.265399 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:41.270980 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:41.271745 29219 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:41.275362 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:41.228443 31625 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.229029 31611 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:41.116511 31602 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.238914 31406 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:41.273593 31279 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:41.279397 31429 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.689s	user 0.014s	sys 0.000s
I20260709 13:04:41.215468 31551 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.273857 31517 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:41.301604 29039 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.897s (SENT)
W20260709 13:04:41.303416 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 126170us
I20260709 13:04:41.293097 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:41.295495 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 327) took 1431 ms. Trace:
I20260709 13:04:41.305559 29219 rpcz_store.cc:276] 0709 13:04:39.864049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.278016 (+1413967us) service_pool.cc:224] Handling call
0709 13:04:41.292162 (+ 14146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.295105 (+  2943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.295131 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.295192 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:41.295238 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.295308 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.295435 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:41.287961 31572 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:41.294549 31624 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:04:41.290490 31611 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:41.316141 31596 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:04:41.318746 31551 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:41.319613 31619 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 13:04:41.320360 31609 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:04:41.320281 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1882) took 2131 ms. Trace:
I20260709 13:04:41.321257 29220 rpcz_store.cc:276] 0709 13:04:39.188404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.825966 (+1637562us) service_pool.cc:224] Handling call
0709 13:04:40.826375 (+   409us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:40.848908 (+ 22533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.848941 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:40.849577 (+   636us) log.cc:844] Serialized 52 byte log entry
0709 13:04:41.056865 (+207288us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:41.103385 (+ 46520us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.115842 (+ 12457us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.116066 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.116154 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.317828 (+201674us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b18002b57c0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:04:41.320012 (+  2184us) raft_consensus.cc:1685] finished
0709 13:04:41.320058 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.320213 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34036,"prepare.queue_time_us":30246,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":9739,"raft.queue_time_us":141321,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":16504,"spinlock_wait_cycles":106570752,"thread_start_us":151955,"threads_started":1,"wal-append.queue_time_us":326046}
I20260709 13:04:41.329532 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 329) took 1103 ms. Trace:
I20260709 13:04:41.330334 31611 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.330747 31604 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:41.331471 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:41.334556 31583 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.305223 31517 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.329967 29220 rpcz_store.cc:276] 0709 13:04:40.226465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.327563 (+1101098us) service_pool.cc:224] Handling call
0709 13:04:41.328914 (+  1351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.329133 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.329170 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.329232 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:41.329277 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.329339 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.329462 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:41.337916 31515 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.341054 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:41.315001 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:41.332108 31618 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:41.344334 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1738) took 1481 ms. Trace:
I20260709 13:04:41.345754 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 328) took 1137 ms. Trace:
I20260709 13:04:41.333590 31370 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:41.344844 29217 rpcz_store.cc:276] 0709 13:04:39.862449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.724182 (+861733us) service_pool.cc:224] Handling call
0709 13:04:40.724647 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.965165 (+240518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.965215 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:40.970056 (+  4841us) log.cc:844] Serialized 26 byte log entry
0709 13:04:41.057365 (+ 87309us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:41.057422 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.057496 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.344044 (+286548us) raft_consensus.cc:1685] finished
0709 13:04:41.344078 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.344258 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97126,"raft.queue_time_us":156090,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":38543,"thread_start_us":205985,"threads_started":2,"wal-append.queue_time_us":162353}
W20260709 13:04:41.352864 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.346257 29219 rpcz_store.cc:276] 0709 13:04:40.208194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.306723 (+1098529us) service_pool.cc:224] Handling call
0709 13:04:41.345090 (+ 38367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.345321 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.345371 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.345431 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:41.345482 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.345562 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.345685 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:41.343887 31628 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:41.296756 31279 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:41.355250 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 62244.8us
W20260709 13:04:41.353685 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 78391us
I20260709 13:04:41.356894 31572 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:41.357681 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:41.358144 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:41.354031 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.343981 31618 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:04:41.359229 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.357548 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:41.301664 31429 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.295135 31406 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.362390 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:41.352540 31370 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:04:41.364048 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.363864 29217 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:41.373627 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:41.374452 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:41.358247 31572 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:41.352299 31504 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.360749 31618 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:41.380282 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:41.372630 31628 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:41.381599 31476 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:41.376981 31620 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:41.363503 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:41.382771 31406 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.380719 31572 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:41.383523 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:41.380441 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.380647 29910 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:41.387905 29219 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:41.364207 29906 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:41.391172 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:41.392323 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:41.392789 31624 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:41.393312 31624 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:41.393752 31624 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:41.395378 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:41.377715 31370 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:41.384037 31591 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:41.395829 31476 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:41.417250 31591 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:41.417865 31591 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:41.390146 31450 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:41.418509 31450 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.420056 31620 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:41.420504 31620 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:41.420923 31620 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.423108 31572 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.397229 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.396303 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:41.392045 31478 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.424654 31478 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.425242 31406 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:41.425815 31406 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.426209 31406 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.427353 31478 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.396842 31628 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.396333 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.397945 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.398597 31279 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.436439 29067 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:41.436367 29071 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:41.437762 31504 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.412187 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:41.410249 29072 raft_consensus.cc:2376] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4. Re-sending same reply.
I20260709 13:04:41.440368 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:41.441191 31464 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:41.441618 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:41.441835 31464 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:41.442238 31464 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:41.442639 31464 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:41.366413 31443 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.445379 31616 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:41.446030 31443 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.446705 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:41.379283 31586 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:41.449862 31586 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:41.450032 31616 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:41.450268 31586 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:41.451742 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 324) took 1588 ms. Trace:
I20260709 13:04:41.452170 29221 rpcz_store.cc:276] 0709 13:04:39.863216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.746725 (+883509us) service_pool.cc:224] Handling call
0709 13:04:40.747019 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.988464 (+241445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.451502 (+463038us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54000bd454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:41.451678 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":528704256}
I20260709 13:04:41.415307 31618 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.400324 31517 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.400053 31429 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.415354 29068 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:04:41.396340 31633 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:41.454833 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:41.447665 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1731) took 2090 ms. Trace:
I20260709 13:04:41.460608 31279 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.463321 29214 rpcz_store.cc:276] 0709 13:04:39.357340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.820843 (+463503us) service_pool.cc:224] Handling call
0709 13:04:39.821178 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.821437 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:04:39.874799 (+ 53362us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.447236 (+1572437us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b18003f7d20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287
0709 13:04:41.447338 (+   102us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.447389 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:41.447432 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.447492 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.447606 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16423,"prepare.queue_time_us":77693,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":647567,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":11299,"spinlock_wait_cycles":4799872,"thread_start_us":916888,"threads_started":2,"wal-append.queue_time_us":910258}
I20260709 13:04:41.464618 31279 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.466645 31633 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:41.466871 31279 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.467101 31633 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:41.467463 31640 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.467594 31633 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:41.469266 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:41.475226 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 468) took 1659 ms. Trace:
W20260709 13:04:41.478960 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 87878.4us
I20260709 13:04:41.479890 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:41.478166 31448 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.481084 31448 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.483690 31450 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.481300 31604 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:41.490993 31448 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:41.493818 31370 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.468122 31640 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.496256 31370 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.497481 31464 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.498629 31640 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.500599 31586 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.501885 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:41.502406 29140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1888) took 2447 ms (client timeout 1678 ms). Trace:
I20260709 13:04:41.502734 31586 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:41.502887 29140 rpcz_store.cc:269] 0709 13:04:39.055534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.060166 (+  4632us) service_pool.cc:224] Handling call
0709 13:04:41.502179 (+2442013us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b54003a4dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:41.502299 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4448242560}
W20260709 13:04:41.505720 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.506589 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:41.517194 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:41.523563 29188 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:41.527815 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:41.528352 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:41.529683 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:41.533638 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:41.510639 31604 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.537917 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.540223 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:41.540452 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.543152 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:41.543745 29139 raft_consensus.cc:2376] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4. Re-sending same reply.
I20260709 13:04:41.510720 31279 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:41.503592 31586 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:41.521402 31478 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:41.510708 31577 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:41.544634 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:41.532876 31590 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:41.545989 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:41.526788 31578 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:41.526844 31339 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.547006 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.547610 29144 raft_consensus.cc:2410] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:41.493814 31406 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.549310 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:41.478880 29905 rpcz_store.cc:276] 0709 13:04:39.816184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.829516 (+ 13332us) service_pool.cc:224] Handling call
0709 13:04:41.474955 (+1645439us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b540063d654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:41.475143 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876181376}
I20260709 13:04:41.545367 29145 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:41.581673 29112 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:41.546867 31598 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:41.535107 31448 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:41.534994 31568 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:41.585371 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.543234 31438 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.536427 31616 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:41.547420 31370 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:41.562107 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.577715 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.546700 31587 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.582010 29145 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:41.553509 29139 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:41.589712 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.561120 31578 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:41.586028 29144 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:41.592913 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.583640 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:41.593979 29072 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:41.589733 29146 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:41.548802 31604 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.584805 31458 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:41.598138 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.594521 29145 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.585745 31521 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:41.552762 31279 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.602054 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:41.585830 31568 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:41.585299 31448 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.605312 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.605342 29067 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:41.608261 29146 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:41.590626 31598 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.597277 31578 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.588426 31616 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:41.613464 31616 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:41.606151 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:41.627741 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.598356 31587 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.637683 31572 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:41.638033 31587 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.604681 31633 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.640249 31464 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:41.640439 31633 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.640314 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:41.643247 29146 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
W20260709 13:04:41.650519 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 68887.9us
I20260709 13:04:41.651512 29112 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:41.652390 31657 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:41.654866 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:41.657480 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 80874.9us
I20260709 13:04:41.660835 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.660888 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.581387 31339 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.604231 31624 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:41.661674 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 111158us
I20260709 13:04:41.661613 29073 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
I20260709 13:04:41.662788 29072 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:41.672669 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:41.673462 29908 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:41.675108 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:41.676192 29067 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:41.676715 29186 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:41.615315 31568 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.669294 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:41.614825 31640 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:41.663378 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.678833 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:41.680106 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:41.684206 29908 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:41.684332 29112 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:41.685256 29146 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:41.616760 31448 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.573544 31577 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.689822 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:41.685259 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:41.691454 29071 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.686739 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:41.686803 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:41.688843 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.696537 29147 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 2.
I20260709 13:04:41.697723 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.681049 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:41.698864 29072 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.701891 29142 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:41.702040 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:41.703029 29147 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:41.652964 31657 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.708405 29072 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:41.677757 31598 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.709816 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.683405 31633 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:41.704046 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.709054 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:41.715288 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:41.716773 29146 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:41.691083 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:41.725355 29072 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:41.720671 29071 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:41.726430 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:41.700940 31464 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:41.705809 31572 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:41.705399 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:41.728029 29146 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:41.698561 31656 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:41.707070 31577 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.699968 31587 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.707465 31657 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:41.720701 31647 log.cc:927] Time spent T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:04:41.741612 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:41.577454 31444 raft_consensus.cc:2955] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:41.715837 31633 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.690917 31620 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:41.762732 29072 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:41.764010 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.764869 29072 raft_consensus.cc:2393] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:41.726020 31643 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:41.732537 31429 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:41.763744 29147 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:41.738004 31572 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:41.744936 31656 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.752435 31464 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:41.732393 31628 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:04:41.753376 31279 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.765064 31633 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.767728 31406 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.772989 31429 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.742359 31591 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.776859 31572 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.765604 31587 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.776706 31656 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:41.782536 31628 consensus_queue.cc:1243] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:04:41.784446 31429 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:41.790436 31620 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:41.793392 31604 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:41.810300 31532 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:41.811753 31560 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 2.084s	user 0.006s	sys 0.008s
W20260709 13:04:41.812762 31581 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.143s	user 0.002s	sys 0.000s
I20260709 13:04:41.811753 31653 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:41.839968 31653 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:41.840893 31587 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:41.841354 31653 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.847064 31591 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:41.814608 31624 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:41.857314 31553 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 2.255s	user 0.000s	sys 0.006s
I20260709 13:04:41.873247 31583 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.878131 31656 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.879096 31592 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.880388 31656 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:41.876981 31602 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.884646 31657 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.881356 31572 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.881407 31616 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.887576 31578 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:41.889360 31616 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.889715 31578 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:41.891083 31572 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:41.891762 31576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.189s	user 0.002s	sys 0.007s
W20260709 13:04:41.892278 31576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.190s	user 0.002s	sys 0.007s
I20260709 13:04:41.887418 31657 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:41.894937 31633 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:41.932303 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:41.948761 29066 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:41.765924 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:41.954228 29072 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:41.942929 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1715) took 2969 ms. Trace:
I20260709 13:04:41.960489 29215 rpcz_store.cc:276] 0709 13:04:38.973054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.981191 (+  8137us) service_pool.cc:224] Handling call
0709 13:04:38.990516 (+  9325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.244311 (+253795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.244358 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.293876 (+ 49518us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.538092 (+244216us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:39.538145 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.538235 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.809208 (+2270973us) spinlock_profiling.cc:243] Waited 6.13 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:41.942001 (+132793us) raft_consensus.cc:1685] finished
0709 13:04:41.942053 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.942612 (+   559us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10317,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":14094208,"thread_start_us":412009,"threads_started":2,"wal-append.queue_time_us":401420}
I20260709 13:04:41.932476 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.936056 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.969362 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:41.969602 29066 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:04:41.970868 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.971063 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:41.937139 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.972186 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:41.934739 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 197613us
I20260709 13:04:41.941088 29140 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:41.952880 29068 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:41.973773 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.974656 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:41.975025 29140 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:41.975400 29068 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:41.975528 29066 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:04:41.975878 29066 pending_rounds.cc:85] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 2
I20260709 13:04:41.976249 29066 pending_rounds.cc:107] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:04:41.976687 29066 raft_consensus.cc:2889] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:41.979593 29222 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1732) took 2621 ms (client timeout 1882 ms). Trace:
I20260709 13:04:41.980012 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:04:41.980155 29222 rpcz_store.cc:269] 0709 13:04:39.357648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.434680 (+ 77032us) service_pool.cc:224] Handling call
0709 13:04:41.979133 (+2544453us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400132ed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:41.979319 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347162496}
I20260709 13:04:41.984458 29071 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:41.985203 29144 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:41.986838 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:41.987854 29906 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.997505 31279 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:41.998201 31339 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:42.014329 31464 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:42.016587 31530 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.017043 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:42.017786 31530 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:42.025269 31635 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:42.026196 31635 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:42.026167 31568 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:42.028874 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55295.1us
I20260709 13:04:42.030148 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:42.032490 31661 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:42.033380 31661 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:42.032629 31279 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:42.034613 31635 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:42.037514 31633 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:42.041551 31577 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.042768 31530 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:42.042552 31577 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:42.044615 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:42.045858 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:42.035348 31661 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:42.047606 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:42.056928 29213 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:42.051774 29871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.956s	user 0.018s	sys 0.014s
I20260709 13:04:42.076818 29903 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:42.077056 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:42.077443 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:42.078685 31633 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:42.079666 31633 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:42.081480 31633 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:42.084501 31577 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:42.085245 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 367) took 2482 ms. Trace:
I20260709 13:04:42.085446 31530 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:42.086609 31530 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:42.061486 29902 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:04:42.087889 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 456) took 2479 ms. Trace:
I20260709 13:04:42.085860 29904 rpcz_store.cc:276] 0709 13:04:39.602177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.605051 (+  2874us) service_pool.cc:224] Handling call
0709 13:04:42.062610 (+2457559us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8eee 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:42.063110 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.063755 (+   645us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.063811 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.064717 (+   906us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:42.072743 (+  8026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.084825 (+ 12082us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.084998 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58667776}
I20260709 13:04:42.088963 31573 ts_tablet_manager.cc:933] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:42.088981 29902 rpcz_store.cc:276] 0709 13:04:39.608202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.615749 (+  7547us) service_pool.cc:224] Handling call
0709 13:04:42.060089 (+2444340us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:42.060634 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.087511 (+ 26877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.087820 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35087232}
I20260709 13:04:42.077059 31608 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.105937 29072 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
W20260709 13:04:42.110833 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 66637.9us
I20260709 13:04:42.111994 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:42.112527 31531 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:42.113283 31531 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:42.114745 31531 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:42.118140 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 459) took 2475 ms. Trace:
W20260709 13:04:42.120519 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 44163.9us
I20260709 13:04:42.118806 29909 rpcz_store.cc:276] 0709 13:04:39.642995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.643129 (+   134us) service_pool.cc:224] Handling call
0709 13:04:42.116548 (+2473419us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:42.117008 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.117389 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.117429 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.117508 (+    79us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:42.117606 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.117686 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.117888 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53015808}
W20260709 13:04:42.124234 29871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.029s	user 0.019s	sys 0.014s
I20260709 13:04:42.135277 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:42.136267 29071 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:04:42.137928 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 326) took 2274 ms. Trace:
I20260709 13:04:42.138375 29213 rpcz_store.cc:276] 0709 13:04:39.863734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.277500 (+1413766us) service_pool.cc:224] Handling call
0709 13:04:42.008980 (+731480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.105129 (+ 96149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.137619 (+ 32490us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b5400132ed0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:42.137810 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653758208}
I20260709 13:04:42.139479 29070 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:42.140257 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:42.141781 29911 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 465) took 2445 ms (client timeout 1839 ms). Trace:
W20260709 13:04:42.142375 29911 rpcz_store.cc:269] 0709 13:04:39.695945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.814214 (+118269us) service_pool.cc:224] Handling call
0709 13:04:42.141202 (+2326988us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54003fdc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:42.141501 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778277248}
I20260709 13:04:42.112010 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:42.091706 31608 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:42.145881 31657 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:42.146622 31657 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:42.148053 31657 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:42.151942 31568 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:42.153438 31568 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:42.154263 31568 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:42.154965 31547 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:42.158905 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 3.06369e+06us
I20260709 13:04:42.164352 31279 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:42.136267 29910 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 370) took 2533 ms (client timeout 1880 ms). Trace:
W20260709 13:04:42.165803 29910 rpcz_store.cc:269] 0709 13:04:39.603030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.604878 (+  1848us) service_pool.cc:224] Handling call
0709 13:04:41.331318 (+1726440us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:42.135758 (+804440us) spinlock_profiling.cc:243] Waited 46.1 ms on lock 0x7b54007e0254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:42.136011 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217237632}
I20260709 13:04:42.167078 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:42.168610 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 121342us
I20260709 13:04:42.169687 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:42.159371 31530 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:42.175788 31461 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:42.177028 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 368) took 2574 ms. Trace:
I20260709 13:04:42.177896 31461 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:04:42.178587 31438 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.588s	user 0.014s	sys 0.008s
I20260709 13:04:42.179026 31461 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:42.180323 31461 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:42.181635 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:42.186281 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:42.186851 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:42.187538 29909 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:04:42.186357 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:42.177472 29907 rpcz_store.cc:276] 0709 13:04:39.602454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.629504 (+ 27050us) service_pool.cc:224] Handling call
0709 13:04:42.175589 (+2546085us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:42.175903 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.176340 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.176398 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.176458 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:42.176566 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.176654 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.176828 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7158016}
W20260709 13:04:42.194957 31523 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.461s	user 0.005s	sys 0.010s
W20260709 13:04:42.195535 31523 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.497s	user 0.005s	sys 0.010s
I20260709 13:04:42.179069 31438 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:42.196591 31429 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:42.198137 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.199527 31663 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:42.200263 31663 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:42.201591 31663 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:42.155599 31568 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:42.205052 31547 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:42.210777 29905 raft_consensus.cc:2368] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4.
I20260709 13:04:42.212721 29910 raft_consensus.cc:2376] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5. Re-sending same reply.
I20260709 13:04:42.172271 31448 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:42.218132 31429 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:42.203502 29906 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:42.277714 31429 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:42.277916 31438 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:42.278282 31429 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:42.322011 31279 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:42.340070 31279 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:42.341141 31604 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.278702 31438 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:43.766711 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08000f9d80 after lost signal to thread 27937
W20260709 13:04:43.800877 31655 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.346s	user 0.002s	sys 0.010s
W20260709 13:04:44.250088 31677 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 13:04:44.405546 29036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.008s	sys 0.026s
W20260709 13:04:44.432346 29036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.008s	sys 0.027s
W20260709 13:04:44.401394 31657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.179s	user 0.020s	sys 0.046s
W20260709 13:04:44.433794 31657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.218s	user 0.021s	sys 0.046s
I20260709 13:04:44.438655 31685 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.439707 31685 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:44.441318 29874 connection.cc:256] RPC call timeout handler was delayed by -1.1166212400003133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2905s for a response
I20260709 13:04:44.445561 31685 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:44.446308 31685 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:44.446955 31685 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.441241 31688 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:44.449488 31688 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.452147 31688 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:44.453233 31573 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 2.363s	user 0.028s	sys 0.037s
W20260709 13:04:44.453663 31573 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 2.363s	user 0.028s	sys 0.037s
I20260709 13:04:44.454034 31685 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:44.454337 31573 tablet_copy_client.cc:323] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.130:35337
W20260709 13:04:44.480939 29903 consensus_meta.cc:287] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 2.403s	user 0.023s	sys 0.004s
I20260709 13:04:44.481353 29903 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
W20260709 13:04:44.481881 29903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 385) took 2438 ms (client timeout 2100 ms). Trace:
W20260709 13:04:44.482169 29903 rpcz_store.cc:269] 0709 13:04:42.043785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.045405 (+  1620us) service_pool.cc:224] Handling call
0709 13:04:44.481814 (+2436409us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:44.604701 31510 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.389s	user 0.007s	sys 0.034s
W20260709 13:04:44.605186 31510 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.390s	user 0.008s	sys 0.034s
W20260709 13:04:44.608778 29874 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.674s (SENT) [suppressed 28 similar messages]
W20260709 13:04:44.612334 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 2.43123e+06us
I20260709 13:04:44.612563 31645 consensus_queue.cc:1059] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:44.619035 31645 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:44.619578 31645 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:44.624823 29875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.440s	user 0.018s	sys 0.035s
W20260709 13:04:44.625258 29875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.440s	user 0.019s	sys 0.036s
W20260709 13:04:44.625561 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 2.44071e+06us
I20260709 13:04:44.626710 31687 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:44.627034 31683 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:44.628619 31690 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:44.620963 31645 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.633764 31661 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:44.634526 31661 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.636270 31496 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.636986 31496 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:42.342007 31458 consensus_meta.cc:287] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.737s	user 0.015s	sys 0.000s
I20260709 13:04:44.639947 31496 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:44.641371 31683 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:44.645365 31683 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:44.646524 31683 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.641469 31661 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:44.648365 31683 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:44.644676 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.650308 29141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1892) took 5245 ms (client timeout 1897 ms). Trace:
W20260709 13:04:44.651693 29141 rpcz_store.cc:269] 0709 13:04:39.404978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.405188 (+   210us) service_pool.cc:224] Handling call
0709 13:04:44.649888 (+5244700us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54001bbcd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:44.650093 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4088931200}
I20260709 13:04:44.654282 31608 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:44.644726 31645 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:44.657246 31531 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.511s	user 0.007s	sys 0.037s
W20260709 13:04:44.657672 31531 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.511s	user 0.007s	sys 0.037s
I20260709 13:04:44.661293 31496 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.662026 31496 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:42.503371 31525 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.529s	user 0.053s	sys 0.075s
W20260709 13:04:44.665542 31525 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 5.044s	user 0.062s	sys 0.083s
I20260709 13:04:44.669971 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:44.670258 31690 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.670863 31690 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.672834 31496 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:44.673998 31690 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:44.664505 31645 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.675316 31645 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.672191 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:44.677563 31645 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:44.678656 31645 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:44.679361 31645 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:44.682025 29157 tablet_copy_service.cc:140] P 730186a8c70f43f2a57d80163bbc37d2: Received BeginTabletCopySession request for tablet 79cdfc6a725145348b70034422ef477b from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:60830)
I20260709 13:04:44.682796 29157 tablet_copy_service.cc:161] P 730186a8c70f43f2a57d80163bbc37d2: Beginning new tablet copy session on tablet 79cdfc6a725145348b70034422ef477b from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:60830: session id = 7e78ffbcc98242068a874a5d9e341477-79cdfc6a725145348b70034422ef477b
I20260709 13:04:42.395804 31429 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.688602 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:44.684911 29910 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:44.697245 31645 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.705245 31695 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:44.707069 31695 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.709123 31685 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:44.709859 31685 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.712810 31685 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:44.713403 31685 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:44.714115 31685 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.717293 31685 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:44.718482 31685 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.719281 31685 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.720259 31429 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.721992 31685 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:44.724367 29216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1897) took 3573 ms (client timeout 1972 ms). Trace:
W20260709 13:04:44.724924 29216 rpcz_store.cc:269] 0709 13:04:41.150526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.209310 (+ 58784us) service_pool.cc:224] Handling call
0709 13:04:44.723983 (+3514673us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x7b540023c154. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:44.724161 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3036544}
W20260709 13:04:44.726289 29038 connection.cc:256] RPC call timeout handler was delayed by -1.9574560660003044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 13:04:44.726696 29038 connection.cc:256] RPC call timeout handler was delayed by -1.9286560660002579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
W20260709 13:04:44.726985 29038 connection.cc:256] RPC call timeout handler was delayed by -1.7720560660000046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
W20260709 13:04:44.727433 29038 connection.cc:256] RPC call timeout handler was delayed by -1.7387560659999508s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
W20260709 13:04:44.727809 29038 connection.cc:256] RPC call timeout handler was delayed by -1.5568758250001338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260709 13:04:44.728132 29038 connection.cc:256] RPC call timeout handler was delayed by -1.455330932999459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
I20260709 13:04:44.728703 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1742) took 3906 ms. Trace:
I20260709 13:04:44.728703 31685 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.729635 29220 rpcz_store.cc:276] 0709 13:04:40.822562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.337379 (+514817us) service_pool.cc:224] Handling call
0709 13:04:44.726206 (+3388827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:44.726538 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.726564 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:44.726603 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:44.726630 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.726694 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.728412 (+  1718us) spinlock_profiling.cc:243] Waited 3.39 s on lock 0x7b540023c150. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:44.728529 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7792738176}
W20260709 13:04:44.730618 29038 connection.cc:256] RPC call timeout handler was delayed by -1.4383000460002222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
W20260709 13:04:44.730973 29038 connection.cc:256] RPC call timeout handler was delayed by -1.409357433999503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
W20260709 13:04:44.735720 29038 connection.cc:256] RPC call timeout handler was delayed by -1.3593446830000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
W20260709 13:04:44.737104 29038 connection.cc:256] RPC call timeout handler was delayed by -1.2320077880003737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
I20260709 13:04:44.724112 31695 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:44.730065 31685 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.736346 31697 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:44.740571 31697 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:44.737957 29071 consensus_meta.cc:287] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 2.601s	user 0.018s	sys 0.005s
I20260709 13:04:44.746027 29071 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
W20260709 13:04:44.746690 29071 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1666) took 2623 ms (client timeout 2100 ms). Trace:
W20260709 13:04:44.747197 29071 rpcz_store.cc:269] 0709 13:04:42.122826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.132813 (+  9987us) service_pool.cc:224] Handling call
0709 13:04:44.746583 (+2613770us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:44.745579 31697 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:44.742017 31685 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:44.737947 29038 connection.cc:256] RPC call timeout handler was delayed by -1.2052000459998453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24700000000000003s for a response
W20260709 13:04:44.750502 29038 connection.cc:256] RPC call timeout handler was delayed by -1.1840363339997566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
W20260709 13:04:44.750999 29038 connection.cc:256] RPC call timeout handler was delayed by -1.1204567450004106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
I20260709 13:04:44.758613 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:44.766297 31685 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:04:44.771874 31429 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:44.774123 31687 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:44.774191 31429 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:44.774641 31687 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:44.775331 31429 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:44.775758 31429 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:44.775722 31687 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.777356 31687 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:44.776981 31429 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.778678 31429 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:44.781064 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.783646 29139 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:44.784148 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:44.785053 31695 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:44.785778 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:44.788424 31695 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:44.789008 31695 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:44.789407 31695 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.795953 31525 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:44.797415 31525 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:44.798321 31525 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.801430 31525 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:44.805992 31438 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:44.807726 29140 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:04:44.808074 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.823361 31687 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:44.807329 31695 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:42.341946 31604 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.830022 31695 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:44.830466 31604 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:44.817152 29909 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 2.620s	user 0.007s	sys 0.016s
I20260709 13:04:44.831374 29909 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
W20260709 13:04:44.832911 29909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 483) took 2649 ms (client timeout 1936 ms). Trace:
W20260709 13:04:44.833339 29909 rpcz_store.cc:269] 0709 13:04:42.183857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.186613 (+  2756us) service_pool.cc:224] Handling call
0709 13:04:44.832835 (+2646222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:44.836024 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:44.851255 31697 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:44.851802 31697 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:44.852232 31697 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:44.857959 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:44.858634 29909 raft_consensus.cc:2376] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5. Re-sending same reply.
I20260709 13:04:44.862612 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:44.863612 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:44.865696 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:44.866536 29072 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
I20260709 13:04:44.866964 29112 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:44.867969 31645 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:44.868518 31645 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:44.869787 31645 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:44.873468 31653 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.880985 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:44.881793 29072 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:44.882189 29148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.709s	user 0.022s	sys 0.041s
W20260709 13:04:44.882606 29148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.709s	user 0.022s	sys 0.041s
I20260709 13:04:44.881244 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:44.892374 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:44.893401 29067 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:44.894901 31631 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:44.895757 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1617) took 4777 ms. Trace:
I20260709 13:04:44.898650 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:44.904420 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:44.904851 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:44.905314 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 482) took 2721 ms. Trace:
I20260709 13:04:44.905859 29911 rpcz_store.cc:276] 0709 13:04:42.183515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.836695 (+2653180us) service_pool.cc:224] Handling call
0709 13:04:44.837056 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:44.837398 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.837426 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.837984 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.859623 (+ 21639us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:44.859658 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.859720 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:44.904999 (+ 45279us) raft_consensus.cc:1685] finished
0709 13:04:44.905029 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.905210 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":18216,"threads_started":1,"wal-append.queue_time_us":62600}
I20260709 13:04:44.896463 29148 rpcz_store.cc:276] 0709 13:04:40.118420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.206569 (+ 88149us) service_pool.cc:224] Handling call
0709 13:04:40.232466 (+ 25897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.254084 (+ 21618us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:04:40.541475 (+287391us) log.cc:844] Serialized 14 byte log entry
0709 13:04:42.173204 (+1631729us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b18004b0f40. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a 00007f7836b17a2a
0709 13:04:44.883409 (+2710205us) log.cc:844] Serialized 14 byte log entry
0709 13:04:44.883639 (+   230us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:44.892573 (+  8934us) log.cc:844] Serialized 26 byte log entry
0709 13:04:44.894071 (+  1498us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:44.894109 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.894246 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:44.894388 (+   142us) raft_consensus.cc:1685] finished
0709 13:04:44.894402 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.895654 (+  1252us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":303242,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":63,"raft.queue_time_us":13442,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":12909,"spinlock_wait_cycles":31874304,"thread_start_us":2709582,"threads_started":1,"wal-append.queue_time_us":3829633}
W20260709 13:04:44.909297 31671 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 2.653s	user 0.007s	sys 0.025s
I20260709 13:04:44.908023 29072 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:44.890985 31697 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.920904 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 318) took 3259 ms. Trace:
I20260709 13:04:44.921355 29146 rpcz_store.cc:276] 0709 13:04:41.653859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.743100 (+ 89241us) service_pool.cc:224] Handling call
0709 13:04:41.744325 (+  1225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:41.937724 (+193399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.937798 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:41.938952 (+  1154us) log.cc:844] Serialized 26 byte log entry
0709 13:04:42.098926 (+159974us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:42.174785 (+ 75859us) log.cc:844] Serialized 14 byte log entry
0709 13:04:42.188751 (+ 13966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.190974 (+  2223us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:44.907878 (+2716904us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:44.912294 (+  4416us) raft_consensus.cc:1685] finished
0709 13:04:44.912631 (+   337us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.913667 (+  1036us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":58450,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":42986,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":206,"spinlock_wait_cycles":2019524352,"thread_start_us":351144,"threads_started":2,"wal-append.queue_time_us":313105}
I20260709 13:04:44.926738 31448 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:44.921787 31697 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:44.943776 29037 connection.cc:256] RPC call timeout handler was delayed by -2.5930825989998993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
W20260709 13:04:44.944252 29037 connection.cc:256] RPC call timeout handler was delayed by -2.5756924060005986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
W20260709 13:04:44.944767 29037 connection.cc:256] RPC call timeout handler was delayed by -2.423869158999878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
W20260709 13:04:44.945191 29037 connection.cc:256] RPC call timeout handler was delayed by -2.3599825989995225s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2147s for a response
I20260709 13:04:44.948530 31525 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:44.946659 31695 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.949986 31695 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:44.946342 29037 connection.cc:256] RPC call timeout handler was delayed by -1.3658272299999226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
W20260709 13:04:44.952354 29037 connection.cc:256] RPC call timeout handler was delayed by -1.3204825989996607s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33020000000000005s for a response
I20260709 13:04:44.951737 31577 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:44.953498 31577 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.954550 31636 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.955065 31636 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.955833 31577 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:44.956467 31525 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.958472 31577 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:44.962805 31577 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:44.960953 31695 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:44.960119 31525 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:44.966614 31577 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:44.957412 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 50300.4us
W20260709 13:04:42.582352 31615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.104s	user 0.007s	sys 0.021s
W20260709 13:04:44.967898 31615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.759s	user 0.022s	sys 0.047s
W20260709 13:04:44.971751 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.979622 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1620) took 3436 ms. Trace:
I20260709 13:04:44.980158 29142 rpcz_store.cc:276] 0709 13:04:41.543195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.545798 (+  2603us) service_pool.cc:224] Handling call
0709 13:04:41.546055 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:41.704524 (+158469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.979010 (+3274486us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b540027d3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:44.979390 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357993856}
W20260709 13:04:44.982674 29871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.779s	user 0.012s	sys 0.007s
W20260709 13:04:44.983158 29871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.779s	user 0.012s	sys 0.007s
W20260709 13:04:44.984740 29871 connection.cc:256] RPC call timeout handler was delayed by -2.8694961489991329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2745s for a response
W20260709 13:04:44.988771 29871 connection.cc:256] RPC call timeout handler was delayed by -1.736146218999238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1895s for a response
W20260709 13:04:44.989142 29871 connection.cc:256] RPC call timeout handler was delayed by -1.6956732349999584s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
W20260709 13:04:44.989444 29871 connection.cc:256] RPC call timeout handler was delayed by -1.1965055890004805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
I20260709 13:04:44.990947 29911 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:44.992703 29911 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:44.994594 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:44.995203 29911 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:44.996251 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:45.009564 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.010341 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.011585 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.013993 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.014691 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.015051 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.014742 29909 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:45.015722 29911 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:45.018896 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.019066 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.019524 29909 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
W20260709 13:04:45.020021 31667 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 2.774s	user 0.016s	sys 0.036s
I20260709 13:04:45.020280 29902 raft_consensus.cc:2410] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:45.029062 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.029361 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.029752 29909 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:45.030090 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.030702 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:45.030797 29903 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:45.031957 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:45.035578 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:45.027626 31461 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.419s	user 0.029s	sys 0.017s
I20260709 13:04:45.039036 29902 raft_consensus.cc:2368] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5.
I20260709 13:04:45.039688 31538 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:45.043306 31608 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.044088 31608 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.046270 31438 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.047242 31438 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:45.048158 29871 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:45.049557 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:45.050351 31547 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:45.050740 31602 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:45.046512 29146 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:45.050932 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:45.060647 29222 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:45.060459 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.062522 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:45.063369 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.057020 31604 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:45.065850 31607 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:45.050988 31547 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:45.066551 29139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 1626) took 3521 ms (client timeout 2125 ms). Trace:
I20260709 13:04:45.066480 31602 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:45.067441 31568 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.067972 31633 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.081550 31608 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:45.081432 31633 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.069453 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.071753 29220 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:45.051324 31429 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:45.076874 29139 rpcz_store.cc:269] 0709 13:04:41.544834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.592649 (+ 47815us) service_pool.cc:224] Handling call
0709 13:04:45.066277 (+3473628us) spinlock_profiling.cc:243] Waited 3.19 s on lock 0x7b5400190d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:45.066460 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7335490816}
I20260709 13:04:45.078083 31547 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:45.080174 29113 connection.cc:256] RPC call timeout handler was delayed by -2.5722052399996755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19970000000000002s for a response
I20260709 13:04:45.079855 31607 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.081348 31568 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:04:45.038492 31461 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.433s	user 0.030s	sys 0.017s
I20260709 13:04:45.125520 29213 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:45.083405 29217 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
W20260709 13:04:45.109444 29113 connection.cc:256] RPC call timeout handler was delayed by -2.543405239999629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
W20260709 13:04:45.140471 29186 connection.cc:256] RPC call timeout handler was delayed by -1.2986961919996247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 13:04:45.142318 29113 connection.cc:256] RPC call timeout handler was delayed by -2.19288820499969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260709 13:04:45.122777 31438 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:45.142829 29113 connection.cc:256] RPC call timeout handler was delayed by -1.9703756620001514s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 13:04:45.143396 29113 connection.cc:256] RPC call timeout handler was delayed by -1.8834067629995843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
W20260709 13:04:45.143774 29113 connection.cc:256] RPC call timeout handler was delayed by -1.7723510309997437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19930000000000003s for a response
W20260709 13:04:45.144230 29113 connection.cc:256] RPC call timeout handler was delayed by -1.5582429120004235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 13:04:45.144754 29113 connection.cc:256] RPC call timeout handler was delayed by -1.5168429120003566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
I20260709 13:04:45.078208 31687 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:45.151129 31339 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:45.144390 31645 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.151782 31339 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.152254 31645 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.154031 31339 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:45.154606 31636 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:45.154836 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.155248 29908 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:45.154526 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:45.155640 29908 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:45.124614 31633 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:45.155581 29902 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:45.102885 31608 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:45.161154 31608 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.161665 31602 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:45.154418 31645 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:45.124854 31578 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:45.158473 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 107617us
I20260709 13:04:45.158087 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.166637 31607 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:45.104108 31604 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.167665 29908 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:45.168834 29908 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:45.168941 31604 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:45.125797 31448 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.169749 31448 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.170261 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.171651 31448 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:45.078346 31464 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:45.163388 31443 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:04:45.172379 31464 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.165356 29222 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:45.174295 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.174391 31464 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:45.167156 31608 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:45.176366 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:45.176985 29219 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:45.177696 29908 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:45.160238 31668 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:04:45.188124 31636 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:04:45.193754 31607 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:45.194032 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:45.194597 31607 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:45.195577 31687 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:45.196175 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:45.211607 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.212383 29064 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:45.212224 29219 raft_consensus.cc:2410] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:04:45.212301 29039 connection.cc:256] RPC call timeout handler was delayed by -2.6938496239999949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 13:04:45.215037 29039 connection.cc:256] RPC call timeout handler was delayed by -2.0953381480003372s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 13:04:45.216673 31480 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:45.216845 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.217658 29071 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:45.217399 31480 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.217860 31443 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:45.218346 31443 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:45.218665 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.220301 31578 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:45.215350 29039 connection.cc:256] RPC call timeout handler was delayed by -2.0290925010003775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260709 13:04:45.222656 29039 connection.cc:256] RPC call timeout handler was delayed by -1.9021925010001723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
I20260709 13:04:45.222880 31578 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:45.223170 29039 connection.cc:256] RPC call timeout handler was delayed by -1.7563925009999366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
W20260709 13:04:45.223686 29039 connection.cc:256] RPC call timeout handler was delayed by -1.7446925009999177s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
W20260709 13:04:45.224117 29039 connection.cc:256] RPC call timeout handler was delayed by -1.7399415060008323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
W20260709 13:04:45.224467 29039 connection.cc:256] RPC call timeout handler was delayed by -1.6240925010006322s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
W20260709 13:04:45.224834 29039 connection.cc:256] RPC call timeout handler was delayed by -1.5152611050007181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2215s for a response
W20260709 13:04:45.225206 29039 connection.cc:256] RPC call timeout handler was delayed by -1.4170925010002975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24700000000000003s for a response
W20260709 13:04:45.225608 29039 connection.cc:256] RPC call timeout handler was delayed by -1.3661054270005479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3078s for a response
I20260709 13:04:45.219306 31443 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.228000 31577 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:45.230496 29905 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 476) took 3559 ms (client timeout 3094 ms). Trace:
I20260709 13:04:45.229663 31429 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:45.231756 29215 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:04:45.176433 31568 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:45.236840 29911 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:45.219714 31480 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:45.219326 29073 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:45.240190 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.241051 31578 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:45.241421 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 86958.9us
I20260709 13:04:45.241757 31710 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:04:45.246434 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 90348.2us
I20260709 13:04:45.242558 31438 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.244330 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:45.242470 29188 connection.cc:256] RPC call timeout handler was delayed by -1.6046790780001174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
I20260709 13:04:45.247062 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.215395 31604 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.249636 29067 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:45.251281 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:45.251314 29188 connection.cc:256] RPC call timeout handler was delayed by -1.3742790779997449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 13:04:45.254443 29188 connection.cc:256] RPC call timeout handler was delayed by -1.3626248379996468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
W20260709 13:04:45.254792 29188 connection.cc:256] RPC call timeout handler was delayed by -1.3158248379995712s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1817s for a response
W20260709 13:04:45.255046 29188 connection.cc:256] RPC call timeout handler was delayed by -1.2932790779996139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
W20260709 13:04:45.255331 29188 connection.cc:256] RPC call timeout handler was delayed by -1.2600082199996905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
W20260709 13:04:45.255554 29188 connection.cc:256] RPC call timeout handler was delayed by -1.2338790779995179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
W20260709 13:04:45.255786 29188 connection.cc:256] RPC call timeout handler was delayed by -1.1762790779994248s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21250000000000002s for a response
I20260709 13:04:45.234920 31429 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:45.256713 29188 connection.cc:256] RPC call timeout handler was delayed by -1.1754082200004632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 13:04:45.257157 29188 connection.cc:256] RPC call timeout handler was delayed by -1.0925790780001989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
W20260709 13:04:45.257483 29188 connection.cc:256] RPC call timeout handler was delayed by -1.0250935109997954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
W20260709 13:04:45.257869 29188 connection.cc:256] RPC call timeout handler was delayed by -1.0133790780000709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23060000000000003s for a response
W20260709 13:04:45.258322 29188 connection.cc:256] RPC call timeout handler was delayed by -1.0098082200001954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
I20260709 13:04:45.259181 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.259544 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.259881 29064 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:45.260304 29073 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:45.262087 29072 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:45.256417 31645 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:45.248111 31443 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:45.268620 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:45.268975 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.269186 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.273072 29067 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:45.269658 29069 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:45.274745 31710 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.276021 31525 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:45.264396 31578 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.277194 31525 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:45.277374 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.277683 31525 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:45.278111 29217 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:45.279138 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.279295 31578 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:45.278837 31525 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:45.231029 29905 rpcz_store.cc:269] 0709 13:04:41.671087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.678565 (+  7478us) service_pool.cc:224] Handling call
0709 13:04:42.181306 (+502741us) spinlock_profiling.cc:243] Waited 8.62 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:45.230217 (+3048911us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b54005a32d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:45.230398 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67602048}
I20260709 13:04:45.259423 31429 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:45.260107 29903 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:45.280062 31438 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.284410 31547 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:45.287676 29037 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:45.297919 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:45.263114 31604 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:45.299746 31438 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:45.300012 31602 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:45.300379 31438 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:45.300817 31438 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.301414 31438 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:45.290844 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.301602 31602 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:45.302320 29070 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:04:45.289753 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.292449 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.303164 31480 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:45.291548 29219 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:45.288329 31695 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:45.296236 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.303665 29073 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:45.304301 31695 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:45.304502 29069 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:45.304695 31695 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:45.306046 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.306171 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.299996 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:45.301249 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.308069 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.290102 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.309918 29064 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:45.310763 31636 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:45.313337 31480 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:45.313783 31480 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:45.314216 31480 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:42.614054 31279 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:45.305732 31695 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:45.248973 31577 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:45.308238 29071 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:45.320042 29072 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:45.310737 31668 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
W20260709 13:04:45.321766 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.272568 31645 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:45.292474 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.321310 29037 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:45.324359 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:45.330247 29213 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:45.334404 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:45.334582 31645 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:45.343118 29069 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:45.332253 29066 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:45.341614 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:45.320070 31695 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:45.337136 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:45.348946 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:45.349172 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:45.350044 29067 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:45.351267 29073 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:45.296224 31607 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:45.354561 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.356392 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:45.340420 31279 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:45.368294 31279 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.368721 31279 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:45.359974 31578 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:45.376525 31578 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.377022 31578 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:45.377349 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.380311 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.381234 29139 raft_consensus.cc:2410] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:45.381744 31653 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.382597 31525 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:45.358847 29070 raft_consensus.cc:2376] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7. Re-sending same reply.
I20260709 13:04:45.340252 31568 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:45.384363 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:45.384750 31568 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:45.385052 29221 raft_consensus.cc:2376] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6. Re-sending same reply.
I20260709 13:04:45.385304 31568 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:45.386107 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.363186 31710 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:45.387828 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08003c8720 after lost signal to thread 29005
I20260709 13:04:45.388365 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:45.386729 29221 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:45.389294 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:45.389686 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:45.390478 29221 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:45.390519 31461 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:45.392160 31710 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:45.392550 31710 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.392585 29185 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:45.393008 31710 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:45.388403 29002 debug-util.cc:398] Leaking SignalData structure 0x7b0800040fe0 after lost signal to thread 29019
I20260709 13:04:45.364403 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.394312 29213 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:45.395495 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.396229 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:45.386631 31687 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:04:45.397531 31529 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:45.397673 29213 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:45.398180 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.387243 31024 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:45.399389 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.399717 31024 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:45.400301 29146 raft_consensus.cc:2368] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:04:45.400763 31024 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:45.342479 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.402501 29139 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:45.360705 29065 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:45.382284 31653 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:45.422572 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.419998 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.445102 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:45.446026 29908 raft_consensus.cc:2376] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5. Re-sending same reply.
I20260709 13:04:45.391587 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:45.420599 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:45.469133 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 136007us
W20260709 13:04:45.469563 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 64674.7us
I20260709 13:04:45.468245 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.475839 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:45.476022 29186 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:45.486050 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:45.488910 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 125026us
I20260709 13:04:45.482518 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:45.493758 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:45.495273 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:45.495519 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.496569 31464 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:45.504746 31464 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.505177 31464 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:45.505892 31653 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:45.506743 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:45.507623 29906 raft_consensus.cc:2368] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4.
I20260709 13:04:45.508041 31577 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:45.501868 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:45.508633 31577 raft_consensus.cc:2764] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:04:45.508972 29904 raft_consensus.cc:2376] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5. Re-sending same reply.
I20260709 13:04:45.509902 31448 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:45.468873 31618 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:45.512326 29217 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:45.510203 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:45.482237 31608 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:45.516312 29187 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
I20260709 13:04:45.518452 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.518935 29908 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:45.519460 29908 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:45.520488 29908 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:45.524961 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.529090 29908 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:45.493412 31697 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.533233 29221 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:45.538858 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:45.540369 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.510299 31448 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:45.566998 31448 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:45.578296 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:45.579262 29113 leader_election.cc:336] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.882s (SENT)
I20260709 13:04:45.552703 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.535816 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:45.557489 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:45.558264 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.564743 29147 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:45.566303 31547 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:45.581123 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:45.602797 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.615504 31532 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:45.616220 31532 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:45.616825 31532 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:45.619692 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.579092 29065 raft_consensus.cc:2376] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5. Re-sending same reply.
I20260709 13:04:45.618611 31711 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:45.621694 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:45.529328 31531 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 13:04:45.604079 31604 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:45.662838 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.516625 29187 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1907)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1920)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1904)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
I20260709 13:04:45.665624 29140 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:45.544297 31429 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:45.602651 31697 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:45.656571 31711 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:45.697407 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.716509 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.716861 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.715142 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.717638 29220 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:45.720582 31572 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:45.741604 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.748292 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:45.749565 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 245570us
I20260709 13:04:45.750119 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.749045 29220 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
W20260709 13:04:45.756863 31663 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.355s	user 0.019s	sys 0.028s
I20260709 13:04:45.799664 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.804375 29148 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:45.804339 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:45.806900 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.808691 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.713701 31604 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.081s
W20260709 13:04:45.805878 31663 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.404s	user 0.019s	sys 0.029s
I20260709 13:04:45.831765 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:45.832564 29071 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:45.835153 29038 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:45.830278 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 355582us
I20260709 13:04:45.838989 29065 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:04:45.839881 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:45.842554 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.843227 29148 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
W20260709 13:04:45.859537 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.836930 31480 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.522s	user 0.004s	sys 0.008s
I20260709 13:04:45.861315 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:45.861560 29144 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:45.888391 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:45.892935 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 422804us
I20260709 13:04:45.893780 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:45.899423 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:45.905635 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 366943us
I20260709 13:04:45.905753 29112 leader_election.cc:400] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4.
I20260709 13:04:45.910645 29112 leader_election.cc:403] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:45.911827 31621 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:45.898128 31604 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:45.905102 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:45.952908 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.837226 31711 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:45.950416 31525 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.011s
I20260709 13:04:45.891283 31480 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:45.987762 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.989373 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 154380us
I20260709 13:04:45.990672 29038 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:46.034312 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 43700.1us
I20260709 13:04:46.036693 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:45.941843 31604 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:46.056639 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:45.975297 31429 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:46.059672 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:45.984299 29904 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:04:45.957046 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.038085 31730 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:46.128778 31631 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:46.129381 31631 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:46.139379 29874 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:46.139686 29187 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1773)
I20260709 13:04:46.035044 29905 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:46.152498 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.146440 29187 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
I20260709 13:04:46.157631 29187 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42964 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
W20260709 13:04:46.169862 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 256073us
I20260709 13:04:46.178880 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.156093 31697 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:46.193487 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.195102 29187 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
I20260709 13:04:46.167784 31730 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:46.201225 29142 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:46.200029 29187 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
I20260709 13:04:46.207585 29071 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:46.201514 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1750) took 1031 ms. Trace:
I20260709 13:04:46.209654 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.207346 29187 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
W20260709 13:04:46.218568 29139 consensus_meta.cc:287] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.816s	user 0.014s	sys 0.013s
I20260709 13:04:46.218640 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:46.213531 29187 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
I20260709 13:04:46.219575 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.220371 29147 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:46.221158 29187 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
I20260709 13:04:46.224946 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:46.228261 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 191671us
I20260709 13:04:46.232123 29139 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:46.229664 29038 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:46.203271 31461 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:46.234716 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.235734 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:46.231887 29187 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
I20260709 13:04:46.210538 29222 rpcz_store.cc:276] 0709 13:04:45.169621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.175253 (+  5632us) service_pool.cc:224] Handling call
0709 13:04:45.175449 (+   196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.175599 (+   150us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:04:45.240549 (+ 64950us) log.cc:844] Serialized 14 byte log entry
0709 13:04:46.201075 (+960526us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:46.201150 (+    75us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:46.201210 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.201304 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.201442 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6919,"prepare.queue_time_us":109171,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":115114,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":924661,"threads_started":3,"wal-append.queue_time_us":637737}
W20260709 13:04:46.240981 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 181424us
I20260709 13:04:46.218398 31443 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:46.239037 29187 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1938)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
I20260709 13:04:46.243373 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:46.243958 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:46.244447 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 82302.4us
I20260709 13:04:46.216789 31737 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:46.244280 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.247460 29071 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:04:46.237257 31717 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
I20260709 13:04:46.227631 31732 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:46.253734 29874 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:46.241417 31668 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:46.240386 31726 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:46.243773 31525 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:46.249979 31718 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.355s	user 0.000s	sys 0.001s
I20260709 13:04:46.245388 31443 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:46.260867 31279 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.892s	user 0.013s	sys 0.005s
I20260709 13:04:46.260691 31730 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.248301 31727 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:46.265529 31448 consensus_meta.cc:287] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.698s	user 0.005s	sys 0.005s
I20260709 13:04:46.265960 31480 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:46.250342 29904 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:46.266305 29071 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
W20260709 13:04:46.268596 31723 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
W20260709 13:04:46.268635 31608 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.752s	user 0.012s	sys 0.003s
I20260709 13:04:46.246572 31725 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:46.254177 31722 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.381s	user 0.000s	sys 0.001s
I20260709 13:04:46.272675 31525 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.258736 31726 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:46.279752 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.255283 31732 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:46.217053 31604 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.286516 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:46.259286 31024 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:46.289767 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.297834 31024 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:46.279752 29038 leader_election.cc:336] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42964 dropped due to backpressure: service queue is full with 50 items
I20260709 13:04:46.298429 31024 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:46.300014 31618 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.789s	user 0.018s	sys 0.016s
I20260709 13:04:46.300741 31657 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.279428 31443 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:46.286773 31645 consensus_meta.cc:287] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.815s	user 0.014s	sys 0.018s
I20260709 13:04:46.301299 31657 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.313153 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.281661 31727 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:46.286710 31464 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.780s	user 0.019s	sys 0.010s
I20260709 13:04:46.295499 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:46.317349 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.288020 31725 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:46.315053 31529 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.917s	user 0.010s	sys 0.007s
W20260709 13:04:46.327191 29907 consensus_meta.cc:287] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.811s	user 0.010s	sys 0.006s
I20260709 13:04:46.327580 29907 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:46.327888 29139 raft_consensus.cc:2410] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:46.292619 31608 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:46.328208 29065 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:46.296406 31726 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:46.328450 31738 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:04:46.283345 31448 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:46.329870 31735 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:04:46.325556 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 334) took 1275 ms. Trace:
I20260709 13:04:46.331024 31448 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:46.331336 31568 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.945s	user 0.010s	sys 0.001s
I20260709 13:04:46.331934 29220 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:04:46.332683 31710 consensus_meta.cc:287] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.939s	user 0.022s	sys 0.015s
I20260709 13:04:46.330426 29216 rpcz_store.cc:276] 0709 13:04:45.050558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.058385 (+  7827us) service_pool.cc:224] Handling call
0709 13:04:45.061699 (+  3314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:45.102848 (+ 41149us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:45.198547 (+ 95699us) log.cc:844] Serialized 14 byte log entry
0709 13:04:45.630401 (+431854us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:45.635734 (+  5333us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.635873 (+   139us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:45.635944 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.636017 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:46.325304 (+689287us) raft_consensus.cc:1685] finished
0709 13:04:46.325335 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.325479 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35058,"prepare.queue_time_us":162790,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":119899,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":21203,"thread_start_us":428919,"threads_started":2,"wal-append.queue_time_us":559320}
I20260709 13:04:46.333168 31710 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.334362 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:46.334998 31695 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.974s	user 0.006s	sys 0.007s
I20260709 13:04:46.335280 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1925) took 1148 ms. Trace:
I20260709 13:04:46.335441 31710 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:46.327373 31529 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:46.321364 31464 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.337827 31464 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:46.338325 31438 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.027s	user 0.010s	sys 0.009s
W20260709 13:04:46.340636 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 451173us
W20260709 13:04:46.325762 31734 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:04:46.296350 31525 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:46.329730 31725 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:46.335664 29216 rpcz_store.cc:276] 0709 13:04:45.186547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.334083 (+1147536us) service_pool.cc:224] Handling call
0709 13:04:46.335219 (+  1136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.321199 31727 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:46.326745 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:46.322090 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 52557.1us
I20260709 13:04:46.296672 31732 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:46.347755 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.348496 29147 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:46.317009 31744 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:46.345896 29002 debug-util.cc:398] Leaking SignalData structure 0x7b080013b9a0 after lost signal to thread 29097
I20260709 13:04:46.343286 29071 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:46.347117 29068 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:46.281371 31279 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.342451 29186 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:46.351635 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 56118.5us
W20260709 13:04:46.351423 29002 debug-util.cc:398] Leaking SignalData structure 0x7b0800130560 after lost signal to thread 29098
I20260709 13:04:46.354107 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:46.358753 29064 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 6. Ops: []
I20260709 13:04:46.316907 31645 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.361810 29186 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:46.363456 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.360644 31547 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:46.365726 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1906) took 1184 ms. Trace:
I20260709 13:04:46.366156 29216 rpcz_store.cc:276] 0709 13:04:45.181109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.355782 (+1174673us) service_pool.cc:224] Handling call
0709 13:04:46.365633 (+  9851us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.331755 31568 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.365880 31608 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
W20260709 13:04:46.376068 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.377287 31461 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:46.377806 31461 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:46.381457 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:46.382375 29222 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:46.346652 31438 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.360512 31448 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:46.384594 31448 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.385008 31448 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:46.386219 31438 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:46.386893 29214 raft_consensus.cc:2368] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 5. Current term is 6.
I20260709 13:04:46.388643 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:46.392925 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45215.8us
I20260709 13:04:46.396921 29147 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:46.397271 29068 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:46.405755 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.407294 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1918) took 1222 ms. Trace:
I20260709 13:04:46.407891 29216 rpcz_store.cc:276] 0709 13:04:45.184620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.392731 (+1208111us) service_pool.cc:224] Handling call
0709 13:04:46.407077 (+ 14346us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:46.400852 29113 leader_election.cc:336] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items
W20260709 13:04:46.409823 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 48088.9us
I20260709 13:04:46.323565 31443 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.409509 29140 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:46.420423 31578 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.043s	user 0.000s	sys 0.040s
I20260709 13:04:46.420311 31751 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:46.404387 31738 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:46.421564 31738 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.424762 31443 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:46.424800 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1685) took 1135 ms. Trace:
I20260709 13:04:46.425750 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:46.426864 29068 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:46.431885 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.425999 29072 rpcz_store.cc:276] 0709 13:04:45.288982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.292155 (+  3173us) service_pool.cc:224] Handling call
0709 13:04:46.424544 (+1132389us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b5400383e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:46.424685 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37534720}
I20260709 13:04:46.435251 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.439647 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.398897 31753 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:04:46.396169 31531 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:04:46.426378 31608 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:46.444937 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:46.446113 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 92130.9us
I20260709 13:04:46.446707 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.446781 29871 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:46.447351 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.432623 31547 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:46.450748 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.459347 29216 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:46.459544 31547 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.452970 31751 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:04:46.461086 31721 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.390s	user 0.004s	sys 0.000s
I20260709 13:04:46.461326 31751 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.453716 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.456161 31461 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.456188 31727 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.457182 31279 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:46.472883 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.474069 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:46.475013 29068 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:46.460188 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1934) took 1130 ms. Trace:
I20260709 13:04:46.433887 31578 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.465539 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.456310 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:46.467581 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 66778.6us
I20260709 13:04:46.476451 29216 rpcz_store.cc:276] 0709 13:04:45.329784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.450548 (+1120764us) service_pool.cc:224] Handling call
0709 13:04:46.460121 (+  9573us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:46.480957 29113 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items
I20260709 13:04:46.483089 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.445086 31683 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:46.481455 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:46.486279 29072 raft_consensus.cc:2393] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:46.487802 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.488533 29147 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:46.490372 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:46.491760 29068 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:46.491333 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 468) took 1476 ms. Trace:
I20260709 13:04:46.492429 29911 rpcz_store.cc:276] 0709 13:04:45.016754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.018135 (+  1381us) service_pool.cc:224] Handling call
0709 13:04:45.231487 (+213352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.240362 (+  8875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.491145 (+1250783us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54005a32d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:46.491276 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471622400}
I20260709 13:04:46.471740 31608 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:46.489531 31752 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:04:46.496978 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:46.497152 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.498965 29068 raft_consensus.cc:2410] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:46.495981 31538 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:46.499889 31538 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:46.500635 31753 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:46.500980 31538 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.502727 31752 consensus_queue.cc:1243] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:04:46.504132 31754 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:04:46.505084 31461 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:46.495644 31024 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.511070 29222 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:46.511559 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1915) took 1328 ms. Trace:
I20260709 13:04:46.511775 29222 rpcz_store.cc:276] 0709 13:04:45.183557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.380998 (+1197441us) service_pool.cc:224] Handling call
0709 13:04:46.511507 (+130509us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.508142 29187 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1954)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1955)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1787)
W20260709 13:04:46.522691 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.512303 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.489677 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 394) took 1170 ms. Trace:
W20260709 13:04:46.534842 29187 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items [suppressed 11 similar messages]
I20260709 13:04:46.536520 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.537712 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1940) took 1029 ms. Trace:
I20260709 13:04:46.538228 29222 rpcz_store.cc:276] 0709 13:04:45.508613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.536182 (+1027569us) service_pool.cc:224] Handling call
0709 13:04:46.537635 (+  1453us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:46.539148 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 125997us
W20260709 13:04:46.539731 29219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1917) took 1347 ms (client timeout 1586 ms). Trace:
W20260709 13:04:46.540129 29219 rpcz_store.cc:269] 0709 13:04:45.184157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.246811 (+ 62654us) service_pool.cc:224] Handling call
0709 13:04:46.530964 (+1284153us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b54003cdcd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:46.531086 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91899264}
I20260709 13:04:46.541919 29147 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:46.535339 29187 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1954)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1790)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1955)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1787)
I20260709 13:04:46.547729 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.548628 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1905) took 1367 ms. Trace:
I20260709 13:04:46.548661 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.549118 29216 rpcz_store.cc:276] 0709 13:04:45.180866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.544211 (+1363345us) service_pool.cc:224] Handling call
0709 13:04:46.548570 (+  4359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.516243 31645 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:46.542672 31532 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.918s	user 0.018s	sys 0.004s
I20260709 13:04:46.547654 31761 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:46.546983 29187 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1954)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1792)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1790)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1955)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1793)
I20260709 13:04:46.517295 31568 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:46.555284 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:46.553160 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.516913 31732 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:46.561501 29113 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:46.569434 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 122771us
I20260709 13:04:46.570250 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:46.570784 29113 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:46.571632 29113 consensus_peers.cc:597] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:46.563639 31738 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:46.572918 31738 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:46.573490 31738 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:46.574602 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:46.574576 31738 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.575114 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:46.575834 31024 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:46.577865 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.578423 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:46.508697 31608 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.575268 29065 raft_consensus.cc:2955] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:04:46.584313 29147 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:46.584851 29147 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:46.583551 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:46.580739 31761 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.588601 29067 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:04:46.591495 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.581615 29187 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1954)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1956)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1792)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1955)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1793)
W20260709 13:04:46.591872 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 123566us
I20260709 13:04:46.593745 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:46.597038 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:46.597884 29905 raft_consensus.cc:2368] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5.
I20260709 13:04:46.598078 31732 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:46.577209 29907 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:04:46.595065 29187 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42964 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1954)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 348)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1792)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1955)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1793)
I20260709 13:04:46.596297 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:46.599967 29068 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:46.562073 31532 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.598753 31578 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:46.653664 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.681945 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.723275 29187 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1954)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 348)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1792)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1955)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1793)
I20260709 13:04:46.734243 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.740869 29070 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:46.682780 31668 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4.
I20260709 13:04:46.745720 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:46.742498 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.675892 31631 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.004s
I20260709 13:04:46.682924 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:46.675529 29147 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.750551 29187 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1954)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 348)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1792)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1955)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1793)
I20260709 13:04:46.541451 29903 rpcz_store.cc:276] 0709 13:04:45.313705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.316582 (+  2877us) service_pool.cc:224] Handling call
0709 13:04:45.316977 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:45.317447 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:45.317487 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:45.318599 (+  1112us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.728880 (+410281us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:46.033403 (+304523us) log.cc:844] Serialized 14 byte log entry
0709 13:04:46.066906 (+ 33503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:46.123405 (+ 56499us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:46.370463 (+247058us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b48000b13d8. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783ba36aa1 00007f783ba2db37 00007f783ba2d9ca 00007f783bb2526f 00007f783bb24f37 00007f783bb3955c 00007f783bb39508 00007f783bb39488 00007f783bb39443 00007f783bb386ec 00007f783aa62a96
0709 13:04:46.374106 (+  3643us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b18002d4060. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db
0709 13:04:46.460327 (+ 86221us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:46.468452 (+  8125us) raft_consensus.cc:1685] finished
0709 13:04:46.468644 (+   192us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.484190 (+ 15546us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110398,"prepare.queue_time_us":356621,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":134150,"raft.run_cpu_time_us":8980,"raft.run_wall_time_us":1050133,"spinlock_wait_cycles":35716096,"thread_start_us":567377,"threads_started":2,"wal-append.queue_time_us":638749}
I20260709 13:04:46.695456 31725 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.753185 29065 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:46.756222 29147 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:46.760105 29113 leader_election.cc:336] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:42970 dropped due to backpressure: service queue is full with 50 items
I20260709 13:04:46.762612 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1916) took 1261 ms. Trace:
I20260709 13:04:46.770295 29142 rpcz_store.cc:276] 0709 13:04:45.501595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.518119 (+ 16524us) service_pool.cc:224] Handling call
0709 13:04:46.762359 (+1244240us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54002fa3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:46.762479 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341672832}
I20260709 13:04:46.770773 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:46.763207 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.763562 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.772043 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.772142 29038 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.790s (SENT)
I20260709 13:04:46.764482 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:46.774281 31684 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:04:46.774641 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.774888 31568 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:04:46.775411 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.775931 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:46.776387 29143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 332) took 2095 ms (client timeout 1534 ms). Trace:
I20260709 13:04:46.776667 29148 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:46.776777 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:46.777559 29217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1907) took 1596 ms (client timeout 2106 ms). Trace:
I20260709 13:04:46.777659 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:46.778013 29217 rpcz_store.cc:269] 0709 13:04:45.181343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.776501 (+1595158us) service_pool.cc:224] Handling call
0709 13:04:46.777503 (+  1002us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.778324 29148 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:46.778472 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:46.779069 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.779359 29148 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:46.690977 31525 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.779762 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1904) took 1599 ms. Trace:
W20260709 13:04:46.784862 29874 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.903s (SENT)
I20260709 13:04:46.788378 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.781598 29185 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:46.793843 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:46.793680 29220 rpcz_store.cc:276] 0709 13:04:45.180458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.810212 (+629754us) service_pool.cc:224] Handling call
0709 13:04:46.779529 (+969317us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b54002841d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:46.779657 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1129337216}
I20260709 13:04:46.779735 29148 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:46.784508 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08003c8a00 after lost signal to thread 29172
W20260709 13:04:46.796211 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08003c89e0 after lost signal to thread 29173
W20260709 13:04:46.797235 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08003a1480 after lost signal to thread 29246
W20260709 13:04:46.798066 29002 debug-util.cc:398] Leaking SignalData structure 0x7b0800338780 after lost signal to thread 29935
W20260709 13:04:46.798231 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08003f4a80 after lost signal to thread 29936
W20260709 13:04:46.798604 29002 debug-util.cc:398] Leaking SignalData structure 0x7b0800387020 after lost signal to thread 31488
W20260709 13:04:46.798772 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08003f5780 after lost signal to thread 31515
W20260709 13:04:46.798895 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08003b49e0 after lost signal to thread 31517
W20260709 13:04:46.799275 29002 debug-util.cc:398] Leaking SignalData structure 0x7b0800361ca0 after lost signal to thread 31586
W20260709 13:04:46.799413 29002 debug-util.cc:398] Leaking SignalData structure 0x7b080035bc80 after lost signal to thread 31590
W20260709 13:04:46.799576 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08003c6ea0 after lost signal to thread 31598
W20260709 13:04:46.799685 29002 debug-util.cc:398] Leaking SignalData structure 0x7b0800373300 after lost signal to thread 31600
W20260709 13:04:46.799818 29002 debug-util.cc:398] Leaking SignalData structure 0x7b0800379640 after lost signal to thread 31605
W20260709 13:04:46.775447 29214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 339) took 1707 ms (client timeout 1903 ms). Trace:
I20260709 13:04:46.790066 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1941) took 1281 ms. Trace:
I20260709 13:04:46.800846 29070 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:46.800683 29217 rpcz_store.cc:276] 0709 13:04:45.508901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.778802 (+1269901us) service_pool.cc:224] Handling call
0709 13:04:46.790001 (+ 11199us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:46.776818 29143 rpcz_store.cc:269] 0709 13:04:44.680707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.776093 (+2095386us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:46.776195 (+   102us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:04:46.761238 29187 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51820 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1954)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 348)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1784)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1792)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1955)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1793)
W20260709 13:04:46.805516 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08003916e0 after lost signal to thread 31611
W20260709 13:04:46.806221 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08002c50a0 after lost signal to thread 31620
I20260709 13:04:46.806310 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:46.760214 31761 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:46.812909 29222 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:46.813611 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1932) took 1574 ms. Trace:
I20260709 13:04:46.813910 29222 rpcz_store.cc:276] 0709 13:04:45.239581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.773241 (+1533660us) service_pool.cc:224] Handling call
0709 13:04:46.813548 (+ 40307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.814896 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.815290 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:04:46.800433 29214 rpcz_store.cc:269] 0709 13:04:45.068133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.305678 (+237545us) service_pool.cc:224] Handling call
0709 13:04:46.775255 (+1469577us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003a89d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:46.775390 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2483759488}
I20260709 13:04:46.817271 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.818500 29146 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 4: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:46.814838 29037 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:46.762458 31532 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:46.825518 29903 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:46.813120 31024 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:04:46.820096 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:46.822053 31578 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:46.849547 29220 raft_consensus.cc:2376] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5. Re-sending same reply.
I20260709 13:04:46.852186 31578 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.852427 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:46.852662 31578 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:46.784251 31657 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:46.831986 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.752621 31631 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.786113 31726 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.837713 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:46.837832 31730 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:46.838599 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:46.864761 29213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1910) took 1682 ms (client timeout 2093 ms). Trace:
W20260709 13:04:46.865121 29213 rpcz_store.cc:269] 0709 13:04:45.182083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.833370 (+1651287us) service_pool.cc:224] Handling call
0709 13:04:46.864696 (+ 31326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.823493 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.838900 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:46.816083 31444 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.698s	user 0.026s	sys 0.057s
I20260709 13:04:46.816285 29148 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:46.821146 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.866598 29142 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:46.761408 31744 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:46.856213 31279 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:46.775465 31568 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.790108 31725 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:46.821597 31572 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:46.829860 29907 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:46.853058 29065 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:46.854051 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:46.854580 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.852447 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1943) took 1342 ms. Trace:
I20260709 13:04:46.803149 31732 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:46.815618 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1945) took 1305 ms. Trace:
I20260709 13:04:46.860682 31448 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:46.877025 29222 rpcz_store.cc:276] 0709 13:04:45.510205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.814615 (+1304410us) service_pool.cc:224] Handling call
0709 13:04:46.815565 (+   950us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.878105 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:46.878747 31448 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:46.879263 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.879506 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:46.888726 29072 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:46.861804 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:46.889189 29220 rpcz_store.cc:276] 0709 13:04:45.509553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.795638 (+1286085us) service_pool.cc:224] Handling call
0709 13:04:46.852385 (+ 56747us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.889935 29070 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:46.850132 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:46.875434 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 60678.1us
I20260709 13:04:46.876809 31684 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:46.840750 31711 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.003s	sys 0.020s
I20260709 13:04:46.880147 29222 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:46.883517 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:46.898887 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:46.886015 31607 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:46.887593 29148 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:46.860795 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 75969.2us
I20260709 13:04:46.902400 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1919) took 1717 ms. Trace:
I20260709 13:04:46.892712 29146 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:46.902858 29222 rpcz_store.cc:276] 0709 13:04:45.184905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.879007 (+1694102us) service_pool.cc:224] Handling call
0709 13:04:46.902335 (+ 23328us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.903743 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.895246 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:46.894055 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.897773 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1914) took 1714 ms. Trace:
I20260709 13:04:46.904692 29066 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:46.904788 29214 rpcz_store.cc:276] 0709 13:04:45.183233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.817950 (+1634717us) service_pool.cc:224] Handling call
0709 13:04:46.897682 (+ 79732us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:46.894873 31444 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.797s	user 0.026s	sys 0.058s
I20260709 13:04:46.884459 31738 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:46.906106 29037 leader_election.cc:400] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5.
I20260709 13:04:46.873705 31749 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:46.907498 29147 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:46.907773 29037 leader_election.cc:403] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:46.908906 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:46.902101 31711 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.004s	sys 0.022s
I20260709 13:04:46.910920 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.911756 29222 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:46.912158 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1952) took 1092 ms. Trace:
I20260709 13:04:46.912104 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:46.912808 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.913978 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.915028 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.869234 31744 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:46.912602 29222 rpcz_store.cc:276] 0709 13:04:45.819700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.910618 (+1090918us) service_pool.cc:224] Handling call
0709 13:04:46.912103 (+  1485us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.920003 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.920786 29143 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:46.887115 31572 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:46.924517 29142 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:46.924535 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.925496 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:46.909370 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.936493 29065 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:46.943681 29072 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:04:46.945077 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.945981 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:46.948554 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:46.902400 31768 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.950230 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.950840 29142 raft_consensus.cc:2368] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:46.902752 31607 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.949368 29072 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:46.949105 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:46.957034 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1946) took 1446 ms. Trace:
I20260709 13:04:46.957437 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:46.957481 29222 rpcz_store.cc:276] 0709 13:04:45.510503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.916763 (+1406260us) service_pool.cc:224] Handling call
0709 13:04:46.956949 (+ 40186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:46.933658 31608 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:46.998555 29066 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:47.000730 29065 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:47.001160 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 67815.4us
I20260709 13:04:47.001355 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.002527 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.003054 29142 raft_consensus.cc:2376] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5. Re-sending same reply.
I20260709 13:04:47.003909 29066 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:04:47.005707 29065 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:04:47.006265 31774 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:04:46.945251 31279 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.008627 29072 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:46.956343 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:47.012564 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:46.964007 31607 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.016480 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:47.017756 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:47.018337 31770 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:04:47.020319 31279 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.022848 29903 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
W20260709 13:04:47.024819 31771 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:04:47.028965 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.060190 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:47.047214 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:47.060803 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.030969 29143 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:47.034905 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.036758 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:47.064167 29142 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:47.064474 29065 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:47.007330 31751 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.991384 31768 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.066020 31781 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5.
I20260709 13:04:47.066933 31781 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:46.910802 31749 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.009377 31608 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:47.050791 29906 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.051635 29216 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
W20260709 13:04:47.037509 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 131518us
W20260709 13:04:47.037007 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 124972us
I20260709 13:04:47.055130 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:47.058871 29218 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:47.078687 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:47.066128 29907 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:47.038192 31444 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:60814 (request call id 10) took 10771 ms. Trace:
I20260709 13:04:46.907598 31684 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.069216 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:47.046563 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 240366us
I20260709 13:04:47.078130 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:47.033536 29066 pending_rounds.cc:85] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 4
I20260709 13:04:47.079617 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:47.079487 29218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1936) took 1678 ms (client timeout 1790 ms). Trace:
I20260709 13:04:47.081300 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1908) took 1899 ms. Trace:
I20260709 13:04:47.082288 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:47.078691 29147 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
W20260709 13:04:47.086196 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48571.1us
I20260709 13:04:47.089123 29066 pending_rounds.cc:107] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:04:47.088753 29222 rpcz_store.cc:276] 0709 13:04:45.181589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.008088 (+1826499us) service_pool.cc:224] Handling call
0709 13:04:47.081240 (+ 73152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.089584 29066 raft_consensus.cc:2889] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:47.090022 29188 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:47.088608 29218 rpcz_store.cc:269] 0709 13:04:45.401200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.016183 (+1614983us) service_pool.cc:224] Handling call
0709 13:04:47.079430 (+ 63247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.095171 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:47.096343 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:47.097288 31631 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:47.098253 31745 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:47.098481 31780 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:47.098734 31745 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:47.099267 31745 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:47.100347 29073 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:47.101646 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:47.102772 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:47.103277 29073 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:47.104616 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:47.105163 29073 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:47.106711 31525 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:47.108145 31639 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:47.108815 31639 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.109215 31639 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:47.086309 31461 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.112110 31726 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:47.112129 31781 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:47.113261 31781 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:47.113848 31781 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:47.085726 31444 rpcz_store.cc:276] 0709 13:04:36.242605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.261445 (+ 18840us) service_pool.cc:224] Handling call
0709 13:04:36.364199 (+102754us) log.cc:844] Serialized 506 byte log entry
0709 13:04:36.728275 (+364076us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b18004230a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20522 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:04:37.147134 (+418859us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b18004230a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:04:46.990445 (+9843311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38898,"raft.queue_time_us":137273,"raft.run_cpu_time_us":2094,"raft.run_wall_time_us":132949,"spinlock_wait_cycles":25175040,"thread_start_us":348649,"threads_started":3,"wal-append.queue_time_us":231210,"wal-append.run_cpu_time_us":2441,"wal-append.run_wall_time_us":1096744}
I20260709 13:04:47.115393 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.058171 31279 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.116215 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1944) took 1606 ms. Trace:
I20260709 13:04:47.116581 29217 rpcz_store.cc:276] 0709 13:04:45.509846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.115103 (+1605257us) service_pool.cc:224] Handling call
0709 13:04:47.116158 (+  1055us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.116987 29145 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:47.077137 31751 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.084331 31510 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:47.117947 31608 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.118654 31510 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.118979 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:47.119611 31510 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:47.120123 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:47.121196 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.121724 29142 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:47.117096 29071 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:47.125077 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.113854 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1738) took 1168 ms. Trace:
I20260709 13:04:47.125574 29071 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:47.112927 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 251) took 1783 ms. Trace:
W20260709 13:04:47.127655 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.127770 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:47.129076 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.131170 29070 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:47.133759 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.134575 29216 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:47.135031 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1765) took 1956 ms. Trace:
I20260709 13:04:47.135365 29070 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:47.135310 29216 rpcz_store.cc:276] 0709 13:04:45.178946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.107185 (+1928239us) service_pool.cc:224] Handling call
0709 13:04:47.134977 (+ 27792us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:47.136360 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 54094.7us
I20260709 13:04:47.125782 29068 rpcz_store.cc:276] 0709 13:04:45.945381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.531057 (+585676us) service_pool.cc:224] Handling call
0709 13:04:46.599793 (+ 68736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:46.717466 (+117673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.113671 (+396205us) spinlock_profiling.cc:243] Waited 68.4 ms on lock 0x7b54002d4450. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:47.113792 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157353856}
I20260709 13:04:47.136860 29072 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:47.126843 29064 rpcz_store.cc:276] 0709 13:04:45.335623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.335751 (+   128us) service_pool.cc:224] Handling call
0709 13:04:45.355568 (+ 19817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:46.359277 (+1003709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.112672 (+753395us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400383954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:47.112852 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2306479360}
I20260709 13:04:47.137194 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:47.138602 31568 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:47.138998 29215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1749) took 1969 ms (client timeout 1786 ms). Trace:
W20260709 13:04:47.139372 29215 rpcz_store.cc:269] 0709 13:04:45.169320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.170013 (+   693us) service_pool.cc:224] Handling call
0709 13:04:47.138827 (+1968814us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b54003395d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:47.138928 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140036480}
I20260709 13:04:47.139714 31524 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:47.140223 31524 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:47.141486 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.141222 31524 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.112808 31684 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:47.142387 31524 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:47.142280 29221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1913) took 1959 ms (client timeout 2028 ms). Trace:
I20260709 13:04:47.145267 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:47.145740 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55828.6us
I20260709 13:04:47.148066 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.148746 29214 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:47.149308 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1770) took 1011 ms. Trace:
I20260709 13:04:47.149611 29214 rpcz_store.cc:276] 0709 13:04:46.137829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.144319 (+1006490us) service_pool.cc:224] Handling call
0709 13:04:47.149234 (+  4915us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.148134 31730 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:47.145522 29221 rpcz_store.cc:269] 0709 13:04:45.182937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.141160 (+1958223us) service_pool.cc:224] Handling call
0709 13:04:47.142219 (+  1059us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.150710 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.151438 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:47.151527 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44461.6us
I20260709 13:04:47.152228 29221 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:47.152633 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1948) took 1641 ms. Trace:
I20260709 13:04:47.153020 31448 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:47.155090 31751 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:47.159258 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 337) took 2373 ms. Trace:
I20260709 13:04:47.159672 29144 rpcz_store.cc:276] 0709 13:04:44.785968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.396831 (+610863us) service_pool.cc:224] Handling call
0709 13:04:45.397171 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.875285 (+478114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.159014 (+1283729us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b540029de54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:47.159177 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1061126912}
I20260709 13:04:47.160187 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.160992 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1903) took 1980 ms. Trace:
I20260709 13:04:47.161309 29222 rpcz_store.cc:276] 0709 13:04:45.180216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.144683 (+1964467us) service_pool.cc:224] Handling call
0709 13:04:47.160922 (+ 16239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.159077 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:47.162230 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:47.165520 31768 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:47.166566 31732 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:47.166967 31732 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:47.167902 31732 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.169483 31732 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:47.152926 29221 rpcz_store.cc:276] 0709 13:04:45.511088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.151207 (+1640119us) service_pool.cc:224] Handling call
0709 13:04:47.152573 (+  1366us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.178625 29222 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:47.180625 29143 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:47.183478 31279 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:47.183732 31727 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:47.186648 31607 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:47.188593 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.189271 29222 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:47.189553 31745 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.190321 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.191058 29222 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:47.191731 31745 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:47.193095 29145 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:47.193585 31615 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.194092 31615 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:47.196792 31615 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:47.200594 29214 raft_consensus.cc:2368] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6.
I20260709 13:04:47.201571 29906 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:47.203104 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.203462 29065 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:47.206187 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.253074 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1737) took 1343 ms. Trace:
I20260709 13:04:47.191821 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.254305 29222 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:47.254752 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1931) took 2066 ms. Trace:
I20260709 13:04:47.255103 29222 rpcz_store.cc:276] 0709 13:04:45.189866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.191578 (+2001712us) service_pool.cc:224] Handling call
0709 13:04:47.254696 (+ 63118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.208425 31749 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:47.208776 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.252436 29214 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:47.263442 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1949) took 1563 ms. Trace:
I20260709 13:04:47.263834 29218 rpcz_store.cc:276] 0709 13:04:45.700529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.208517 (+1507988us) service_pool.cc:224] Handling call
0709 13:04:47.263344 (+ 54827us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.211740 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1909) took 5674 ms. Trace:
W20260709 13:04:47.213586 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 66909.3us
I20260709 13:04:47.265966 29142 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:47.266884 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:47.230239 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.212721 31607 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:47.238694 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:47.237866 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59331.6us
I20260709 13:04:47.265548 29145 rpcz_store.cc:276] 0709 13:04:41.537726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.545010 (+  7284us) service_pool.cc:224] Handling call
0709 13:04:41.545258 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.553162 (+4007904us) spinlock_profiling.cc:243] Waited 3.43 s on lock 0x7b1800125ca0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b48646 00007f7836a8258e 00007f7836b03f9e 00007f7836b057de 00007f7836b0fb49 00007f7836b10416 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4
0709 13:04:47.116594 (+1563432us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5800491fd0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a690aa 00007f7836aeb447 00007f7836b03fe7 00007f7836b057de 00007f7836b0fb49 00007f7836b10416 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4
0709 13:04:47.193461 (+ 76867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.211679 (+ 18218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10816576000}
I20260709 13:04:47.276305 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:47.245143 29037 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:47.253566 29067 rpcz_store.cc:276] 0709 13:04:45.909645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.384782 (+475137us) service_pool.cc:224] Handling call
0709 13:04:46.385185 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:46.589156 (+203971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.252811 (+663655us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54001d8b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:47.253015 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467081600}
I20260709 13:04:47.237834 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.273695 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:47.244808 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.282196 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:47.284144 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.284397 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45772.4us
W20260709 13:04:47.285578 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.286249 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:47.263012 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:47.291646 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.292671 29217 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:47.294049 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.285604 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1950) took 1568 ms. Trace:
I20260709 13:04:47.298362 29215 rpcz_store.cc:276] 0709 13:04:45.717442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.237518 (+1520076us) service_pool.cc:224] Handling call
0709 13:04:47.285542 (+ 48024us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.301812 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1911) took 2119 ms. Trace:
I20260709 13:04:47.302315 29217 rpcz_store.cc:276] 0709 13:04:45.182370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.256960 (+2074590us) service_pool.cc:224] Handling call
0709 13:04:47.301734 (+ 44774us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.306995 29157 tablet_copy_source_session.cc:215] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:04:47.247154 31572 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:47.293324 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:47.325923 29222 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.320139 29216 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:47.320785 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.324613 29218 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:47.330523 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1751) took 2159 ms. Trace:
I20260709 13:04:47.330950 29218 rpcz_store.cc:276] 0709 13:04:45.170989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.264739 (+2093750us) service_pool.cc:224] Handling call
0709 13:04:47.316394 (+ 51655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.330186 (+ 13792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.330453 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:47.332938 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 50806.4us
I20260709 13:04:47.336506 29221 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:47.340080 29217 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 6. Ops: []
I20260709 13:04:47.288523 31739 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:47.356480 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1753) took 2165 ms. Trace:
I20260709 13:04:47.358368 29221 rpcz_store.cc:276] 0709 13:04:45.171568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.301384 (+2129816us) service_pool.cc:224] Handling call
0709 13:04:47.336239 (+ 34855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.337048 (+   809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.337279 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.359606 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 350) took 1139 ms. Trace:
I20260709 13:04:47.356873 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1758) took 2165 ms. Trace:
I20260709 13:04:47.356223 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:47.361106 29217 rpcz_store.cc:276] 0709 13:04:45.176845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.303101 (+2126256us) service_pool.cc:224] Handling call
0709 13:04:47.339822 (+ 36721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.341967 (+  2145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.342351 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.341218 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:47.341883 29911 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:47.316282 31464 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.291216 31568 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:47.297037 31777 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.290978 31727 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:47.290742 31279 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:47.371986 29218 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:47.376175 29186 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:47.374888 29220 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:47.378262 29216 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:04:47.378765 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1759) took 2201 ms. Trace:
I20260709 13:04:47.379020 29216 rpcz_store.cc:276] 0709 13:04:45.177357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.329086 (+2151729us) service_pool.cc:224] Handling call
0709 13:04:47.367015 (+ 37929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.378587 (+ 11572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.378723 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.360680 29141 rpcz_store.cc:276] 0709 13:04:46.220215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.220569 (+   354us) service_pool.cc:224] Handling call
0709 13:04:46.236063 (+ 15494us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:46.247191 (+ 11128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:46.247215 (+    24us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:46.248970 (+  1755us) log.cc:844] Serialized 52 byte log entry
0709 13:04:47.110723 (+861753us) spinlock_profiling.cc:243] Waited 9.75 ms on lock 0x7b1800496300. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:04:47.190480 (+ 79757us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b1800496300. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1
0709 13:04:47.190635 (+   155us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:47.343096 (+152461us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.359177 (+ 16081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.359279 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:47.359335 (+    56us) raft_consensus.cc:1685] finished
0709 13:04:47.359353 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.359529 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15769,"raft.queue_time_us":40458,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":8891,"spinlock_wait_cycles":51126016,"thread_start_us":154405,"threads_started":2,"wal-append.queue_time_us":33820}
W20260709 13:04:47.382169 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.384299 29221 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:47.384769 29065 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:47.385107 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1763) took 2206 ms. Trace:
I20260709 13:04:47.386930 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1761) took 2209 ms. Trace:
I20260709 13:04:47.387399 29218 rpcz_store.cc:276] 0709 13:04:45.177910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.332166 (+2154256us) service_pool.cc:224] Handling call
0709 13:04:47.371738 (+ 39572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.386626 (+ 14888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.386862 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.385586 29221 rpcz_store.cc:276] 0709 13:04:45.178411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.359345 (+2180934us) service_pool.cc:224] Handling call
0709 13:04:47.381173 (+ 21828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.384805 (+  3632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.385040 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.389357 31510 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:47.390301 29221 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:47.391003 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1912) took 2208 ms. Trace:
I20260709 13:04:47.390787 31661 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:47.391948 31661 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:47.392373 31661 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.393253 31661 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:47.334754 29214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3842b36ffd134cd28c657475df48e741"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:47.396695 31730 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.372570 31457 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:04:47.401683 31781 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:47.403981 31618 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:04:47.404873 31618 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:47.405254 31618 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.405014 31639 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:47.405397 29157 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:60830 (request call id 333) took 2724 ms. Trace:
I20260709 13:04:47.390266 31527 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:47.406186 31618 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:47.406700 31527 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.407227 31639 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:47.407591 31618 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:47.381315 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:47.357286 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:47.408772 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:47.425267 29038 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:47.409757 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.404178 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 70053.6us
I20260709 13:04:47.394485 31661 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.411201 29221 rpcz_store.cc:276] 0709 13:04:45.182650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.389571 (+2206921us) service_pool.cc:224] Handling call
0709 13:04:47.389929 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.390760 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.390951 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.417924 29216 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:47.426970 29143 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:47.435738 29222 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:47.429819 31604 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:47.431521 31572 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:47.318331 31668 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:04:47.432116 29214 raft_consensus.cc:606] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:47.441690 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1768) took 2262 ms. Trace:
I20260709 13:04:47.442087 29216 rpcz_store.cc:276] 0709 13:04:45.179680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.379583 (+2199903us) service_pool.cc:224] Handling call
0709 13:04:47.417740 (+ 38157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.441445 (+ 23705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.441634 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.443574 29218 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:47.444394 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1902) took 2264 ms. Trace:
I20260709 13:04:47.444774 29218 rpcz_store.cc:276] 0709 13:04:45.179966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.388360 (+2208394us) service_pool.cc:224] Handling call
0709 13:04:47.437105 (+ 48745us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.444082 (+  6977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.444331 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.382735 31615 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:04:47.428100 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:47.437207 29215 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:47.382143 31780 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:47.403769 31781 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:47.335026 31729 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 13:04:47.380144 31568 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:47.460862 29065 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:47.477401 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.450536 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:47.448976 29220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1923) took 2263 ms (client timeout 2705 ms). Trace:
I20260709 13:04:47.462898 29911 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:04:47.498842 29220 rpcz_store.cc:269] 0709 13:04:45.185959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.898698 (+1712739us) service_pool.cc:224] Handling call
0709 13:04:47.448785 (+550087us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54002f5dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:47.448919 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009686784}
I20260709 13:04:47.405921 29157 rpcz_store.cc:276] 0709 13:04:44.681113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.681733 (+   620us) service_pool.cc:224] Handling call
0709 13:04:47.306497 (+2624764us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b54000ea6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b173f7 00007f783baad036 00007f783bab6a3e 00007f783337a27b 00007f783bab69ba 00007f783bab1acc 00007f783baacc7b 00007f783baa18d1 00007f783ab554b4 00007f783ab55442
0709 13:04:47.405271 (+ 98774us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5563807616}
I20260709 13:04:47.500203 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:47.501052 31578 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.633s	user 0.012s	sys 0.000s
I20260709 13:04:47.501458 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:47.502836 29220 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:04:47.503408 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1928) took 2316 ms. Trace:
I20260709 13:04:47.504532 31450 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:47.504957 31450 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.505906 31450 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.507328 31450 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:47.508664 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:47.501478 31578 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:47.512827 29219 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:47.412087 31727 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:47.482363 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:47.437657 31024 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:47.484009 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:47.433095 31532 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:47.438678 29905 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:47.503707 29220 rpcz_store.cc:276] 0709 13:04:45.187402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.502282 (+2314880us) service_pool.cc:224] Handling call
0709 13:04:47.502598 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.503181 (+   583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.503358 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.537813 29218 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:47.538491 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1942) took 2029 ms. Trace:
I20260709 13:04:47.538595 29220 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:47.538830 29218 rpcz_store.cc:276] 0709 13:04:45.509216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.537320 (+2028104us) service_pool.cc:224] Handling call
0709 13:04:47.537650 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.538253 (+   603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.538435 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.539288 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1951) took 1744 ms. Trace:
I20260709 13:04:47.450544 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:47.530629 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.535809 31631 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:47.536664 31790 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:04:47.514876 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 64434.5us
I20260709 13:04:47.485543 31615 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:47.536980 31532 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:47.537413 31727 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.539687 29220 rpcz_store.cc:276] 0709 13:04:45.795653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.538137 (+1742484us) service_pool.cc:224] Handling call
0709 13:04:47.538429 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.539041 (+   612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.539235 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:47.579008 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 128523us
I20260709 13:04:47.481374 31279 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:47.552292 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.581305 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:47.540114 29218 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:47.546514 29142 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:47.513573 29216 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:47.544673 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.553498 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:47.585583 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:47.481025 31568 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.563123 29146 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:47.546975 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:47.407609 31527 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:47.561504 31729 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:04:47.545063 31578 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:47.583256 29038 leader_election.cc:400] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6.
I20260709 13:04:47.591297 29038 leader_election.cc:403] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:47.592626 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:47.585526 29073 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:47.587872 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.593935 29147 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:47.593973 29140 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:47.443193 31457 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 13:04:47.491551 31663 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 13:04:47.562245 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.548250 31350 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:04:47.548705 31668 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:04:47.590556 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:47.579959 31781 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:47.617065 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:47.593181 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:47.600274 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.534879 31749 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:47.601119 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1754) took 2429 ms. Trace:
I20260709 13:04:47.618525 29215 rpcz_store.cc:276] 0709 13:04:45.171833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.303281 (+2131448us) service_pool.cc:224] Handling call
0709 13:04:47.303665 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.462049 (+158384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.600888 (+138839us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b540023c654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:47.601060 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306710784}
I20260709 13:04:47.619511 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:47.552836 29906 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:47.602146 29068 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:47.595590 29220 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:47.620433 29215 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:47.601421 29216 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:47.622099 31636 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:47.622247 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:47.621879 31532 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.622892 29065 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:47.622938 31661 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:47.624310 31731 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:47.624986 31444 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:04:47.628042 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.630062 31777 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.632248 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:47.632829 31527 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.580103 31631 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.640053 31749 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:47.641057 31457 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:47.641626 31663 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:47.642418 31668 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:47.643353 31683 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:04:47.643440 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 50311.8us
W20260709 13:04:47.646032 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.647465 29067 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:47.596571 31607 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.581401 31726 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.581504 31615 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.651942 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:47.588816 31608 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
W20260709 13:04:47.660190 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 297902us
I20260709 13:04:47.861581 31645 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:47.666702 29147 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:47.862257 31538 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.727125 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1769) took 1581 ms. Trace:
I20260709 13:04:47.862814 31538 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.863044 29218 rpcz_store.cc:276] 0709 13:04:46.137463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.539639 (+1402176us) service_pool.cc:224] Handling call
0709 13:04:47.539946 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.671479 (+131533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.719032 (+ 47553us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:47.760604 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 141215us
W20260709 13:04:47.713682 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 121149us
I20260709 13:04:47.865254 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1955) took 1471 ms. Trace:
I20260709 13:04:47.594182 31711 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.869310 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1772) took 1730 ms. Trace:
I20260709 13:04:47.833670 29222 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:47.870496 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1786) took 1363 ms. Trace:
I20260709 13:04:47.865746 29216 rpcz_store.cc:276] 0709 13:04:46.394079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.585077 (+1190998us) service_pool.cc:224] Handling call
0709 13:04:47.601257 (+ 16180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.865000 (+263743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.865202 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:47.771212 29219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1766) took 2592 ms (client timeout 2481 ms). Trace:
I20260709 13:04:47.871099 29222 rpcz_store.cc:276] 0709 13:04:46.507257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.833190 (+1325933us) service_pool.cc:224] Handling call
0709 13:04:47.833499 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.870175 (+ 36676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.870421 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.711185 29071 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
W20260709 13:04:47.871886 29219 rpcz_store.cc:269] 0709 13:04:45.179193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.831552 (+1652359us) service_pool.cc:224] Handling call
0709 13:04:47.770914 (+939362us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54003cdcd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:47.771116 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1506995072}
I20260709 13:04:47.865530 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:47.875083 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.875137 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:47.867256 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1785) took 1360 ms. Trace:
I20260709 13:04:47.783021 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 20) took 2514 ms. Trace:
I20260709 13:04:47.876516 29214 rpcz_store.cc:276] 0709 13:04:45.272696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.263608 (+1990912us) service_pool.cc:224] Handling call
0709 13:04:47.782762 (+519154us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b54003a89d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b07853 00007f783bad560a 00007f783505b7d4 00007f783505b762 00007f783505b691 00007f783505b60c 00007f783505a8c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:47.782886 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84167552}
I20260709 13:04:47.887171 29070 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:47.884145 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.883579 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.889027 31448 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:47.889317 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1793) took 1343 ms. Trace:
I20260709 13:04:47.890121 31573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79cdfc6a725145348b70034422ef477b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:47.889743 29219 rpcz_store.cc:276] 0709 13:04:46.546186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.873102 (+1326916us) service_pool.cc:224] Handling call
0709 13:04:47.888766 (+ 15664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.888949 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.888979 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.889040 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:47.889068 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.889133 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.889249 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:47.779947 31732 consensus_meta.cc:287] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.004s
I20260709 13:04:47.848639 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 1927) took 2661 ms. Trace:
I20260709 13:04:47.876350 29215 rpcz_store.cc:276] 0709 13:04:46.507014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.619747 (+1112733us) service_pool.cc:224] Handling call
0709 13:04:47.620169 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.866885 (+246716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.867183 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:47.922003 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.923187 29215 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:47.869907 29220 rpcz_store.cc:276] 0709 13:04:46.138419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.578790 (+1440371us) service_pool.cc:224] Handling call
0709 13:04:47.595425 (+ 16635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.868943 (+273518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.869242 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:47.933467 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45159.1us
W20260709 13:04:47.935629 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.936105 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1791) took 1390 ms. Trace:
I20260709 13:04:47.940349 29222 rpcz_store.cc:276] 0709 13:04:46.545397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.872489 (+1327092us) service_pool.cc:224] Handling call
0709 13:04:47.935480 (+ 62991us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.935726 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.935758 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.935802 (+    44us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:47.935834 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.935891 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.936035 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.943005 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:47.862282 31645 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.961603 29219 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:47.876418 31631 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:47.876523 31457 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.868005 31661 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.977569 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 352) took 1074 ms. Trace:
W20260709 13:04:47.965118 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.910332 29221 rpcz_store.cc:276] 0709 13:04:45.187159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.434171 (+2247012us) service_pool.cc:224] Handling call
0709 13:04:47.434503 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.434688 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:47.458809 (+ 24121us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.848134 (+389325us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.848204 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:47.848261 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.848372 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.848552 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3536,"prepare.queue_time_us":29915,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":8587,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":10960,"thread_start_us":349226,"threads_started":1,"wal-append.queue_time_us":350047}
I20260709 13:04:47.967161 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 354) took 1040 ms. Trace:
I20260709 13:04:47.918402 31726 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:47.872818 31663 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.876049 31749 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.962652 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 350) took 1061 ms. Trace:
I20260709 13:04:48.004873 29219 rpcz_store.cc:276] 0709 13:04:46.900863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.890934 (+990071us) service_pool.cc:224] Handling call
0709 13:04:47.961356 (+ 70422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.962314 (+   958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.962575 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.870594 31636 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.876277 31668 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:47.875869 31793 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.890872 31729 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:47.956610 31732 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.994307 31656 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.965775 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 353) took 1039 ms. Trace:
I20260709 13:04:47.878302 31739 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:48.000605 29222 rpcz_store.cc:276] 0709 13:04:46.926639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.941633 (+1014994us) service_pool.cc:224] Handling call
0709 13:04:47.966520 (+ 24887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.966742 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.966779 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.966833 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:47.966877 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.966946 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.967088 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:48.001317 29215 rpcz_store.cc:276] 0709 13:04:46.902876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.921736 (+1018860us) service_pool.cc:224] Handling call
0709 13:04:47.922965 (+  1229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.959596 (+ 36631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.977291 (+ 17695us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.998651 29213 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:48.005928 29874 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:48.020881 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 77977.7us
I20260709 13:04:48.032508 29188 leader_election.cc:400] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:48.033272 29188 leader_election.cc:403] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:04:48.034020 29220 rpcz_store.cc:276] 0709 13:04:46.926280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.928940 (+1002660us) service_pool.cc:224] Handling call
0709 13:04:47.965057 (+ 36117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.965286 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.965327 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.965385 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:47.965433 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.965506 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.965682 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:47.727452 31744 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.039351 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:48.017704 31604 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:48.012799 31608 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:48.041957 31618 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.000s
I20260709 13:04:48.031347 31538 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:48.052436 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:48.040592 29070 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:48.040760 31656 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:48.018293 31729 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:48.058295 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.038414 31572 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.053188 29214 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:48.042943 31687 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:04:48.052627 31450 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.545s	user 0.004s	sys 0.008s
I20260709 13:04:48.260799 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1801) took 1232 ms. Trace:
I20260709 13:04:48.067888 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:48.261360 29214 rpcz_store.cc:276] 0709 13:04:47.028326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.052561 (+1024235us) service_pool.cc:224] Handling call
0709 13:04:48.052928 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.260444 (+207516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.260720 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:48.053622 31024 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.060128 31744 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:48.045418 31798 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:48.023327 31615 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:48.054262 31715 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:48.065089 31525 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:04:48.110661 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 273644us
I20260709 13:04:48.310087 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:48.056468 31279 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:04:48.116291 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 241230us
I20260709 13:04:48.041597 31793 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:48.158799 29213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1775) took 2012 ms (client timeout 1636 ms). Trace:
W20260709 13:04:48.319041 29213 rpcz_store.cc:269] 0709 13:04:46.145956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.883242 (+737286us) service_pool.cc:224] Handling call
0709 13:04:48.158527 (+1275285us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540026bbd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:48.158700 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2420081920}
W20260709 13:04:48.238612 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.051350 31777 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:04:48.079876 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 258) took 1506 ms. Trace:
I20260709 13:04:48.039685 31645 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.079252 31604 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:48.269968 31656 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:48.271570 31794 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.453s	user 0.002s	sys 0.007s
I20260709 13:04:48.151319 31683 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:48.278824 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:48.282336 31457 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:48.287864 29216 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:48.289314 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:48.344360 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:48.345105 29139 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:48.346098 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:48.063778 31608 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.088663 31778 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:48.053023 31636 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.115980 31668 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:48.378027 29069 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:48.303901 31687 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:48.315577 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:48.382901 29071 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: [3.3-6.4]
W20260709 13:04:48.316442 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 438886us
I20260709 13:04:48.317951 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:48.320717 29066 rpcz_store.cc:276] 0709 13:04:46.573766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.004709 (+430943us) service_pool.cc:224] Handling call
0709 13:04:47.033196 (+ 28487us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:47.180437 (+147241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.180480 (+    43us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:47.257429 (+ 76949us) log.cc:844] Serialized 78 byte log entry
0709 13:04:47.438333 (+180904us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:47.491489 (+ 53156us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.491803 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.824685 (+332882us) log.cc:844] Serialized 14 byte log entry
0709 13:04:48.078625 (+253940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.078717 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:48.078760 (+    43us) raft_consensus.cc:1685] finished
0709 13:04:48.078771 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.079813 (+  1042us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32636,"prepare.queue_time_us":73708,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":112,"raft.queue_time_us":279032,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":334,"thread_start_us":712006,"threads_started":3,"wal-append.queue_time_us":302924}
I20260709 13:04:48.304435 31578 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.329650 29214 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:48.336105 29220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
W20260709 13:04:48.390273 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 72403.2us
I20260709 13:04:48.390869 29220 raft_consensus.cc:606] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:48.391280 29220 raft_consensus.cc:614] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:48.323619 31350 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.260653 31572 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:48.090131 31663 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:48.301024 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 233214us
W20260709 13:04:48.414140 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 98698.8us
I20260709 13:04:48.368434 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:48.369213 29039 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.022s	sys 0.084s
W20260709 13:04:48.415550 29039 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.022s	sys 0.084s
W20260709 13:04:48.416091 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 551537us
I20260709 13:04:48.417227 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:48.369720 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:48.418247 29148 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:48.376650 29068 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:48.325147 31450 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:48.299604 29146 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:48.378593 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:48.379904 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:48.389789 31457 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:48.389111 29073 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:48.391805 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:48.392580 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:48.396426 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:48.427080 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.427376 31753 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:04:48.397006 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.428187 29220 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:04:48.428740 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.429355 31504 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:48.430418 31504 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:48.430665 29905 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:48.430807 31504 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:48.420977 31806 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.337626 31729 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.431975 31683 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.433130 31279 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:48.435745 31527 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:48.436404 31527 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:48.396438 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:48.428234 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:04:48.438757 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.398572 29072 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:48.416200 31780 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.952s	user 0.011s	sys 0.001s
I20260709 13:04:48.400961 29222 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:48.404500 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 493) took 1163 ms. Trace:
I20260709 13:04:48.414984 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:48.419975 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.381544 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:48.422325 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:48.423391 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:48.387861 31731 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:48.414903 31726 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:48.411828 31444 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:48.438758 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.429445 29220 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:48.448314 29911 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:48.449020 29069 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:48.391606 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:48.412407 31350 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:48.488027 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 73121.1us
I20260709 13:04:48.421525 31450 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:48.447431 29908 rpcz_store.cc:276] 0709 13:04:47.241032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.246022 (+  4990us) service_pool.cc:224] Handling call
0709 13:04:47.246465 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.246743 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.246780 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.272573 (+ 25793us) log.cc:844] Serialized 26 byte log entry
0709 13:04:47.618626 (+346053us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:47.618664 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.618737 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:48.270809 (+652072us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:48.396020 (+125211us) raft_consensus.cc:1685] finished
0709 13:04:48.396052 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.404387 (+  8335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5879040,"thread_start_us":302384,"threads_started":1,"wal-append.queue_time_us":420989}
I20260709 13:04:48.462471 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:48.467000 29037 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:48.440099 31663 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:48.436522 29904 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:48.442842 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:48.473886 29139 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
W20260709 13:04:48.498149 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.385003 31661 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:48.577278 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 110358us
W20260709 13:04:48.577471 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 58148.7us
I20260709 13:04:48.491722 31780 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.456388 31753 raft_consensus.cc:2764] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:04:48.507865 29148 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:48.502344 31450 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:48.575390 29910 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:04:48.582743 29143 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:48.424758 31808 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 13:04:48.487843 31578 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.516019 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:48.526088 29908 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:04:48.585983 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 60819.4us
I20260709 13:04:48.495867 31525 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:48.594066 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 93139.6us
W20260709 13:04:48.594952 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 180399us
I20260709 13:04:48.451457 31607 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
W20260709 13:04:48.602424 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 86501.9us
I20260709 13:04:48.463234 31731 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.607604 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:48.590953 31568 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.974s	user 0.010s	sys 0.001s
I20260709 13:04:48.450407 31687 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.431871 31504 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:48.617766 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:48.614548 31024 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:48.620846 31024 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:48.622123 31024 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:48.617481 31780 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.594347 31726 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:04:48.624316 31780 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.601187 31661 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:48.626381 31780 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:48.627147 31608 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:48.598605 31527 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:48.605846 31781 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.954s	user 0.009s	sys 0.003s
I20260709 13:04:48.632268 31781 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:48.633160 29070 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:48.634071 31350 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:48.634933 31350 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:48.493664 31604 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.610051 31753 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:48.618662 31568 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:48.640424 31799 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:48.642896 31568 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:48.643651 29217 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:48.524032 31805 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.109s
W20260709 13:04:48.652252 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.661959 29147 raft_consensus.cc:2410] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:04:48.663060 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:48.663189 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63149.9us
I20260709 13:04:48.603089 31803 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:48.677651 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 360024us
I20260709 13:04:48.594758 31809 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:04:48.661623 31607 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:48.660239 31525 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:48.614218 31796 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:48.706831 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 365) took 1152 ms. Trace:
I20260709 13:04:48.729445 29146 rpcz_store.cc:276] 0709 13:04:47.554165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.592492 (+ 38327us) service_pool.cc:224] Handling call
0709 13:04:47.592749 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.421763 (+829014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.706531 (+284768us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b5400192154. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:48.706756 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1624640384}
I20260709 13:04:48.720093 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1799) took 1878 ms. Trace:
I20260709 13:04:48.670187 31781 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.730958 29216 rpcz_store.cc:276] 0709 13:04:46.841579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.873510 (+1031931us) service_pool.cc:224] Handling call
0709 13:04:47.935369 (+ 61859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.347738 (+412369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.719693 (+371955us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b54003ccdd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:48.719987 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810152192}
I20260709 13:04:48.705560 29188 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:48.693984 31803 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:48.748888 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:48.756407 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 50918.3us
I20260709 13:04:48.757603 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:48.769713 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 378159us
I20260709 13:04:48.770812 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:48.771467 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 247353us
I20260709 13:04:48.658689 31798 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.107s
I20260709 13:04:48.695794 31806 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 13:04:48.777395 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 336410us
I20260709 13:04:48.725342 31607 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:48.777518 29113 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:48.784137 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:48.785377 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 167711us
I20260709 13:04:48.788813 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 355) took 1779 ms. Trace:
I20260709 13:04:48.754765 31803 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:48.789904 29221 rpcz_store.cc:276] 0709 13:04:47.010065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.042305 (+1032240us) service_pool.cc:224] Handling call
0709 13:04:48.054693 (+ 12388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.054892 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:48.501351 (+446459us) log.cc:844] Serialized 14 byte log entry
0709 13:04:48.788291 (+286940us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:48.788359 (+    68us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:48.788423 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.788589 (+   166us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.788737 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8425,"prepare.queue_time_us":589294,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":8604,"thread_start_us":723959,"threads_started":2}
I20260709 13:04:48.774197 31578 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:04:48.793030 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 268915us
W20260709 13:04:48.801354 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 218608us
I20260709 13:04:48.779376 31350 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:48.786041 31607 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:48.755512 31781 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.811298 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:48.791986 31568 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.768594 31796 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.776705 31683 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:04:48.819562 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:48.817229 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 223412us
I20260709 13:04:48.822491 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:48.847481 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:48.849165 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:48.850015 29069 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:48.852878 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:48.853586 29072 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:48.853669 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:48.854072 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:48.854388 29073 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:48.856984 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:48.857659 29073 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:48.867295 29066 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:04:48.807111 31801 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.803507 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:48.814361 31808 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:48.872995 29221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
W20260709 13:04:48.820178 31631 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.825s	user 0.013s	sys 0.001s
I20260709 13:04:48.878123 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:48.813122 31819 raft_consensus.cc:2764] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:04:48.884514 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:48.884980 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 73672.4us
I20260709 13:04:48.880673 29221 raft_consensus.cc:606] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:48.887197 29221 raft_consensus.cc:614] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:48.879720 29073 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:48.883695 29066 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
W20260709 13:04:48.892361 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55777.6us
W20260709 13:04:48.896059 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 54471us
I20260709 13:04:48.897495 29072 raft_consensus.cc:2368] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
I20260709 13:04:48.900532 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:48.901551 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:48.871156 31527 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.815717 31350 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:48.909001 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:48.909859 29908 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
W20260709 13:04:48.911013 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 89111.6us
I20260709 13:04:48.815999 31778 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:04:48.904727 31731 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:48.922736 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1803) took 1484 ms. Trace:
W20260709 13:04:48.923810 31813 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:04:48.923218 29219 rpcz_store.cc:276] 0709 13:04:47.438467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.066153 (+627686us) service_pool.cc:224] Handling call
0709 13:04:48.089056 (+ 22903us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:48.155875 (+ 66819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:48.155936 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:48.259340 (+103404us) log.cc:844] Serialized 26 byte log entry
0709 13:04:48.657958 (+398618us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:48.658018 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.658124 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:48.922481 (+264357us) raft_consensus.cc:1685] finished
0709 13:04:48.922505 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.922665 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5134,"raft.queue_time_us":635,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":90922,"thread_start_us":331566,"threads_started":1,"wal-append.queue_time_us":487974}
I20260709 13:04:48.930518 31350 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:48.945598 31796 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:48.948073 31781 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:48.948839 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:48.951891 31525 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.880597 31801 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:48.955636 29069 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:48.956857 31801 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:48.963478 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:48.969322 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 184188us
I20260709 13:04:48.880419 31808 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:48.897789 31738 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:48.980703 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.897750 31828 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:48.975843 29220 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:04:48.901610 31604 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:48.968436 31819 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:48.971638 31350 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.887454 31631 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:48.999713 31749 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.950s	user 0.011s	sys 0.002s
W20260709 13:04:49.032855 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 62551.6us
W20260709 13:04:49.033130 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.049219 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.049690 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.050144 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.051606 29112 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:49.026283 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 126843us
I20260709 13:04:48.972519 31836 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.963636 31739 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 13:04:49.053865 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 141660us
W20260709 13:04:49.055361 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 161297us
I20260709 13:04:49.072470 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.005029 31801 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:49.088227 29140 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:04:49.089001 29144 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
W20260709 13:04:49.036062 31457 consensus_meta.cc:287] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.003s
I20260709 13:04:49.004135 31841 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:04:49.004580 31604 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:49.016153 31828 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:04:49.087772 31732 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.046s	user 0.007s	sys 0.006s
I20260709 13:04:49.037639 31661 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.060410 31824 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:04:49.115864 31739 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:49.088195 31801 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:04:49.117862 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet 2e40d8d1f334428db02ee54c8467d058: Illegal state: Replica a936878f59c840a6bd176a1d2d7347b1 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:04:49.037168 31350 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.119853 29220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "813e480732fd4061837e784d4d768b1f"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:04:49.108121 31839 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.084354 31836 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.064330 31837 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:49.141129 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 240868us
I20260709 13:04:49.148008 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:49.142230 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 244728us
I20260709 13:04:49.001310 31738 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.056478 31749 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.150774 29909 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:49.151683 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 251414us
I20260709 13:04:49.114535 31604 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:49.139523 31744 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.834s	user 0.001s	sys 0.011s
I20260709 13:04:49.155778 29113 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.136065 31732 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6.
I20260709 13:04:49.011694 31825 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.142s
I20260709 13:04:49.091265 31532 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:04:49.140612 31663 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:49.181677 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.181519 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.140421 31527 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
W20260709 13:04:49.153376 31821 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.171s	user 0.002s	sys 0.001s
W20260709 13:04:49.187397 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 135900us
I20260709 13:04:49.087558 31808 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.196441 31808 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:49.197855 31749 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:49.209821 31724 consensus_queue.cc:1175] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a936878f59c840a6bd176a1d2d7347b1" responder_term: 7 status { last_received { term: 7 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 7 index: 7 } }
W20260709 13:04:49.211354 31835 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:04:49.212481 31738 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:49.183117 29906 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:49.196583 29065 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 6. Ops: []
W20260709 13:04:49.214665 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 129417us
I20260709 13:04:49.210178 31836 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.138872 31661 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.220161 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:49.225975 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.225853 29185 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:49.214910 31834 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:04:49.234450 29185 leader_election.cc:400] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
I20260709 13:04:49.234854 29185 leader_election.cc:403] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:04:49.238849 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:49.239466 29073 raft_consensus.cc:2376] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7. Re-sending same reply.
I20260709 13:04:49.213928 31538 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.227001 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:49.256946 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 166597us
I20260709 13:04:49.260749 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:49.261387 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:49.261389 29072 raft_consensus.cc:2376] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5. Re-sending same reply.
I20260709 13:04:49.225145 31663 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:49.234608 31841 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:49.245569 31825 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:49.269738 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:49.251724 31833 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:49.221300 31532 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:04:49.244611 31739 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:49.152545 31683 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 13:04:49.235041 31738 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:49.222299 31731 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:04:49.267669 31843 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:49.280529 31793 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:49.282831 31444 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:49.284781 31615 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.285071 31618 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:49.286679 31504 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.291652 31778 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:49.295639 31811 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.600s	user 0.002s	sys 0.000s
I20260709 13:04:49.296939 31621 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.256865 31817 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.284s
I20260709 13:04:49.159741 31842 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.122s
W20260709 13:04:49.271808 31715 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.945s	user 0.014s	sys 0.000s
I20260709 13:04:49.217936 31801 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.275770 31749 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:49.267024 31631 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:49.272691 31841 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.268853 31819 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:49.267349 31527 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:49.323004 29113 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.271050 31825 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:49.222299 31837 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:04:49.260555 31538 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.152949 31839 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:49.308933 31450 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.707s	user 0.013s	sys 0.000s
I20260709 13:04:49.305018 31833 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.225487 31578 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:04:49.267239 31836 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 13:04:49.418572 31607 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.547s	user 0.006s	sys 0.004s
I20260709 13:04:49.349124 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.354712 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:49.328969 31444 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.314468 31739 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:49.330735 31618 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.349275 31715 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:49.360993 31817 raft_consensus.cc:3055] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:49.360754 31731 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:49.361811 31858 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.363054 31857 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:49.335738 31621 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.365335 29903 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:49.368443 31661 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.325425 31663 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:04:49.370183 31801 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.443430 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:49.429683 31817 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:49.329979 31639 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.013s	sys 0.010s
I20260709 13:04:49.330768 31820 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:49.313520 31738 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.336158 31829 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:49.416941 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 94007.9us
I20260709 13:04:49.424245 31607 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.419803 31856 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:49.403060 31527 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:49.360162 31350 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:49.422477 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 350112us
I20260709 13:04:49.265945 31828 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:49.424412 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 243007us
I20260709 13:04:49.395972 31841 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:49.412667 31810 log.cc:927] Time spent T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.740s	user 0.000s	sys 0.012s
I20260709 13:04:49.415082 31833 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:49.432418 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 162776us
I20260709 13:04:49.424019 31444 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.349754 31778 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.483307 31843 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
I20260709 13:04:49.484555 31778 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:49.428085 31618 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.441809 29140 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:49.403671 31861 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:04:49.375893 31572 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:49.414734 31450 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.434424 31857 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:49.417021 31825 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:49.456102 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.457985 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.500000 29143 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:49.458566 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:49.500571 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44573.6us
I20260709 13:04:49.501279 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:49.449709 31639 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.014s	sys 0.011s
I20260709 13:04:49.462661 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:49.500430 29069 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:49.448627 31687 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:49.491259 29067 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:04:49.506785 29067 pending_rounds.cc:85] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 4
I20260709 13:04:49.507057 29067 pending_rounds.cc:107] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:04:49.507289 29067 raft_consensus.cc:2889] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:49.451187 31829 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.455262 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 502) took 1035 ms. Trace:
I20260709 13:04:49.497880 31527 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.498905 31450 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:49.458127 31631 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:04:49.521126 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.521806 29066 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:49.523062 29066 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:49.509927 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:49.505511 29173 heartbeater.cc:646] Failed to heartbeat to 127.27.71.190:37537 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.27.71.190:37537 timed out after 15.000s (SENT)
W20260709 13:04:49.511153 31854 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:04:49.511226 31572 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:49.547711 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 503) took 1123 ms. Trace:
I20260709 13:04:49.548171 29909 rpcz_store.cc:276] 0709 13:04:48.424105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.424224 (+   119us) service_pool.cc:224] Handling call
0709 13:04:48.432227 (+  8003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.151354 (+719127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.547436 (+396082us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b5400023d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:49.547647 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651877248}
W20260709 13:04:49.549418 31803 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.747s	user 0.011s	sys 0.000s
I20260709 13:04:49.549718 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:49.549845 31803 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.551036 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:49.550957 31604 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.552181 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:49.553419 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:49.553763 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1785) took 1178 ms. Trace:
I20260709 13:04:49.554186 29065 rpcz_store.cc:276] 0709 13:04:48.375112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.375527 (+   415us) service_pool.cc:224] Handling call
0709 13:04:48.375796 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.215293 (+839497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.553523 (+338230us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b5400383e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:49.553702 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1886957312}
I20260709 13:04:49.555868 31607 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:49.516477 29903 rpcz_store.cc:276] 0709 13:04:48.416746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.420953 (+  4207us) service_pool.cc:224] Handling call
0709 13:04:49.364879 (+943926us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:49.365122 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.442501 (+ 77379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.451995 (+  9494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10326784}
I20260709 13:04:49.557668 31645 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.558326 29903 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:49.564594 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1787) took 3057 ms. Trace:
I20260709 13:04:49.565043 29218 rpcz_store.cc:276] 0709 13:04:46.507490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.863825 (+1356335us) service_pool.cc:224] Handling call
0709 13:04:47.864248 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.952184 (+ 87936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.952222 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.975372 (+ 23150us) log.cc:844] Serialized 26 byte log entry
0709 13:04:48.327275 (+351903us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:48.676857 (+349582us) log.cc:844] Serialized 14 byte log entry
0709 13:04:48.677181 (+   324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.677288 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.367745 (+690457us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:49.564241 (+196496us) raft_consensus.cc:1685] finished
0709 13:04:49.564343 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.564520 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":317180,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":49009,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":18868,"spinlock_wait_cycles":235646848,"thread_start_us":602094,"threads_started":2,"wal-append.queue_time_us":593691}
I20260709 13:04:49.567931 31578 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:49.568538 31578 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.569767 29070 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:49.569918 28968 catalog_manager.cc:5697] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 2 to 4, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 4 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:49.512279 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:49.572540 31738 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:49.572744 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.572846 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:49.573529 31527 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.513134 31858 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:49.499125 31848 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:04:49.575203 31858 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.490953 31806 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.575703 31858 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.576910 31806 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.577932 31858 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.579034 31806 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.519556 29067 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:04:49.524066 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.513890 31464 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:49.508543 31817 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.494762 31739 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.533449 31866 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:49.454775 31820 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.535140 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:49.516196 31829 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:49.601209 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.536747 31636 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.536917 31621 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.516559 31745 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:04:49.537451 31780 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:49.538404 31853 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:49.539104 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.612970 29066 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:49.537108 31839 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.540512 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.543834 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 1691) took 1179 ms. Trace:
I20260709 13:04:49.614737 29071 raft_consensus.cc:2410] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:49.614409 29147 rpcz_store.cc:276] 0709 13:04:48.364925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.365259 (+   334us) service_pool.cc:224] Handling call
0709 13:04:49.543591 (+1178332us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b540027d8d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:49.543767 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567014656}
I20260709 13:04:49.544452 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:49.545298 31825 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.570732 31578 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:49.508069 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 504) took 1080 ms. Trace:
I20260709 13:04:49.490178 31856 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:49.579763 29068 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:49.586123 29220 raft_consensus.cc:606] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:49.623509 29220 raft_consensus.cc:614] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:49.587813 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:49.588696 29909 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:49.587154 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.590010 31715 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
W20260709 13:04:49.606411 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.627498 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.628108 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.585095 31799 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:04:49.589912 31809 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:49.611302 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:49.631942 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45049us
I20260709 13:04:49.450235 31819 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.608516 31753 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.637966 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:49.608330 31450 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.638856 29143 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:49.638551 29188 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.638772 31450 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.641644 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:49.610718 31780 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:49.612614 31853 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:49.616746 31817 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.637120 31828 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.634389 31806 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:49.638550 31753 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.672027 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1764) took 4493 ms. Trace:
I20260709 13:04:49.672803 31753 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:49.674750 31618 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.676570 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.677217 29147 raft_consensus.cc:2376] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7. Re-sending same reply.
I20260709 13:04:49.678125 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.678678 29147 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:49.679464 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:49.679587 31820 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.658514 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:49.681277 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.683550 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 364) took 2135 ms. Trace:
I20260709 13:04:49.683943 29144 rpcz_store.cc:276] 0709 13:04:47.547807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.551707 (+  3900us) service_pool.cc:224] Handling call
0709 13:04:47.552107 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.091046 (+1538939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.683322 (+592276us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54003467d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:49.683480 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3534041344}
I20260709 13:04:49.672425 29217 rpcz_store.cc:276] 0709 13:04:45.178689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.362118 (+2183429us) service_pool.cc:224] Handling call
0709 13:04:47.384861 (+ 22743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:48.644142 (+1259281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.671766 (+1027624us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400282dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:49.671967 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2894477056}
I20260709 13:04:49.684978 29188 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.610859 31444 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.689015 29147 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:49.662092 29139 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:49.663228 31715 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.636793 31578 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:49.657111 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.691428 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:49.666363 31828 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.667335 31780 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.669358 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.642544 31839 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.687247 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:49.688297 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.660794 31687 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:49.648824 29903 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:49.692113 29141 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:49.692243 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.712304 29147 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:49.622632 31841 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:49.693668 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.694772 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.694885 31856 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.695619 31668 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:49.696043 31621 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:49.696802 31715 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.697343 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.697862 31450 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.697965 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.713826 31687 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:49.698961 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:49.699541 31873 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:04:49.699962 31803 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:49.649329 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:49.748548 31532 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:49.750114 29071 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:49.680796 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:04:49.687953 31748 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:49.702564 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.756743 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.757228 29070 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:49.757400 29073 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:49.757512 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.758098 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.758318 29219 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:49.706508 29217 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:49.706519 29143 raft_consensus.cc:2368] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5.
I20260709 13:04:49.708387 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.656458 31607 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:49.665807 31806 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.713104 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.763054 29147 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:49.761114 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:49.764272 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:49.765057 29147 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:49.765265 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:49.765532 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 366) took 1337 ms. Trace:
I20260709 13:04:49.686825 31538 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.721828 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.722726 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.700140 31578 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:49.732918 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.770399 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.770435 29213 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:49.738343 29216 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:49.739106 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.772719 29222 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 3.
I20260709 13:04:49.655367 31809 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.743564 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.773964 29141 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:49.702921 31753 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:04:49.745796 31879 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:04:49.746124 31844 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.751223 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.780519 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.783061 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:49.783385 29066 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:49.777470 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.761783 31532 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:49.734444 31621 consensus_queue.cc:1243] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:49.760948 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:49.787801 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.788174 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.754952 29139 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:49.733914 31799 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:49.790967 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:49.765879 29145 rpcz_store.cc:276] 0709 13:04:48.427932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.453415 (+ 25483us) service_pool.cc:224] Handling call
0709 13:04:49.543742 (+1090327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.578824 (+ 35082us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:49.579091 (+   267us) log.cc:844] Serialized 14 byte log entry
0709 13:04:49.628650 (+ 49559us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:49.628713 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:49.628774 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.629265 (+   491us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.765368 (+136103us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540027d8d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:49.765469 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6241,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":113,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":2408121216,"thread_start_us":48828,"threads_started":1,"wal-append.queue_time_us":49140}
I20260709 13:04:49.770846 29215 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:49.792330 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.771055 29067 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:49.717015 31839 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.741456 31739 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.743197 31604 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:49.777768 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.778170 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.782491 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:49.806048 29066 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:04:49.806346 29213 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:49.785382 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.807578 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.783560 29219 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:49.786863 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.771570 31450 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:49.765784 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.766026 29217 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:49.782923 31837 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:49.788663 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:49.814536 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 96299.7us
I20260709 13:04:49.814896 29222 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:49.788776 29214 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:49.698632 31464 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:49.824437 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.794448 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.774919 31866 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:49.798026 29113 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:49.801312 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:49.802501 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:49.827668 29143 raft_consensus.cc:2410] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:49.803169 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:49.828222 29215 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.793742 31798 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:49.777379 31745 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:04:49.803936 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:49.785697 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet b09437a3ae8d46379ccdbb1829b14f27: Illegal state: Replica 730186a8c70f43f2a57d80163bbc37d2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:04:49.794140 31607 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.787465 31618 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.766772 31806 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:49.844924 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:49.808064 31656 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:49.815646 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:49.800660 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:49.827736 29113 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:49.852267 29139 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 3.
W20260709 13:04:49.861361 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 60800.3us
I20260709 13:04:49.862370 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.785262 31844 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:49.825021 31874 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:04:49.873158 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:49.852320 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:49.874827 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:49.876042 29213 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
W20260709 13:04:49.858404 31876 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:04:49.863188 29139 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:49.864020 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:49.864926 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 49345.2us
I20260709 13:04:49.880828 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:49.829396 31798 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.882619 31798 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.817766 31450 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:49.829749 31845 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:49.875859 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:49.758733 31853 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:49.891075 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 63414.6us
I20260709 13:04:49.900851 29220 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:49.903138 29219 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
W20260709 13:04:49.891469 31880 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:04:49.616518 29902 rpcz_store.cc:276] 0709 13:04:48.425720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.436769 (+ 11049us) service_pool.cc:224] Handling call
0709 13:04:48.501255 (+ 64486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.106360 (+605105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.108439 (+  2079us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.181956 (+ 73517us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.439278 (+257322us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:49.445557 (+  6279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.445718 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.492265 (+ 46547us) raft_consensus.cc:1685] finished
0709 13:04:49.492287 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.506393 (+ 14106us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":205844,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5820,"thread_start_us":638784,"threads_started":2,"wal-append.queue_time_us":245295}
I20260709 13:04:49.816563 31739 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.890074 31663 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:49.824975 31464 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:49.830727 31803 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:49.888315 29144 raft_consensus.cc:2410] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:49.925714 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:49.905562 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:49.810573 29066 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:49.906090 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.855139 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:49.892681 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 51271.9us
I20260709 13:04:49.899326 29218 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:49.897255 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:49.877046 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.916721 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:49.805146 31639 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:04:49.831607 31837 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.928911 29143 raft_consensus.cc:2410] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:49.877977 31538 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:49.878151 31781 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:49.896024 31615 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:49.932857 29188 leader_election.cc:336] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.619s (SENT)
I20260709 13:04:49.933225 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:49.803048 31799 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.934592 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:49.950860 29216 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.935878 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.872887 31607 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:49.891748 31884 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.158s
I20260709 13:04:49.893290 31450 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:49.956106 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:49.946751 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:49.902562 31444 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:49.930382 31886 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:04:49.959604 29067 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:49.955314 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.924638 31573 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:54892 (request call id 460) took 10207 ms. Trace:
I20260709 13:04:49.930047 31663 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.928876 31801 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:49.954691 31869 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:49.899768 31731 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:04:49.966024 29213 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:49.985939 31749 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:49.986325 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:49.986706 31749 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.987155 29213 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:49.989200 31607 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:49.904488 31618 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:04:49.941629 31538 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.025003 31538 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.027352 31844 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:49.940783 31803 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:50.029462 31803 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:49.959560 31778 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.032253 31778 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.033749 31799 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.034718 31778 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.977834 31444 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:50.035640 31444 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.036093 31444 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.964154 31656 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:50.040220 31656 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.040781 31656 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.990501 29145 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:49.993048 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:50.048846 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55877.9us
I20260709 13:04:49.945779 31350 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:49.946388 31615 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.997689 29219 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:49.997607 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:50.065038 29067 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:49.998781 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:50.004431 31862 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:04:50.004935 31878 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:04:50.006059 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:50.077881 29065 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:50.009461 31883 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:04:49.974251 31604 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:50.010298 31882 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:04:49.896078 31857 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.069139 31615 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.064877 31350 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:49.929704 31849 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:04:50.013144 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:50.021499 31739 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.050034 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:50.125813 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 75855.3us
I20260709 13:04:50.126830 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:50.128351 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:49.993884 31850 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:49.931075 31845 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.050662 29144 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:49.953765 31853 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.054778 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:50.116667 31636 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 13:04:49.952471 31809 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:49.892468 31578 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.079102 31738 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:04:50.116225 31778 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.006960 31464 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.120976 31350 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.088025 31798 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.129700 31857 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.975219 31663 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:50.072291 31833 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.579s	user 0.007s	sys 0.004s
I20260709 13:04:50.111052 29145 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:04:50.159229 29145 pending_rounds.cc:85] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 2
I20260709 13:04:50.159570 29145 pending_rounds.cc:107] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:04:50.159854 29145 raft_consensus.cc:2889] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:49.965631 31841 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.123567 29065 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:50.133116 31820 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:50.150128 31645 consensus_meta.cc:287] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.592s	user 0.012s	sys 0.000s
I20260709 13:04:49.969612 31828 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:50.138317 29146 raft_consensus.cc:2410] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:50.176296 29145 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:50.175066 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.097967 29216 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:04:50.165266 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 209232us
I20260709 13:04:50.182329 31833 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.183250 29039 leader_election.cc:400] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5.
I20260709 13:04:50.183611 29143 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:04:50.183638 29039 leader_election.cc:403] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:04:50.185539 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:50.186765 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:50.187884 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.188141 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:50.055907 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:50.137362 31724 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:50.131692 29072 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:50.109723 31829 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.136656 31839 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.164563 31867 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:50.189360 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 133584us
I20260709 13:04:50.192291 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:50.129522 31845 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.198464 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.198760 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:50.199465 29142 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:04:50.174095 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 257527us
I20260709 13:04:50.212308 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:50.173647 29215 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:50.200839 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:50.213857 29072 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:50.167025 31464 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:50.203464 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.181380 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.221221 29218 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:50.193878 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:50.205863 29067 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:50.206096 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:50.224165 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:50.208760 29145 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:50.218078 29215 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:50.226715 29215 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:04:50.155848 31778 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.197010 31858 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:50.201705 31781 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:50.231475 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.235414 29910 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:04:50.241358 29216 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:50.203812 31828 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:50.249747 31828 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:50.250691 31828 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.188952 31861 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.252147 31731 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:50.252220 31861 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.204018 31889 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:50.205675 31867 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:04:50.237989 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.254279 31867 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.970387 31573 rpcz_store.cc:276] 0709 13:04:39.694317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.703065 (+  8748us) service_pool.cc:224] Handling call
0709 13:04:41.322887 (+1619822us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e91a2 00007f783c92abb9 00007f783ba5bd88 00007f783bb1fa07 00007f783bb51f4e 00007f783bb36fdd 00007f783bb36f89 00007f783bb36f19 00007f783bb36ee1
0709 13:04:42.089447 (+766560us) ts_tablet_manager.cc:934] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
0709 13:04:47.874817 (+5785370us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:04:49.251812 (+1376995us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:49.264262 (+ 12450us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:04:49.752944 (+488682us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:04:49.786547 (+ 33603us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:49.836857 (+ 50310us) ts_tablet_manager.cc:989] Replica registered.
0709 13:04:49.897011 (+ 60154us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":303,"spinlock_wait_cycles":18639232,"tablet-copy.queue_time_us":208618,"thread_start_us":2478153,"threads_started":2}
I20260709 13:04:50.256331 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.256907 31573 tablet_copy_client.cc:806] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:04:50.257530 31573 tablet_copy_client.cc:670] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:04:50.257875 29216 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:50.257777 31889 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:50.258841 31889 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:50.259217 31889 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:50.260123 31889 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.261458 31889 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:50.262374 31615 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.248638 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.237323 31748 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:50.237219 31568 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:04:50.239547 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.175836 31450 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:50.243024 31668 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.004s
I20260709 13:04:50.273409 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.276996 31893 raft_consensus.cc:2764] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:04:50.292520 31893 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:50.293370 31893 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:50.293915 31894 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:50.293866 31893 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.295946 29147 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:50.237219 31841 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:50.230652 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:50.285068 29215 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:50.289940 31450 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.289739 31538 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.291580 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:50.116312 31753 consensus_queue.cc:1175] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a936878f59c840a6bd176a1d2d7347b1" responder_term: 5 status { last_received { term: 4 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:04:50.340713 31891 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:04:50.341377 31778 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.342301 31631 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:50.342763 31631 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:50.344669 31798 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.325250 31806 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:50.360869 29038 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.552s (SENT) [suppressed 92 similar messages]
W20260709 13:04:50.362972 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 71483us
I20260709 13:04:50.322099 31809 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.319380 31568 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:50.323666 31841 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:50.367874 31636 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:50.369174 31538 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.369678 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 424) took 1045 ms. Trace:
I20260709 13:04:50.294553 31894 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.372215 29068 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:50.372699 31833 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.374321 31615 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.323061 31837 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:50.374188 31806 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.380508 29911 rpcz_store.cc:276] 0709 13:04:49.324616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.324837 (+   221us) service_pool.cc:224] Handling call
0709 13:04:49.325191 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:49.329838 (+  4647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.329865 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:49.748174 (+418309us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b2c00012928. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:04:49.748523 (+   349us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.176321 (+427798us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:50.176396 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.176490 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.369342 (+192852us) raft_consensus.cc:1685] finished
0709 13:04:50.369367 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.369610 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4264,"raft.queue_time_us":5127,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":2029,"spinlock_wait_cycles":3506560,"thread_start_us":427393,"threads_started":1,"wal-append.queue_time_us":596791}
I20260709 13:04:50.394449 29216 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:04:50.383204 31841 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:50.397221 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 210533us
I20260709 13:04:50.375581 31656 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.382633 31636 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:50.395419 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 156812us
I20260709 13:04:50.357220 31867 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.402670 31845 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:50.403589 31893 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.403851 31867 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.405042 31568 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.403501 31350 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.413287 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:50.413923 29139 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:50.413998 31350 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.414763 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:50.415342 29139 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:50.411530 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:50.419843 29185 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:50.419414 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:50.421521 29147 raft_consensus.cc:2368] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
I20260709 13:04:50.422560 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:50.423460 29147 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:50.343736 31631 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.432847 29139 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:50.379560 31809 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.434695 29147 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:50.401780 31636 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:50.418200 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:04:50.436595 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 75778.8us
I20260709 13:04:50.433691 31781 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5.
I20260709 13:04:50.446076 31663 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:50.446841 31817 consensus_meta.cc:287] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.791s	user 0.007s	sys 0.002s
I20260709 13:04:50.451241 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:50.452292 29139 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:50.457691 29147 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:04:50.457832 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:50.436457 31844 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:50.416396 31837 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:04:50.435498 31668 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:04:50.459709 31905 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:50.500387 31905 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:50.467075 31901 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:50.489346 31656 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.489029 31444 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.489617 31845 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:50.467856 31844 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.468523 31829 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.483394 31796 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:50.491250 31894 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.491593 31663 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:50.495286 31687 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.736s	user 0.010s	sys 0.001s
I20260709 13:04:50.527735 31687 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.529662 29144 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:04:50.529994 31687 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:50.503249 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:50.506278 31803 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.501762 31905 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:50.520795 31749 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.533s	user 0.014s	sys 0.000s
I20260709 13:04:50.508261 31845 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:50.518826 31796 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.521437 31444 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:50.532701 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64921.9us
I20260709 13:04:50.526369 31636 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.507097 31867 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:04:50.536880 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 117155us
I20260709 13:04:50.509327 31844 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:50.518126 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 275955us
I20260709 13:04:50.540330 29065 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:50.542143 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:50.543556 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.549230 29214 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:04:50.534184 31799 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:50.552340 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:50.553411 31636 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.554952 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:50.556259 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:50.557241 29907 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:04:50.556428 29903 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:50.551823 31867 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.561729 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:50.562981 31903 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:50.563494 31903 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.563673 31867 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.564009 31903 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.543080 31796 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:50.568419 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:50.568908 29072 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:50.569365 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.569895 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:50.571561 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:50.537293 31803 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.574954 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:50.575603 29066 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.569864 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:50.579311 31806 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:50.580327 31739 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:50.580914 31739 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.581425 31663 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.581996 31663 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.581491 31739 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:50.582180 31745 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.580812 31806 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:50.540813 31749 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.584422 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:50.589102 31663 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.572432 31837 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:50.597463 31837 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:50.598505 31837 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.599238 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:50.589661 31893 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:50.589599 31745 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.572445 31845 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.603839 31845 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:50.606232 31893 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.606439 31604 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.607664 31729 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:50.607285 31604 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.590621 31798 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.595211 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:50.585161 31906 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:50.602085 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:50.576510 31902 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.615290 31604 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:50.616593 31749 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:50.616775 31745 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.617851 31858 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:04:50.625425 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.628417 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:50.636587 29070 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:50.638022 29066 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:50.641855 29072 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:04:50.643467 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.645071 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:50.645840 29145 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:50.652988 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:50.650426 31745 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.639861 31799 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:50.640544 31858 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.657691 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:50.661990 29145 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:50.610461 31841 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.654886 31645 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.657634 31866 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:50.645135 31803 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.620422 31798 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.653532 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:50.672705 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.674386 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.675345 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:50.675963 29070 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.676191 29188 leader_election.cc:400] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
I20260709 13:04:50.676535 29188 leader_election.cc:403] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:04:50.659359 31799 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.685395 29218 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:50.689388 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 273) took 1020 ms. Trace:
I20260709 13:04:50.689849 29068 rpcz_store.cc:276] 0709 13:04:49.668759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.669021 (+   262us) service_pool.cc:224] Handling call
0709 13:04:50.689177 (+1020156us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b54002ac454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:50.689322 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1531421952}
I20260709 13:04:50.700032 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:50.700685 29145 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:50.673074 31866 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.694499 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.709496 31798 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.690945 31799 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.712229 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:50.723716 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:50.724464 29067 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:50.656286 31745 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.673760 31645 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.630769 31902 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.723598 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.713425 29214 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
W20260709 13:04:50.713604 31820 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.001s
I20260709 13:04:50.713361 31444 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.735977 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.737015 31444 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.711759 31903 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.718698 31866 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:50.715087 31828 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
I20260709 13:04:50.739146 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.739049 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.741546 29214 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:50.743775 29073 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:04:50.744920 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68867.8us
I20260709 13:04:50.751227 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.752216 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1810) took 1065 ms. Trace:
I20260709 13:04:50.752890 29216 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:50.740357 31820 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:50.752768 29220 rpcz_store.cc:276] 0709 13:04:49.687103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.705476 (+ 18373us) service_pool.cc:224] Handling call
0709 13:04:49.727352 (+ 21876us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.901243 (+173891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.751977 (+850734us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54002f5dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:50.752158 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398571392}
I20260709 13:04:50.755326 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.755899 29215 raft_consensus.cc:2376] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8. Re-sending same reply.
I20260709 13:04:50.756867 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.752204 29217 raft_consensus.cc:2376] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7. Re-sending same reply.
I20260709 13:04:50.756109 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.730749 31796 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.763089 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.760569 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.767508 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.768121 29216 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.769295 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1703) took 1008 ms. Trace:
I20260709 13:04:50.776564 29070 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:50.777120 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.777788 29145 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:50.778010 29220 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:50.777920 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:50.779479 31739 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.781703 31739 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:50.784893 31444 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.786281 31645 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.786599 31809 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.787611 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 367) took 2348 ms. Trace:
I20260709 13:04:50.787983 29142 rpcz_store.cc:276] 0709 13:04:48.439510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.441609 (+  2099us) service_pool.cc:224] Handling call
0709 13:04:48.464414 (+ 22805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.210757 (+1746343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.787400 (+576643us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b5400190354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:50.787556 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3989520256}
I20260709 13:04:50.789201 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:50.792048 31841 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:50.763515 31837 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:04:50.793174 31796 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:50.761234 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:50.793939 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:50.769701 29140 rpcz_store.cc:276] 0709 13:04:49.760600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.434078 (+673478us) service_pool.cc:224] Handling call
0709 13:04:50.452959 (+ 18881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.453294 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.453336 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.465787 (+ 12451us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.546379 (+ 80592us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:50.673309 (+126930us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.673996 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.674076 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.766427 (+ 92351us) raft_consensus.cc:1685] finished
0709 13:04:50.766447 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.769224 (+  2777us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":350,"prepare.queue_time_us":163644,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":84,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":10754,"thread_start_us":144752,"threads_started":2,"wal-append.queue_time_us":156405}
I20260709 13:04:50.792951 28968 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 4 to 7, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 7 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:50.797951 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:50.798631 29070 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:50.757625 31578 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
W20260709 13:04:50.799475 28971 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 271) took 17996 ms (client timeout 15000 ms). Trace:
I20260709 13:04:50.800165 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.801267 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.802244 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.802947 31444 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.802467 28968 catalog_manager.cc:5697] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 3 to 6, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 6 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.803438 31444 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.804199 31856 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:50.804632 31856 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:50.804493 28968 catalog_manager.cc:5697] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 1 to 3, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 3 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.805588 31444 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.805491 31856 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.806571 31856 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:50.807418 29220 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.806831 28968 catalog_manager.cc:5697] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 2 to 3, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 3 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.798081 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:50.808748 29071 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:50.790791 31844 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.809410 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:50.812595 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:50.814940 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:50.816322 29214 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:50.816776 29216 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:50.817099 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.817633 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.791862 31830 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:50.761399 31828 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:04:50.797137 29903 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:50.821163 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.821403 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.822075 29216 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:04:50.829617 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:50.830376 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:50.827123 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.832073 29140 raft_consensus.cc:2368] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:50.820050 31830 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:50.842561 29188 leader_election.cc:400] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:50.842955 29188 leader_election.cc:403] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:50.834367 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:50.835597 31745 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.836076 31903 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:50.836189 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.848222 29221 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 13:04:50.799907 28971 rpcz_store.cc:269] 0709 13:04:32.803201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.847756 (+ 44555us) service_pool.cc:224] Handling call
0709 13:04:50.799176 (+17951420us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:38.831747 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:38.964869 (+133122us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:38.965767 (+   898us) write_op.cc:435] Acquired schema lock
0709 13:04:38.966284 (+   517us) tablet.cc:662] Decoding operations
0709 13:04:38.999665 (+ 33381us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:38.999736 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:38.999769 (+    33us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:04:39.000863 (+  1094us) tablet.cc:701] Row locks acquired
0709 13:04:39.000881 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:04:39.001042 (+   161us) write_op.cc:270] Start()
0709 13:04:39.001153 (+   111us) write_op.cc:276] Timestamp: P: 1783602279000991 usec, L: 0
0709 13:04:39.001171 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:04:39.001823 (+   652us) log.cc:844] Serialized 4398 byte log entry
0709 13:04:39.733370 (+731547us) op_driver.cc:464] REPLICATION: finished
0709 13:04:40.027988 (+294618us) write_op.cc:301] APPLY: starting
0709 13:04:40.035806 (+  7818us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:40.112230 (+ 76424us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:40.320834 (+208604us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:40.701654 (+380820us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:40.701915 (+   261us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:04:40.701978 (+    63us) write_op.cc:312] APPLY: finished
0709 13:04:40.702282 (+   304us) log.cc:844] Serialized 83 byte log entry
0709 13:04:41.882650 (+1180368us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f7833a030ac 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b
0709 13:04:41.892797 (+ 10147us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:41.892987 (+   190us) write_op.cc:454] Released schema lock
0709 13:04:41.893059 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:41.893495 (+   436us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:41.893533 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":23097,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":257824,"apply.run_cpu_time_us":22514,"apply.run_wall_time_us":1870565,"num_ops":11,"prepare.queue_time_us":801233,"prepare.run_cpu_time_us":24332,"prepare.run_wall_time_us":414295,"raft.queue_time_us":314130,"raft.run_cpu_time_us":45067,"raft.run_wall_time_us":165291,"replication_time_us":732038,"spinlock_wait_cycles":38421760,"thread_start_us":1933178,"threads_started":5,"wal-append.queue_time_us":1467656,"wal-append.run_cpu_time_us":44051,"wal-append.run_wall_time_us":3174876}]]}
I20260709 13:04:50.819844 31809 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:04:50.831233 31921 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:04:50.852624 29221 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:50.853936 29215 raft_consensus.cc:2410] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:50.853863 29221 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:50.821847 31444 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:04:50.857599 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:50.845204 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.860585 29214 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:50.845717 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:50.846097 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:50.864172 29140 raft_consensus.cc:2410] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:50.833460 31884 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:50.867856 29214 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:50.870325 29221 pending_rounds.cc:85] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 5
I20260709 13:04:50.871289 29221 pending_rounds.cc:107] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:04:50.871594 29221 raft_consensus.cc:2889] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:50.873021 29220 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:50.847931 31636 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.874289 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:50.878453 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:50.879612 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:50.874519 29067 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:04:50.882210 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:50.854873 31828 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5.
I20260709 13:04:50.861438 31578 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:50.863245 31809 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:50.892189 31869 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:50.894577 31663 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.893360 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:50.869760 31645 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:50.895475 29140 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.870920 31884 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:50.896692 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:50.870306 31849 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:04:50.880890 31902 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.885931 31895 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:50.885632 31745 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:50.870172 31749 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:04:50.894855 31869 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.907132 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:50.884644 31636 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.901398 31621 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:50.915998 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:50.904209 31895 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.896034 31578 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.918079 31444 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:50.926584 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 363) took 1222 ms. Trace:
I20260709 13:04:50.927316 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 378) took 1228 ms. Trace:
I20260709 13:04:50.909337 31869 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.936070 29144 rpcz_store.cc:276] 0709 13:04:49.698689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.133936 (+435247us) service_pool.cc:224] Handling call
0709 13:04:50.158208 (+ 24272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.530167 (+371959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.927089 (+396922us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b5400192154. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:50.927260 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837181312}
I20260709 13:04:50.928951 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:50.923144 31745 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:50.924947 31809 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.923382 31895 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:50.899677 31663 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:50.934231 29213 rpcz_store.cc:276] 0709 13:04:49.704005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.987838 (+283833us) service_pool.cc:224] Handling call
0709 13:04:49.994744 (+  6906us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.117733 (+122989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.117763 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.125027 (+  7264us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.329070 (+204043us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:50.330123 (+  1053us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.350740 (+ 20617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.926206 (+575466us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400355cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:50.926253 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.926293 (+    40us) raft_consensus.cc:1685] finished
0709 13:04:50.926380 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.926527 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18406,"prepare.queue_time_us":506,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":22817,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":7678,"spinlock_wait_cycles":281901440,"thread_start_us":171474,"threads_started":1,"wal-append.queue_time_us":187735}
I20260709 13:04:50.937284 29147 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:50.900897 31830 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.924566 31621 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:50.931365 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:50.942116 31869 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:50.939807 31745 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.933538 31857 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:04:50.929625 31618 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:04:50.955991 31532 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.958535 31893 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:50.938014 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 428) took 1369 ms. Trace:
I20260709 13:04:50.964597 29148 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:04:50.973286 29148 pending_rounds.cc:85] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 6
I20260709 13:04:50.973536 29148 pending_rounds.cc:107] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:04:50.973855 29148 raft_consensus.cc:2889] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:50.946123 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1966) took 1150 ms. Trace:
I20260709 13:04:50.974752 29219 rpcz_store.cc:276] 0709 13:04:49.796052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.935669 (+139617us) service_pool.cc:224] Handling call
0709 13:04:50.945929 (+1010260us) spinlock_profiling.cc:243] Waited 46 ms on lock 0x7b540033b3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:50.946044 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105888640}
I20260709 13:04:50.972543 29113 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:50.965204 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:50.968164 29140 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:50.971762 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:50.980710 29147 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:50.980857 29144 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:50.969061 31745 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:50.985755 31857 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:50.970119 31869 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:50.984546 31729 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:04:50.933900 31884 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:50.948812 31858 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:50.992529 29071 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:50.972033 31830 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:50.976078 31527 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.999421 31869 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.955760 31621 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:51.015026 31869 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:51.015352 31857 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:51.016183 31926 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:04:51.016549 31858 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:51.021656 31866 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:51.022161 29146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 373) took 1674 ms (client timeout 1993 ms). Trace:
W20260709 13:04:51.030602 29146 rpcz_store.cc:269] 0709 13:04:49.347969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.348864 (+   895us) service_pool.cc:224] Handling call
0709 13:04:51.021948 (+1673084us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b54001cbe54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:51.022103 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1646461696}
I20260709 13:04:51.023068 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1965) took 1369 ms. Trace:
I20260709 13:04:51.031945 29143 rpcz_store.cc:276] 0709 13:04:49.654013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.939440 (+285427us) service_pool.cc:224] Handling call
0709 13:04:49.955294 (+ 15854us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.184039 (+228745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.022775 (+838736us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54000e9cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:51.022996 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":524553472}
I20260709 13:04:51.013545 31856 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:51.029799 31884 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:51.052743 29216 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:51.023664 31527 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:51.045624 31856 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:50.973150 29905 rpcz_store.cc:276] 0709 13:04:49.567422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.573981 (+  6559us) service_pool.cc:224] Handling call
0709 13:04:49.574303 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.634788 (+ 60485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.636747 (+  1959us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:49.643752 (+  7005us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:49.645618 (+  1866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.932053 (+1286435us) spinlock_profiling.cc:243] Waited 42.2 ms on lock 0x7b5400768c54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:50.932853 (+   800us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.936710 (+  3857us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97169280}
I20260709 13:04:51.061806 31858 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:51.066210 31856 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:51.073935 31856 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:51.077841 31715 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:51.081590 31844 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:51.084252 31858 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:51.066855 31663 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:51.094751 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.089874 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1964) took 1436 ms. Trace:
I20260709 13:04:51.096128 29141 rpcz_store.cc:276] 0709 13:04:49.653854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.928210 (+274356us) service_pool.cc:224] Handling call
0709 13:04:51.023449 (+1095239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.023766 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:51.023796 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:51.023838 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:51.023865 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.023956 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.089619 (+ 65663us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001cbe50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:51.089787 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1516,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":2102,"spinlock_wait_cycles":2488886528}
I20260709 13:04:51.104204 31631 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:51.109280 31866 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:51.112129 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 458690us
I20260709 13:04:51.112438 31798 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:51.112886 31798 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.113255 31798 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:51.113677 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.114574 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.114451 31803 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:51.115216 31803 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:51.119449 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.067230 31857 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:51.121376 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 275773us
I20260709 13:04:51.122377 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:51.125460 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:51.129415 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:51.133018 29067 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:51.133942 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:51.130085 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:51.135159 29069 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:04:51.130510 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:51.136853 29066 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:51.131170 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:51.137959 31778 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.140277 31778 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:51.141053 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:51.138074 31905 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:51.142675 31905 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:51.143024 31778 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:51.144814 31532 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:51.127449 31893 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:51.120898 31796 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:51.146219 31796 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:51.148267 31796 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:51.149139 31913 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.687s	user 0.004s	sys 0.000s
I20260709 13:04:51.144838 31905 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:51.150573 31905 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:51.152495 31803 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:51.153461 31928 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:04:51.160360 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:51.161207 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:51.162324 29145 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:51.161343 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:51.167465 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:51.169695 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:51.170140 29903 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:51.171414 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:51.172686 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:04:51.173777 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.174415 29911 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:51.177006 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:51.181257 31749 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:51.182078 31749 consensus_queue.cc:1243] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:51.182540 31749 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:51.182809 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 376) took 1769 ms. Trace:
I20260709 13:04:51.182976 31849 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:51.183223 29148 rpcz_store.cc:276] 0709 13:04:49.413468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.413644 (+   176us) service_pool.cc:224] Handling call
0709 13:04:49.414022 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.974391 (+1560369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.182567 (+208176us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54002e8bd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:51.182747 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3565163776}
I20260709 13:04:51.185307 31842 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:51.185956 31842 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.186403 29906 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:51.186455 31842 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:51.188234 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:51.189365 31636 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:51.189528 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:51.178385 31906 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:51.192557 31906 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:51.197093 31825 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:51.213932 31825 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:51.182835 31749 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:51.215062 31825 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:51.216776 31820 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:51.217206 31820 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:51.215461 31749 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:51.218086 31820 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:51.218537 31749 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:51.219069 31820 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:51.217378 31858 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:51.220562 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 43660.1us
W20260709 13:04:51.221328 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.224449 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.227499 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:51.229274 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 425) took 1899 ms. Trace:
W20260709 13:04:51.234598 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.246873 29910 rpcz_store.cc:276] 0709 13:04:49.329979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.330140 (+   161us) service_pool.cc:224] Handling call
0709 13:04:49.330486 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.260917 (+930431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.228992 (+968075us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54005bd1d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:51.229212 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2080408832}
I20260709 13:04:51.215552 31780 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:51.253437 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:51.254251 29065 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:51.254266 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:51.255465 29221 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:51.259066 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:51.260797 29215 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:51.262969 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:51.267091 29215 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:04:51.259250 29113 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:04:51.279443 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52005.5us
I20260709 13:04:51.280579 29113 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:51.286248 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:51.292559 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:51.293390 29073 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:51.299088 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:51.201447 31806 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.207677 31905 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.204011 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:51.316901 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:04:51.304705 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 56900.3us
I20260709 13:04:51.301652 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:51.320518 29221 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:51.322346 29113 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:51.202128 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:51.310812 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 188508us
I20260709 13:04:51.325351 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:51.330291 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:51.300015 31444 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:51.332568 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:51.338771 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:51.339156 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 65572.2us
I20260709 13:04:51.307379 31715 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:51.252839 31801 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:04:51.303150 31753 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:51.313293 31825 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:51.343856 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 101238us
I20260709 13:04:51.345474 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:51.339707 31621 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:51.329753 29906 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:04:51.334841 31902 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:51.355787 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:51.337735 31444 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:51.358156 31799 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.636s	user 0.013s	sys 0.000s
I20260709 13:04:51.351873 31753 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.343748 31778 raft_consensus.cc:2764] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:04:51.343503 31639 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:51.315501 31905 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:51.392584 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:51.396224 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:51.315259 31806 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:51.335193 31901 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:51.348140 31663 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:51.388448 31621 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.409618 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:51.440002 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:51.403407 31753 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:51.348405 31893 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:51.346613 31842 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:51.358942 31781 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:04:51.400480 31645 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:51.399115 31444 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:51.450552 29142 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:51.452297 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:51.404763 31639 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.340055 31906 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:51.465433 31853 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.417740 31856 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:51.390266 31922 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.121s
I20260709 13:04:51.461275 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:51.548385 29141 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:51.462798 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:51.550562 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.466260 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:51.447003 31621 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:51.477293 31645 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.481168 29148 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:51.453774 31618 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:51.453922 31819 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:51.405467 31799 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:51.447223 31904 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:04:51.528729 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.456055 29072 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:51.421296 31715 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:51.453585 31663 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:51.413312 31937 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:04:51.558045 31645 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:51.550910 31853 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:51.471302 31934 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
W20260709 13:04:51.552500 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 160021us
I20260709 13:04:51.541924 31636 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.541730 31639 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:51.541496 31901 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:51.574704 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 229279us
I20260709 13:04:51.578186 31715 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:51.614729 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 265316us
I20260709 13:04:51.619748 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:51.620718 29073 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:51.591842 31904 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:51.624403 29038 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:51.610220 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 129060us
I20260709 13:04:51.631085 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:51.635236 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:51.638134 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:51.639648 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:51.640523 31901 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:51.640321 31836 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:51.643378 31803 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:51.643831 31803 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:51.644230 31803 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:51.640332 31904 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:51.648233 31953 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:51.648718 31953 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.649044 31953 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:51.640831 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:51.597852 31801 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:04:51.641724 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:51.655527 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:51.449314 31893 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:04:51.712546 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.713840 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 78709.3us
W20260709 13:04:51.797230 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.713656 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 377878us
I20260709 13:04:51.842398 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:51.836081 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:51.672537 29064 raft_consensus.cc:2410] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:51.843317 29148 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
W20260709 13:04:51.845759 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.858479 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.864796 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 223190us
I20260709 13:04:51.850171 31796 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.623479 31636 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:51.871929 29144 pending_rounds.cc:85] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 6
I20260709 13:04:51.872362 29144 pending_rounds.cc:107] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:04:51.664695 31801 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:51.872730 29144 raft_consensus.cc:2889] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:51.847062 31780 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:51.884125 31780 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:51.884646 31780 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:51.666024 31663 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:51.776888 31893 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:51.664427 31901 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:51.778481 31944 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.246s
I20260709 13:04:51.868559 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:51.859464 31889 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:51.865221 31961 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:51.913091 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44661.2us
I20260709 13:04:51.880009 31806 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:51.871616 31905 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:51.890846 31798 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.777s	user 0.011s	sys 0.000s
W20260709 13:04:51.898453 31631 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.794s	user 0.009s	sys 0.002s
I20260709 13:04:51.879284 31801 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.939338 31856 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:51.906394 31954 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.949622 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.904755 31884 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.845s	user 0.005s	sys 0.006s
I20260709 13:04:51.902375 31850 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:51.970391 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 314880us
I20260709 13:04:51.898790 31937 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:51.978955 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:51.935225 31936 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.436s	user 0.001s	sys 0.000s
I20260709 13:04:52.000452 29145 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:51.880883 31796 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.001914 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:52.002350 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:52.002811 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 221971us
I20260709 13:04:52.002943 29070 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:51.956204 31886 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.005879 29071 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:52.005964 29068 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:51.975566 31801 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:51.975245 31856 consensus_queue.cc:1243] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:04:51.993214 31905 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:51.992858 31922 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.978006 31464 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:52.048733 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.049436 29215 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:51.971896 31949 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:52.029677 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:52.053700 29215 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:52.032955 31901 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.034176 31853 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:51.993362 31954 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.004433 31631 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.004671 31884 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:52.036265 31825 consensus_meta.cc:287] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.681s	user 0.008s	sys 0.003s
I20260709 13:04:52.010571 31799 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.038385 31578 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.025243 31905 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.007596 31956 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.042901 31803 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.004860 31959 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.043789 31922 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.051327 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.109428 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:52.052435 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.056540 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:51.961066 31889 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:51.975188 31798 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.074834 31969 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:52.066493 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.067118 29146 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:52.068435 29221 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:04:52.047823 31749 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.829s	user 0.012s	sys 0.002s
I20260709 13:04:52.073309 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.073958 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.135452 29220 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:52.038836 31444 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.047412 31573 tablet_copy_client.cc:538] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:04:52.082257 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.007937 31778 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:52.100163 29073 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:04:52.024509 31951 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:52.093918 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:52.113443 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:51.999519 31850 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.066424 31967 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.107s
I20260709 13:04:52.078347 31806 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.055800 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:52.126240 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.129633 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.105052 31905 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:52.008893 31886 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.064425 31464 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.079663 31631 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.113469 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:52.123828 31845 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:52.087805 31884 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.207438 29143 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:52.128166 31964 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:52.146941 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:52.188710 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.218654 29214 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:52.187408 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:52.151506 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.149844 29068 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:04:52.217545 29038 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.554s (SENT)
I20260709 13:04:52.226693 29142 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:52.217803 29908 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:52.142397 31861 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:52.149937 31639 consensus_meta.cc:287] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.000s
I20260709 13:04:52.239063 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:52.239660 29213 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:52.241055 29073 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:52.228005 31806 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.245353 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:52.247085 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 133721us
I20260709 13:04:52.116252 31956 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.152117 29905 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:04:52.220343 31749 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.144481 31937 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 13:04:52.125219 31959 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.326260 29907 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:52.328956 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:52.330281 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:52.261265 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.229539 31825 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:52.149400 31799 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
W20260709 13:04:52.304360 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.235540 31856 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.331339 31937 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:52.250998 31607 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:52.332445 31954 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:52.332907 31975 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:04:52.333340 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 504) took 1133 ms. Trace:
I20260709 13:04:52.394691 29905 rpcz_store.cc:276] 0709 13:04:51.200403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.201736 (+  1333us) service_pool.cc:224] Handling call
0709 13:04:52.333129 (+1131393us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b5400767fd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:52.333260 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1875523456}
I20260709 13:04:52.333811 31444 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.334134 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.246986 31636 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:52.321077 31955 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.350s	user 0.001s	sys 0.000s
I20260709 13:04:52.334004 31984 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:52.392756 31937 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:52.387991 29907 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:52.388233 29067 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:04:52.336144 29142 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:52.390015 31799 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:52.400991 31464 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.400151 31799 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.391623 31850 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.257277 31963 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:52.333905 31778 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:52.247215 31639 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:52.338373 31895 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.386s	user 0.008s	sys 0.007s
I20260709 13:04:52.404907 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:52.140626 31969 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.412437 29142 raft_consensus.cc:2410] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:04:52.331373 31983 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.260526 31964 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.136s
W20260709 13:04:52.417019 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 164790us
I20260709 13:04:52.416188 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:52.426648 29142 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:52.427392 29140 raft_consensus.cc:2368] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:04:52.249545 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:52.428380 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 178894us
W20260709 13:04:52.429678 29874 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.011s	sys 0.019s
W20260709 13:04:52.431005 29874 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.011s	sys 0.019s
I20260709 13:04:52.408475 31967 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:52.321533 31749 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.432116 29907 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:52.419327 31745 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.415s	user 0.010s	sys 0.004s
I20260709 13:04:52.435097 29143 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:52.410868 31578 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.410101 31778 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:52.391676 29073 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:04:52.408252 31825 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.405212 31856 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.443780 31745 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.458994 31631 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:52.469189 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 63018.7us
I20260709 13:04:52.469707 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.450486 31976 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:52.476491 31889 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.476835 31976 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.477262 31976 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:52.419070 31895 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.479588 31895 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.481357 29142 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:52.484638 29148 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:52.482318 29139 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:04:52.482494 31893 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.000s
W20260709 13:04:52.489852 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 56420.3us
I20260709 13:04:52.433349 31954 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:04:52.442907 31464 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.034s
W20260709 13:04:52.480628 31960 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:04:52.481423 31974 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:04:52.460798 31457 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.787s	user 0.018s	sys 0.003s
I20260709 13:04:52.481256 31749 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.418418 31963 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:52.535211 31869 consensus_meta.cc:287] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.505s	user 0.005s	sys 0.010s
W20260709 13:04:52.514133 29216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1977) took 2167 ms (client timeout 2445 ms). Trace:
I20260709 13:04:52.449363 31578 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.428200 31983 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:52.484676 31820 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 1.265s	user 0.012s	sys 0.002s
I20260709 13:04:52.489591 31799 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.550846 31982 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.468624 31937 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.571059 31937 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:52.572811 31953 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.923s	user 0.009s	sys 0.003s
W20260709 13:04:52.547461 31457 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.873s	user 0.019s	sys 0.003s
I20260709 13:04:52.544600 31987 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:52.557070 31922 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:52.557528 31983 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:52.557698 31969 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.557914 31825 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.551311 31884 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.558490 31745 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.558604 31959 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.581106 31976 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:52.562800 31954 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:04:52.572957 31645 consensus_meta.cc:287] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.964s	user 0.007s	sys 0.003s
W20260709 13:04:52.754088 31780 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.869s	user 0.011s	sys 0.000s
I20260709 13:04:52.754020 31668 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:52.754434 31645 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.755076 31780 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.461200 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:52.756829 29073 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:52.678758 29072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 1835) took 1435 ms (client timeout 1521 ms). Trace:
W20260709 13:04:52.757964 29072 rpcz_store.cc:269] 0709 13:04:51.243064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.243205 (+   141us) service_pool.cc:224] Handling call
0709 13:04:52.678141 (+1434936us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400383e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:52.678468 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358884608}
W20260709 13:04:52.679141 29064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1744) took 2080 ms (client timeout 1618 ms). Trace:
I20260709 13:04:52.581056 31856 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.760233 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:52.760107 29064 rpcz_store.cc:269] 0709 13:04:50.598759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.598934 (+   175us) service_pool.cc:224] Handling call
0709 13:04:52.678462 (+2079528us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002080d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:52.678848 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2465228672}
I20260709 13:04:52.760838 31856 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:52.762069 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:52.761858 31856 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.762475 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:52.763120 29064 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 3.
W20260709 13:04:52.676338 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.704329 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1989) took 1253 ms. Trace:
I20260709 13:04:52.764340 31809 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.686420 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:52.764676 29070 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:52.705766 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1985) took 1256 ms. Trace:
I20260709 13:04:52.765714 29145 rpcz_store.cc:276] 0709 13:04:51.449117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.453265 (+  4148us) service_pool.cc:224] Handling call
0709 13:04:52.705353 (+1252088us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54003b4054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:52.705535 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1032645504}
I20260709 13:04:52.764609 29147 rpcz_store.cc:276] 0709 13:04:51.450393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.450625 (+   232us) service_pool.cc:224] Handling call
0709 13:04:51.450907 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.569901 (+118994us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:04:51.587672 (+ 17771us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.901940 (+314268us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:51.901997 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:51.902046 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.703495 (+801449us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400190d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:52.703637 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.704070 (+   433us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3007,"prepare.queue_time_us":692621,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":5638,"raft.queue_time_us":17281,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":7942,"spinlock_wait_cycles":273000832,"thread_start_us":313965,"threads_started":1}
I20260709 13:04:52.707346 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1986) took 1257 ms. Trace:
I20260709 13:04:52.555365 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.768095 29140 rpcz_store.cc:276] 0709 13:04:51.449433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.449563 (+   130us) service_pool.cc:224] Handling call
0709 13:04:52.706865 (+1257302us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b54001bd5d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:52.707160 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2018795904}
I20260709 13:04:52.768671 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:52.769094 29906 raft_consensus.cc:2410] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:04:52.769483 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:52.770398 29140 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:52.770125 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:52.770884 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.771366 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.772465 31809 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.772694 31856 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.772745 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.709429 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:52.710465 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1722) took 1259 ms. Trace:
I20260709 13:04:52.711585 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1981) took 1263 ms. Trace:
I20260709 13:04:52.514230 31893 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:52.492549 29217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 1980) took 1888 ms (client timeout 1554 ms). Trace:
I20260709 13:04:52.564196 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 378) took 1303 ms. Trace:
I20260709 13:04:52.797668 31901 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:52.798689 31901 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.800277 29143 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:52.801642 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:52.802740 29143 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:04:52.728626 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 181210us
I20260709 13:04:52.804065 31961 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.805424 31961 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.807370 31645 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:52.552141 29216 rpcz_store.cc:269] 0709 13:04:50.346864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.830143 (+483279us) service_pool.cc:224] Handling call
0709 13:04:52.513930 (+1683787us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54001333d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:52.514072 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507379584}
I20260709 13:04:52.809413 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.810856 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:52.741937 29215 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:52.744972 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.812552 29218 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:52.747812 29220 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:52.749207 29214 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:52.796674 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:52.750219 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:52.557130 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.559415 29910 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:04:52.828568 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.830446 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.751904 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.513765 31778 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.838860 31893 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.839094 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.575069 31578 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:52.586630 31621 consensus_meta.cc:287] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.019s	user 0.008s	sys 0.001s
I20260709 13:04:52.752935 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.755203 31668 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:52.562342 31753 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.103s	user 0.010s	sys 0.000s
I20260709 13:04:52.693709 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:52.704468 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:52.565683 31869 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.615868 31905 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.926314 31905 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:52.603538 31924 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.002s	user 0.002s	sys 0.011s
W20260709 13:04:52.927661 31924 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.351s	user 0.002s	sys 0.012s
I20260709 13:04:52.927043 31753 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:52.929162 29065 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:52.524466 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.931620 29141 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:52.934104 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:52.935177 29142 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
W20260709 13:04:52.617324 31948 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.839s	user 0.002s	sys 0.000s
W20260709 13:04:52.618186 31947 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.038s	user 0.002s	sys 0.000s
I20260709 13:04:52.944177 31806 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.946785 31984 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:52.947944 31984 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.948261 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:52.950129 31895 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:52.950919 31984 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.922109 31922 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.951467 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1832) took 1724 ms. Trace:
I20260709 13:04:52.952383 31969 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:52.622639 31952 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.998s	user 0.005s	sys 0.000s
I20260709 13:04:52.958565 31645 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:52.573381 31953 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.952826 31961 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.964267 31527 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.964629 31953 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.953102 31969 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:52.604782 31954 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:52.965454 31969 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:52.622430 31958 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.762s	user 0.002s	sys 0.000s
W20260709 13:04:52.969655 31977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.003s	sys 0.015s
W20260709 13:04:52.970316 31977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.003s	sys 0.015s
I20260709 13:04:52.971028 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:52.971264 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:52.606946 31963 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:52.620076 31973 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.287s	user 0.002s	sys 0.000s
W20260709 13:04:52.972958 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68709.6us
I20260709 13:04:52.974475 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:52.974591 31889 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:52.975724 31902 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:52.976404 31902 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:52.621932 31976 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:52.625051 31978 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.294s	user 0.002s	sys 0.001s
I20260709 13:04:52.569286 31982 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:52.574455 31987 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:52.607687 31988 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:04:53.037734 31527 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:52.767726 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:53.040427 29145 raft_consensus.cc:2368] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:04:53.042378 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:53.043491 29145 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:52.774302 29139 rpcz_store.cc:276] 0709 13:04:51.447793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.447953 (+   160us) service_pool.cc:224] Handling call
0709 13:04:51.448351 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:52.703048 (+1254697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.711110 (+  8062us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540029ed54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:52.711422 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2377349888}
I20260709 13:04:53.045059 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:53.045989 29139 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:53.045780 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:53.048058 31967 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:52.774214 29146 rpcz_store.cc:276] 0709 13:04:51.450740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.450853 (+   113us) service_pool.cc:224] Handling call
0709 13:04:51.451214 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.130830 (+679616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.709960 (+579130us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54003a52d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:52.710303 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415902464}
I20260709 13:04:53.048606 31967 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.048959 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 1990) took 1207 ms. Trace:
I20260709 13:04:53.049103 31967 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.049410 29144 rpcz_store.cc:276] 0709 13:04:51.841615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.843469 (+  1854us) service_pool.cc:224] Handling call
0709 13:04:51.871725 (+ 28256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:51.882924 (+ 11199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.882988 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:51.883539 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:04:52.010828 (+127289us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:52.180006 (+169178us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.182293 (+  2287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.182407 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.618828 (+436421us) spinlock_profiling.cc:243] Waited 53.9 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:53.048459 (+429631us) raft_consensus.cc:1685] finished
0709 13:04:53.048585 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.048784 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4822,"prepare.queue_time_us":320643,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":111,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":15387,"spinlock_wait_cycles":123984768,"thread_start_us":225965,"threads_started":2,"wal-append.queue_time_us":448756}
W20260709 13:04:52.796648 29217 rpcz_store.cc:269] 0709 13:04:50.603773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.735614 (+131841us) service_pool.cc:224] Handling call
0709 13:04:52.492326 (+1756712us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b54003561d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:52.492463 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35727616}
I20260709 13:04:52.814190 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:52.775715 29902 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:04:52.817209 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:52.817315 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.812824 31901 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.812857 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.825114 29221 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:52.825922 31983 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.826095 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.828680 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:52.565255 31639 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.831718 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:52.837179 29904 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
W20260709 13:04:53.076293 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 102095us
I20260709 13:04:53.077418 29907 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:52.837772 31951 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.547438 31464 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:53.080475 31951 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:53.080749 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:53.081229 31464 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:53.081753 29217 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:53.082525 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.798297 29219 rpcz_store.cc:276] 0709 13:04:51.260903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.261296 (+   393us) service_pool.cc:224] Handling call
0709 13:04:51.261811 (+   515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:51.300939 (+ 39128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.300984 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:51.308078 (+  7094us) log.cc:844] Serialized 26 byte log entry
0709 13:04:51.606486 (+298408us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:51.606547 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.606649 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.563984 (+957335us) raft_consensus.cc:1685] finished
0709 13:04:52.564011 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.564134 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53322,"raft.queue_time_us":45639,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":5874,"thread_start_us":261988,"threads_started":1,"wal-append.queue_time_us":921063}
W20260709 13:04:52.847736 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50947.1us
I20260709 13:04:53.083063 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.847843 29072 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
W20260709 13:04:52.872573 31906 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.004s	sys 0.010s
I20260709 13:04:53.084084 29214 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:53.084529 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.084844 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 519) took 1913 ms. Trace:
I20260709 13:04:53.085536 29072 raft_consensus.cc:2410] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:53.085879 29145 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:04:53.086549 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:53.087338 29072 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:04:53.083802 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:52.893520 31780 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.909682 29147 raft_consensus.cc:2410] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:04:52.921926 29067 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:52.922041 31621 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:52.917501 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:52.920480 31904 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.238s	user 0.008s	sys 0.006s
I20260709 13:04:52.569494 31956 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:52.909019 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:53.005582 29073 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:52.984429 29908 raft_consensus.cc:2410] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:52.980461 29071 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 6. Ops: []
I20260709 13:04:52.966986 31954 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:52.973157 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1836) took 1725 ms. Trace:
I20260709 13:04:53.014835 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:52.922035 31995 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.953341 31809 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:52.982762 31866 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:52.979848 31796 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:53.025738 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 117117us
I20260709 13:04:52.978294 31902 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:53.122573 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 247458us
I20260709 13:04:53.123195 31902 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:53.026966 31618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.303s	user 0.006s	sys 0.023s
W20260709 13:04:53.030551 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 213097us
I20260709 13:04:52.952109 29066 rpcz_store.cc:276] 0709 13:04:51.226506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.227000 (+   494us) service_pool.cc:224] Handling call
0709 13:04:51.227360 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:51.227775 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.227805 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:51.250375 (+ 22570us) log.cc:844] Serialized 26 byte log entry
0709 13:04:51.389348 (+138973us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:51.389387 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.389438 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.617603 (+1228165us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:52.950873 (+333270us) raft_consensus.cc:1685] finished
0709 13:04:52.950910 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.951229 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":771,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":27727,"spinlock_wait_cycles":555722624,"thread_start_us":89638,"threads_started":1,"wal-append.queue_time_us":295231}
I20260709 13:04:52.796869 31966 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.036926 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:53.047556 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:52.952275 31922 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.060456 29216 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:53.066879 29218 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:53.068845 31982 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.070128 29039 leader_election.cc:400] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:04:53.073614 29905 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:53.088871 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.091248 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 279) took 1015 ms. Trace:
I20260709 13:04:53.136112 29039 leader_election.cc:403] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:53.088227 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1846) took 1833 ms. Trace:
I20260709 13:04:53.088318 29220 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:53.036058 31946 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.280s	user 0.016s	sys 0.034s
I20260709 13:04:53.089834 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:53.087853 29145 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:53.094498 31904 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.445s	user 0.008s	sys 0.006s
I20260709 13:04:53.117110 29069 rpcz_store.cc:276] 0709 13:04:51.247292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.247433 (+   141us) service_pool.cc:224] Handling call
0709 13:04:51.253231 (+  5798us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:51.253594 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:51.253620 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:51.278287 (+ 24667us) log.cc:844] Serialized 26 byte log entry
0709 13:04:51.555384 (+277097us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:51.555423 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.555495 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.621688 (+1066193us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:52.972461 (+350773us) raft_consensus.cc:1685] finished
0709 13:04:52.972540 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.972940 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16536,"raft.queue_time_us":25870,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":18844,"spinlock_wait_cycles":33455744,"thread_start_us":219708,"threads_started":1,"wal-append.queue_time_us":295717}
I20260709 13:04:53.122210 31796 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:53.118036 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 285) took 1031 ms. Trace:
I20260709 13:04:53.126389 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:53.084102 31906 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.005s	sys 0.010s
W20260709 13:04:53.126029 31618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.534s	user 0.006s	sys 0.025s
I20260709 13:04:53.093493 31621 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.089223 31901 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.136404 29067 rpcz_store.cc:276] 0709 13:04:52.076065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.693154 (+617089us) service_pool.cc:224] Handling call
0709 13:04:53.091164 (+398010us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.137583 29144 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:53.137897 29142 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:53.144407 29073 rpcz_store.cc:276] 0709 13:04:52.099463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.454427 (+354964us) service_pool.cc:224] Handling call
0709 13:04:53.106504 (+652077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.085290 29902 rpcz_store.cc:276] 0709 13:04:51.171774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.172011 (+   237us) service_pool.cc:224] Handling call
0709 13:04:52.751399 (+1579388us) spinlock_profiling.cc:243] Waited 6.91 ms on lock 0x7b5c003bfe0c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783a477161 00007f783a618a79 00007f783bad29b6 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:53.084728 (+333329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15902080}
W20260709 13:04:53.141510 31845 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.000s	sys 0.013s
I20260709 13:04:53.153823 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:53.149762 29219 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
W20260709 13:04:53.147840 31946 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.522s	user 0.019s	sys 0.034s
I20260709 13:04:53.159533 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.162391 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:53.162919 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.163277 29073 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:53.159906 31959 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.164049 31983 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.164894 31959 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:53.159799 29215 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:53.167018 31983 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:53.143450 31796 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:53.171375 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:53.149515 31954 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:53.156576 31845 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.000s	sys 0.013s
W20260709 13:04:53.181579 31836 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.494s	user 0.001s	sys 0.015s
I20260709 13:04:53.181945 31924 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:53.171228 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:53.185098 31922 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:53.188304 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.184705 31901 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:53.187043 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.137403 29222 rpcz_store.cc:276] 0709 13:04:51.254607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.292362 (+ 37755us) service_pool.cc:224] Handling call
0709 13:04:51.311696 (+ 19334us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:51.335631 (+ 23935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.335664 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:51.336260 (+   596us) log.cc:844] Serialized 52 byte log entry
0709 13:04:51.476214 (+139954us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:51.543800 (+ 67586us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.556228 (+ 12428us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.556524 (+   296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.556614 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.616732 (+1060118us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:52.775607 (+158875us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b1800388900. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:04:53.087751 (+312144us) raft_consensus.cc:1685] finished
0709 13:04:53.087789 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.088048 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91369,"prepare.queue_time_us":86238,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":7217,"raft.queue_time_us":37605,"raft.run_cpu_time_us":619,"raft.run_wall_time_us":9660,"spinlock_wait_cycles":284314240,"thread_start_us":87828,"threads_started":1,"wal-append.queue_time_us":250883}
I20260709 13:04:53.190085 29148 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
W20260709 13:04:53.187776 31836 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.544s	user 0.001s	sys 0.016s
I20260709 13:04:53.194198 31798 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.195187 31944 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:53.195616 31944 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:53.196786 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1759) took 1159 ms. Trace:
I20260709 13:04:53.198182 31849 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:53.141347 31966 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:53.200939 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:53.201740 31983 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:53.202617 29066 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:04:53.141602 31995 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.190927 31464 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:53.203855 29069 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:04:53.208122 31987 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:53.208774 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:53.215831 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.216506 29145 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:53.216923 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:53.218904 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:53.260586 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 59791.8us
I20260709 13:04:53.220759 29140 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
W20260709 13:04:53.224756 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 154922us
I20260709 13:04:53.225521 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:53.172214 31527 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:53.228413 31886 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:53.174420 31796 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:53.201792 31801 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.172s	user 0.008s	sys 0.013s
I20260709 13:04:53.313385 29220 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:04:53.234375 31901 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:53.235817 31668 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.236454 29073 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:04:53.186448 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:53.315824 32012 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:53.238183 29215 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:53.315568 31969 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.324620 29139 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:53.325255 31969 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.328372 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1757) took 1308 ms. Trace:
I20260709 13:04:53.328832 29071 rpcz_store.cc:276] 0709 13:04:52.019973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.027782 (+  7809us) service_pool.cc:224] Handling call
0709 13:04:52.028174 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.135447 (+1107273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.328068 (+192621us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b54002aba54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:53.328302 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2188846976}
I20260709 13:04:53.330339 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.332226 29218 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:04:53.332739 29218 pending_rounds.cc:85] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 4
I20260709 13:04:53.333039 29218 pending_rounds.cc:107] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:04:53.333370 29218 raft_consensus.cc:2889] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:53.334766 31889 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:53.335300 29071 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:53.335604 29217 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:53.336751 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:53.337105 29141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 1992) took 1494 ms (client timeout 1986 ms). Trace:
I20260709 13:04:53.337380 29071 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:53.338289 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:53.339211 29071 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:53.343827 31532 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:53.345467 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:53.339779 31796 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:53.350837 31778 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:53.239310 32004 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.293s	user 0.000s	sys 0.000s
I20260709 13:04:53.242075 31967 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.357770 31803 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:53.359205 31967 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.246263 29219 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:53.361055 29064 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:53.247489 31573 tablet_bootstrap.cc:492] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:04:53.251138 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:53.252956 29065 rpcz_store.cc:276] 0709 13:04:52.037590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.037728 (+   138us) service_pool.cc:224] Handling call
0709 13:04:52.038200 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.929633 (+891433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.196441 (+266808us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b5400129b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:53.196725 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2047915776}
I20260709 13:04:53.258092 29216 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:53.263906 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 277) took 1182 ms. Trace:
I20260709 13:04:53.366762 29066 rpcz_store.cc:276] 0709 13:04:52.075468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.187018 (+1111550us) service_pool.cc:224] Handling call
0709 13:04:53.202267 (+ 15249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.257474 (+ 55207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.257711 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.270200 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:53.270280 31944 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:53.270599 31866 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:53.271510 31527 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.272126 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.273528 31959 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:53.377431 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:53.377869 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:53.274401 31966 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.274886 29909 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:53.275933 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.276731 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:53.277676 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.254128 31621 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.289707 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:53.293766 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 124767us
I20260709 13:04:53.298471 31995 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:53.299556 31924 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:53.300109 29221 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:04:53.266433 31983 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:53.306178 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:53.313253 31801 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.153798 31618 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:53.314579 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52397.1us
I20260709 13:04:53.266301 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:53.316033 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.237170 29072 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
W20260709 13:04:53.316066 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 162835us
I20260709 13:04:53.266992 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1732) took 1010 ms. Trace:
I20260709 13:04:53.362064 31796 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:53.337754 29141 rpcz_store.cc:269] 0709 13:04:51.842280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.844650 (+  2370us) service_pool.cc:224] Handling call
0709 13:04:53.336757 (+1492107us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b54000ea6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:53.337025 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1878231296}
I20260709 13:04:53.367625 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.382550 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:53.388636 29214 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:53.392241 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:53.375000 31866 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:53.394811 29073 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:53.394323 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:53.393560 29070 raft_consensus.cc:2410] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:53.402166 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.415181 29145 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:53.401404 29069 raft_consensus.cc:2410] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:53.414410 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:53.393332 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.395200 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:53.417371 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 155768us
I20260709 13:04:53.415784 29067 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:53.418270 29217 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:53.396896 31953 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:04:53.433252 29072 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:53.417757 31866 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:53.435683 29065 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:53.395593 29146 rpcz_store.cc:276] 0709 13:04:52.250010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.049156 (+799146us) service_pool.cc:224] Handling call
0709 13:04:53.049545 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.049985 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:53.050289 (+   304us) log.cc:844] Serialized 14 byte log entry
0709 13:04:53.259482 (+209193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:53.259536 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:53.259995 (+   459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.260065 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.260218 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6355,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":49,"raft.queue_time_us":4474,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":49153,"thread_start_us":204258,"threads_started":1,"wal-append.queue_time_us":324441}
I20260709 13:04:53.433501 29218 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:53.436697 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:53.439397 29069 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:04:53.439495 29066 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:53.440836 31944 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:53.442447 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 278) took 1362 ms. Trace:
I20260709 13:04:53.445797 29069 pending_rounds.cc:85] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 3
I20260709 13:04:53.446352 29069 pending_rounds.cc:107] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:04:53.445844 29072 rpcz_store.cc:276] 0709 13:04:52.075731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.417528 (+1341797us) service_pool.cc:224] Handling call
0709 13:04:53.430303 (+ 12775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.437505 (+  7202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.437719 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.446995 29069 raft_consensus.cc:2889] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:53.387717 31621 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.424824 31959 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:53.453478 31959 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.452800 31618 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.453974 31959 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:53.423048 31969 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:53.450685 31885 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:53.451447 32015 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:04:53.452975 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.490279 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.471287 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.457103 31889 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:53.456782 29221 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.438704 31796 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.494730 29142 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:53.497614 31527 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.498003 31967 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.499069 31967 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.501791 31967 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.496142 29219 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:53.490964 31885 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:53.506346 31885 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:53.508900 31801 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.451901 31578 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:04:53.513520 31983 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.516007 31842 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.517007 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:53.519222 31796 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:53.520658 29071 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:53.522451 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:04:53.523473 32000 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.561s	user 0.001s	sys 0.002s
I20260709 13:04:53.526335 29067 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:53.527383 29064 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:04:53.528928 29067 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: [3.3-7.5]
I20260709 13:04:53.530691 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:53.531489 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:53.532106 29139 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.532300 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:53.532821 31998 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.571s	user 0.000s	sys 0.003s
I20260709 13:04:53.532137 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:53.534055 29141 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:53.534696 29067 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:53.535171 31753 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.535796 29066 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:53.537664 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:53.491439 29217 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:53.538244 29145 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:53.537407 31869 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:53.548087 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 120983us
I20260709 13:04:53.550879 29221 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:53.549780 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:53.552105 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 246089us
W20260709 13:04:53.554323 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 67265.8us
I20260709 13:04:53.556234 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:53.554032 29073 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:04:53.562347 29072 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:53.567500 29148 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:53.569082 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 284) took 1481 ms. Trace:
I20260709 13:04:53.569336 29073 rpcz_store.cc:276] 0709 13:04:52.077638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.430337 (+1352699us) service_pool.cc:224] Handling call
0709 13:04:53.487149 (+ 56812us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.558866 (+ 71717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.559142 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:53.570472 29141 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.577407 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:53.578341 29219 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:53.568799 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:53.581458 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:53.558689 31753 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:53.592710 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:53.586104 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:53.591421 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:53.595036 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:53.582019 29112 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:53.609020 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:53.612391 29039 leader_election.cc:400] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:04:53.612924 29039 leader_election.cc:403] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:04:53.622419 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:53.607098 31842 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:53.590329 31753 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:04:53.631426 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.635665 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:53.636647 29221 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 4: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:04:53.643378 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.646903 31951 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.616741 31621 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.652091 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:53.653426 31858 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.029s	sys 0.025s
I20260709 13:04:53.656091 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 435) took 1482 ms. Trace:
W20260709 13:04:53.655391 31858 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.029s	sys 0.026s
I20260709 13:04:53.657866 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 433) took 1564 ms. Trace:
I20260709 13:04:53.658282 29910 rpcz_store.cc:276] 0709 13:04:52.093506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.093650 (+   144us) service_pool.cc:224] Handling call
0709 13:04:53.657451 (+1563801us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b54007e0254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:53.657665 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":953121152}
I20260709 13:04:53.657045 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:53.662302 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:53.654866 29217 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:53.664743 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:53.665845 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:53.669225 31956 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:53.669708 31956 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.670207 31464 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:53.670617 31956 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.670953 31464 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.671279 31884 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:53.671752 31884 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:53.672366 31999 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:53.672945 31884 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:53.671525 31464 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:53.674404 31884 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:53.672899 31999 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:53.676673 31999 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:53.678292 31999 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:53.679427 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 78100.4us
I20260709 13:04:53.669687 29221 raft_consensus.cc:2410] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:53.680737 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:04:53.682407 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 64131.4us
I20260709 13:04:53.683239 31922 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:53.684689 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:53.685484 32009 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:53.685956 32009 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:53.687153 29186 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:53.687872 31729 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:53.686966 32009 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:53.689987 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 131402us
I20260709 13:04:53.649456 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:53.635273 31753 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.634532 31966 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:53.635234 31801 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:04:53.755956 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:53.713858 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 74921.2us
I20260709 13:04:53.749567 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2002) took 1046 ms. Trace:
I20260709 13:04:53.757906 29147 rpcz_store.cc:276] 0709 13:04:52.687775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.769795 (+ 82020us) service_pool.cc:224] Handling call
0709 13:04:53.708591 (+938796us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54001cc354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:53.734402 (+ 25811us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317395328}
I20260709 13:04:53.760637 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:53.640662 31796 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:53.656567 29911 rpcz_store.cc:276] 0709 13:04:52.150645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.150776 (+   131us) service_pool.cc:224] Handling call
0709 13:04:52.151145 (+   369us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:52.223354 (+ 72209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:52.223395 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:52.235302 (+ 11907us) log.cc:844] Serialized 52 byte log entry
0709 13:04:52.423718 (+188416us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:52.423778 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.607334 (+1183556us) spinlock_profiling.cc:243] Waited 69.9 ms on lock 0x7b54007a0154. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:53.615802 (+  8468us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.615866 (+    64us) raft_consensus.cc:1685] finished
0709 13:04:53.615887 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.633241 (+ 17354us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102776,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":141,"spinlock_wait_cycles":160761216,"thread_start_us":111536,"threads_started":1,"wal-append.queue_time_us":258893}
I20260709 13:04:53.763064 29219 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:53.770072 29215 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:53.690587 31842 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:53.773782 29113 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:53.748178 31956 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:53.781901 32019 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
W20260709 13:04:53.787714 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 65357.9us
I20260709 13:04:53.788341 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:53.789155 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 102187us
I20260709 13:04:53.748495 32009 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:53.740365 31886 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:53.799206 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.799368 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 234346us
I20260709 13:04:53.796187 31607 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:53.802225 31621 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:53.802675 31621 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.803043 31621 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:53.801764 31885 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:53.812099 31966 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:53.812803 31966 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.813103 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:53.813298 31966 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:53.811856 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 63043.8us
I20260709 13:04:53.819977 31987 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:53.820448 31987 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.820876 31987 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:53.825915 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:53.826593 29218 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:53.827060 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 49224us
I20260709 13:04:53.828817 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1844) took 1158 ms. Trace:
I20260709 13:04:53.829312 31801 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:53.830885 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:53.831149 31967 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:53.833177 29070 pending_rounds.cc:85] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 1
I20260709 13:04:53.833613 29070 pending_rounds.cc:107] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:04:53.833987 29070 raft_consensus.cc:2889] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:53.829392 29071 rpcz_store.cc:276] 0709 13:04:52.670394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.427671 (+757277us) service_pool.cc:224] Handling call
0709 13:04:53.442975 (+ 15304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.521035 (+ 78060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.828345 (+307310us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b5400186354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:53.828702 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177782016}
I20260709 13:04:53.834986 29214 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:53.831637 31967 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:53.839581 29217 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:04:53.841223 29871 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:53.841195 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:53.848135 29141 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:53.846710 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:53.852833 29213 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.852970 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:53.853667 29071 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.878496 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:04:53.879043 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 81282.8us
I20260709 13:04:53.842394 31951 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:53.884183 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 92507.8us
W20260709 13:04:53.888865 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 335819us
I20260709 13:04:53.886020 29910 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:04:53.890983 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.906234 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.850239 31869 raft_consensus.cc:2764] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:04:53.913125 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:53.916400 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:53.926208 29072 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:53.887050 31902 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.888028 31527 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:53.931804 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:53.932174 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.934341 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.939169 29069 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:04:53.941850 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:53.941953 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:53.942796 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:53.948221 29068 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:04:53.949065 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 68696us
I20260709 13:04:53.948307 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:53.949839 29113 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:53.950476 29073 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:53.957929 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:53.940455 31527 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.877189 31729 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:04:53.932977 31778 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:53.933184 31796 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:53.975947 31999 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:04:53.962826 31842 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.027823 31999 consensus_queue.cc:1243] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:04:53.976094 31748 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:54.018378 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:54.028851 31999 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:54.030174 31999 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:04:54.003420 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 53657us
I20260709 13:04:53.976035 31830 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:54.007315 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:54.056881 29139 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 13:04:54.060626 29139 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:54.062847 29213 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:54.068399 29072 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:53.939570 31849 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:53.929361 31866 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:54.067090 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 102489us
I20260709 13:04:53.992185 31844 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:54.025530 31969 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:54.003786 31869 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:54.109143 29909 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:04:54.120687 29909 pending_rounds.cc:85] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 7
I20260709 13:04:54.122874 29909 pending_rounds.cc:107] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:04:54.123235 29909 raft_consensus.cc:2889] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:53.998872 31801 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:53.996325 31618 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:54.052915 31527 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:54.013468 31956 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:54.026624 31464 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.064880 31999 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.144011 31966 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.148161 29911 raft_consensus.cc:2368] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
I20260709 13:04:54.137977 31753 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.149562 31464 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:54.109869 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 197050us
I20260709 13:04:53.976131 29902 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:54.086046 29071 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:54.130616 29222 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:53.940872 31902 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:54.137126 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.082679 31849 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:54.146107 31999 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.087795 31748 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.146994 31715 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.089912 31844 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:53.954331 31951 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.078831 31995 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:54.151121 31753 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:54.151379 31885 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:54.180692 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:54.166563 31849 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:53.963202 31967 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.117786 31621 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.092386 31830 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.134865 31796 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.170787 31858 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:54.193660 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:54.139183 31801 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.171674 31844 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:54.204443 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 50903.9us
I20260709 13:04:54.137862 31778 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.245916 31778 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:54.246577 31778 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:54.208326 31796 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:54.214401 31866 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:54.216066 31976 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:04:54.218335 31801 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:54.219462 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 67091.6us
I20260709 13:04:54.225165 31621 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:54.225320 31830 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:54.227659 32023 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:04:54.228099 31902 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:54.229641 31748 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:54.180444 31715 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.230911 31858 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:04:54.231547 31924 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.831s	user 0.012s	sys 0.001s
I20260709 13:04:54.205220 31983 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:54.200460 31753 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.032s
W20260709 13:04:54.234253 31959 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.780s	user 0.012s	sys 0.000s
W20260709 13:04:54.236395 31884 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.561s	user 0.003s	sys 0.009s
I20260709 13:04:54.236872 31607 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:54.206983 31944 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.745s	user 0.013s	sys 0.001s
I20260709 13:04:54.252700 31951 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:54.256768 32026 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:04:54.289173 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.259328 31778 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:54.312567 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.337983 31886 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.514s	user 0.012s	sys 0.000s
W20260709 13:04:54.327304 31987 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.002s
I20260709 13:04:54.348045 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:04:54.374043 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.346390 31607 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.341368 31924 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:54.345213 31753 consensus_queue.cc:1243] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:04:54.408190 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:54.468655 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:54.446242 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:54.470227 29068 raft_consensus.cc:2410] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:04:54.472440 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 105680us
I20260709 13:04:54.415552 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:54.474242 29215 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:54.474305 29071 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 4.
I20260709 13:04:54.489362 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:54.346082 31983 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.510190 31983 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:54.373169 31886 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:54.354992 31844 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.513963 31844 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:54.343257 31995 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:04:54.517110 31995 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:54.517498 31995 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:54.518424 31995 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.520171 31621 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:54.520509 31527 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.521098 31621 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:54.520784 28967 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.522428 31621 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:54.522897 31527 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:54.330088 31866 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.138s
I20260709 13:04:54.519683 31995 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.337575 31858 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.371248 31987 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.345602 31959 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.523947 31944 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:54.530766 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 461865us
I20260709 13:04:54.522926 28967 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 2 to 4, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 4 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.527096 31621 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:54.539654 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 602425us
I20260709 13:04:54.546908 29069 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:04:54.540408 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51185.8us
W20260709 13:04:54.537838 29038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.003s	sys 0.011s
W20260709 13:04:54.547528 29038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.003s	sys 0.011s
W20260709 13:04:54.571677 28968 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 238) took 19154 ms (client timeout 15000 ms). Trace:
W20260709 13:04:54.551111 32028 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:04:54.552759 31801 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:54.554440 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1769) took 1355 ms. Trace:
W20260709 13:04:54.575742 32009 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.735s	user 0.008s	sys 0.000s
I20260709 13:04:54.538546 31944 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.565184 31959 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:54.570150 31621 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.548858 31715 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:54.574690 31967 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:54.575182 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 581995us
I20260709 13:04:54.558305 31987 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:54.546487 31951 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:54.576198 28967 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.590328 29871 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:54.592437 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 47960.9us
I20260709 13:04:54.602770 29220 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:54.586638 29065 rpcz_store.cc:276] 0709 13:04:53.198987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.488987 (+290000us) service_pool.cc:224] Handling call
0709 13:04:53.561958 (+ 72971us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.586982 (+ 25024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.587030 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:53.633615 (+ 46585us) log.cc:844] Serialized 26 byte log entry
0709 13:04:54.160191 (+526576us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:54.160261 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.160366 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.553122 (+392756us) raft_consensus.cc:1685] finished
0709 13:04:54.553143 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.554163 (+  1020us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b5400187c50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:54.554371 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10360,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":40082,"spinlock_wait_cycles":32547584,"thread_start_us":416151,"threads_started":1,"wal-append.queue_time_us":713502}
W20260709 13:04:54.604910 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.016s	sys 0.027s
W20260709 13:04:54.592010 31729 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.082s	sys 0.113s
W20260709 13:04:54.610479 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.016s	sys 0.027s
I20260709 13:04:54.586496 31801 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:54.612515 28967 catalog_manager.cc:5697] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 2 to 5, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 5 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.611361 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1731) took 2361 ms. Trace:
I20260709 13:04:54.589318 31924 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.614804 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:54.600456 31944 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.616020 28967 catalog_manager.cc:5697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:54.611604 29111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.012s	sys 0.031s
W20260709 13:04:54.620934 29111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.012s	sys 0.031s
I20260709 13:04:54.608561 31951 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:54.610864 31729 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.083s	sys 0.114s
I20260709 13:04:54.600630 31969 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:54.625927 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:54.630708 29141 raft_consensus.cc:2368] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
W20260709 13:04:54.611155 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 578893us
I20260709 13:04:54.627959 29139 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:54.605386 31869 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:54.630518 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:54.615429 29143 rpcz_store.cc:276] 0709 13:04:52.249782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.803529 (+553747us) service_pool.cc:224] Handling call
0709 13:04:53.732626 (+929097us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:53.732886 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:53.745388 (+ 12502us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.544455 (+799067us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.544514 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:54.544562 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.544628 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.611192 (+ 66564us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b54001cc350. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:54.611304 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27750,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":383404,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"spinlock_wait_cycles":2054437120,"thread_start_us":798497,"threads_started":2,"wal-append.queue_time_us":459473}
I20260709 13:04:54.600109 31830 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:04:54.634151 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:54.635835 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:54.638489 29146 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:54.619987 31959 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.598448 31607 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.635856 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:54.576818 28968 rpcz_store.cc:269] 0709 13:04:35.417218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.448102 (+ 30884us) service_pool.cc:224] Handling call
0709 13:04:54.571403 (+19123301us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:51.066708 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:51.190714 (+124006us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:51.190764 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:04:51.190789 (+    25us) tablet.cc:662] Decoding operations
0709 13:04:51.191573 (+   784us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:51.191667 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:51.191694 (+    27us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:04:51.192358 (+   664us) tablet.cc:701] Row locks acquired
0709 13:04:51.192410 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:04:51.192703 (+   293us) write_op.cc:270] Start()
0709 13:04:51.192857 (+   154us) write_op.cc:276] Timestamp: P: 1783602291192626 usec, L: 0
0709 13:04:51.192874 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:04:51.216339 (+ 23465us) log.cc:844] Serialized 2084 byte log entry
0709 13:04:51.551763 (+335424us) op_driver.cc:464] REPLICATION: finished
0709 13:04:52.624856 (+1073093us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f7833a030ac 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f783a543ef0
0709 13:04:52.959257 (+334401us) write_op.cc:301] APPLY: starting
0709 13:04:52.959474 (+   217us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:52.959863 (+   389us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:52.959943 (+    80us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:52.961214 (+  1271us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:53.005439 (+ 44225us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:04:53.016634 (+ 11195us) write_op.cc:312] APPLY: finished
0709 13:04:53.070738 (+ 54104us) log.cc:844] Serialized 47 byte log entry
0709 13:04:53.320147 (+249409us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:53.348043 (+ 27896us) write_op.cc:454] Released schema lock
0709 13:04:53.348119 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:53.355898 (+  7779us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:53.355965 (+    67us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1509055,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":1392064,"apply.run_cpu_time_us":47790,"apply.run_wall_time_us":396731,"num_ops":5,"prepare.queue_time_us":95594,"prepare.run_cpu_time_us":6340,"prepare.run_wall_time_us":208281,"raft.queue_time_us":161393,"raft.run_cpu_time_us":57245,"raft.run_wall_time_us":1717663,"replication_time_us":275019,"spinlock_wait_cycles":284819200,"thread_start_us":1841886,"threads_started":5,"wal-append.queue_time_us":309598,"wal-append.run_cpu_time_us":40720,"wal-append.run_wall_time_us":2124251}]]}
I20260709 13:04:54.643275 31796 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.629709 31951 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:54.607543 31849 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:54.648403 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 63409.9us
I20260709 13:04:54.653587 28967 catalog_manager.cc:5697] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:54.640424 31842 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.011s	sys 0.012s
I20260709 13:04:54.634837 31953 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.636265 31902 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:54.663084 31902 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.663501 31902 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:54.649160 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 399550us
I20260709 13:04:54.640758 31976 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:54.646762 31527 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:54.646467 31729 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:04:54.661777 32035 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:54.662148 31953 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.622958 31924 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.644625 31778 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:54.665328 31621 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:54.673903 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.655782 31842 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.012s	sys 0.012s
I20260709 13:04:54.671748 29113 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:54.692981 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:54.693733 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:54.649782 31796 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.704085 28967 catalog_manager.cc:5697] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 3 to 4, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 4 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:54.702920 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55213.2us
I20260709 13:04:54.704077 31987 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:54.655947 31959 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.706817 28967 catalog_manager.cc:5697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:54.713044 29113 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:54.718763 31527 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:54.723742 28967 catalog_manager.cc:5697] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 4, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 4 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:54.725311 28967 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 3 to 4, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 4 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:54.726655 32031 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.212s	user 0.002s	sys 0.000s
I20260709 13:04:54.727880 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2010) took 1299 ms. Trace:
I20260709 13:04:54.728277 29216 rpcz_store.cc:276] 0709 13:04:53.428594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.428738 (+   144us) service_pool.cc:224] Handling call
0709 13:04:53.438528 (+  9790us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:53.461784 (+ 23256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.486845 (+ 25061us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:53.498659 (+ 11814us) log.cc:844] Serialized 26 byte log entry
0709 13:04:53.823396 (+324737us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b18002a2da0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:04:54.174164 (+350768us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:54.641930 (+467766us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.643406 (+  1476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.643494 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.725857 (+ 82363us) spinlock_profiling.cc:243] Waited 9.87 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:54.727641 (+  1784us) raft_consensus.cc:1685] finished
0709 13:04:54.727666 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.727819 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":214786,"prepare.queue_time_us":20806,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":2859,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":31297,"spinlock_wait_cycles":74118528,"thread_start_us":787578,"threads_started":2,"wal-append.queue_time_us":986851}
I20260709 13:04:54.731381 28967 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:04:54.733572 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:54.655566 31801 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.755379 32037 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:54.756698 31778 raft_consensus.cc:2764] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:04:54.756222 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:54.759438 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:54.758708 31527 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:54.783214 31527 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.784049 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:54.758751 31778 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:54.715031 31715 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.759693 31858 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:54.818444 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:54.824357 29145 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:54.822150 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 116239us
W20260709 13:04:54.824869 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.819123 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:54.778195 31618 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:04:54.832244 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 156600us
W20260709 13:04:54.820312 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 112395us
I20260709 13:04:54.822628 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:54.823190 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:54.834538 29072 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.829025 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:54.829596 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:54.835579 29143 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:54.830471 29146 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:04:54.796473 31976 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:54.842176 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:54.838179 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:54.843823 29147 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:54.716853 31987 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.841116 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:54.814824 31842 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:54.848672 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:54.815961 32037 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:54.812168 31966 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:54.860798 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:54.861415 29143 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:54.852016 29185 leader_election.cc:336] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.618s (SENT)
I20260709 13:04:54.865259 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:54.865655 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:54.832041 31618 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:54.828635 32009 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:54.888100 29140 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:54.866755 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:54.850529 32031 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.123s	user 0.001s	sys 0.004s
W20260709 13:04:54.881814 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.893558 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.761381 31969 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.860533 32037 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:54.888549 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:54.903213 32037 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:54.903367 29066 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:04:54.889684 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:54.901583 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:54.903873 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:54.904289 29911 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:54.904495 29067 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:54.896760 29141 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:54.901526 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:54.897193 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:54.908018 32042 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:04:54.908214 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:54.908753 29216 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.909826 31849 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:54.910696 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:54.911353 29146 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
W20260709 13:04:54.912519 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 76394.6us
I20260709 13:04:54.818771 32039 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:04:54.759298 31869 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:04:54.902879 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 283613us
I20260709 13:04:54.914464 29072 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:54.888666 31842 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:54.917096 29139 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:54.881567 31729 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.918612 29216 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:54.928859 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:54.894631 31801 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:54.907032 29142 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:54.853298 31715 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.924280 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:54.892544 31953 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:54.933784 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 405270us
I20260709 13:04:54.933774 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:54.939088 32038 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:54.929528 31995 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.952697 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:54.974611 29910 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:54.953380 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:54.954366 29906 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:54.982218 29186 leader_election.cc:400] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
I20260709 13:04:54.954744 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:54.957070 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:54.960096 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.964306 29141 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:54.985596 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:54.986313 29146 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:54.964781 29143 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:04:54.937005 31951 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:54.881997 32034 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.252s
I20260709 13:04:54.970669 31959 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:54.920288 31866 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:54.997519 29073 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:54.998152 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:54.975209 29145 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:54.982595 29186 leader_election.cc:403] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:54.926600 31607 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:54.989586 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:54.991719 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:54.992082 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:54.992244 31796 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:54.992910 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:54.905970 32009 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.993815 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:55.025960 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.026711 29216 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:55.028160 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1847) took 2356 ms. Trace:
I20260709 13:04:55.028247 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:55.000758 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 82887.6us
I20260709 13:04:55.030699 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.033136 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.031541 29909 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:55.033679 29217 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:54.982110 29906 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:55.032713 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.034864 29221 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:55.032809 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:55.028610 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:54.993656 29902 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:55.054032 29218 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:55.032881 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.061416 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.062291 29214 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:04:54.995891 32038 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:55.036018 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:55.036171 29219 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:55.000064 31924 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:55.047880 29146 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.974431 31729 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:54.992713 31999 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:04:55.093331 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:55.093590 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 57677.4us
I20260709 13:04:54.985045 31995 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.026285 31976 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:55.029485 32045 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:55.032419 31830 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.048103 31803 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:55.102646 29907 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:55.106595 29141 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:55.109246 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.109869 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.110450 29064 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:04:55.112840 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:55.106035 29906 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.117338 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.086869 31953 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:04:55.125093 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.105898 31604 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.989749 31618 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.101133 32046 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:55.094518 31607 raft_consensus.cc:2764] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:04:55.120937 29906 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:55.118186 31778 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:55.106312 31866 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:04:55.128305 32041 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:55.115185 31999 consensus_queue.cc:1243] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:04:55.142237 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.119155 32045 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:55.135274 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:55.118183 32054 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:55.181950 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 46766.5us
I20260709 13:04:55.134347 31715 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:55.114262 31976 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:55.180630 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 86392.5us
I20260709 13:04:55.118640 31830 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:55.192199 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 213958us
I20260709 13:04:55.131335 31953 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.131958 31803 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.195703 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.111740 31842 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.182551 32046 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:55.187513 31618 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:55.141849 31604 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.185320 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:55.207739 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.207891 29908 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:04:55.187974 32057 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:55.196470 31987 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:55.213274 29064 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:55.193751 32041 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:55.214280 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 215714us
I20260709 13:04:55.198437 31999 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:55.216575 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 215451us
I20260709 13:04:55.218448 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:55.220326 29147 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:55.209347 31527 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:04:55.231909 29871 leader_election.cc:336] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.604s (SENT)
I20260709 13:04:55.236441 29144 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:55.237730 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.238225 29146 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:55.238694 29067 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:55.228756 31604 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:55.240176 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:55.228255 29904 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 4: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:55.249089 29065 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:04:55.216001 32057 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:55.214043 32059 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.255169 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:55.213479 32046 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.224527 31830 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:55.258757 32041 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.229637 31951 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:55.118422 32034 consensus_queue.cc:1175] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "730186a8c70f43f2a57d80163bbc37d2" responder_term: 5 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: [4.5-4.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:04:55.262113 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:55.287331 29067 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:55.229139 31953 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:55.255173 31778 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.237397 31999 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.223343 32065 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:55.280009 29911 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
W20260709 13:04:55.280119 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48234.9us
I20260709 13:04:55.281900 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:55.270687 31967 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.003s
I20260709 13:04:55.263151 32009 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:55.294813 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:55.240054 31527 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.293334 32067 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:04:55.327893 32076 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:55.333921 29065 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:55.212981 32038 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.336863 31969 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:55.293932 31987 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.339970 29069 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
W20260709 13:04:55.341041 32072 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:04:55.341763 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.279417 32059 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:55.334589 31944 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.701s	user 0.007s	sys 0.005s
I20260709 13:04:55.346017 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:55.348089 29067 raft_consensus.cc:2376] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7. Re-sending same reply.
I20260709 13:04:55.329557 31729 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:55.361965 29910 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:04:55.364868 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:55.365455 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 73020.8us
W20260709 13:04:55.366349 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 114678us
I20260709 13:04:55.283277 32045 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.334628 32065 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.374580 32065 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:55.376248 32059 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:55.371210 32064 log.cc:927] Time spent T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:04:55.378527 31902 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.000s
I20260709 13:04:55.378957 31902 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.363035 31944 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.380151 29071 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:55.381618 31944 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:55.383795 32059 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:55.384372 29906 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:04:55.384898 29906 pending_rounds.cc:85] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 4
I20260709 13:04:55.384423 32059 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.385205 29906 pending_rounds.cc:107] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:04:55.386127 29067 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:55.348604 31830 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:04:55.385531 29906 raft_consensus.cc:2889] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:55.358865 31987 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.393149 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:55.361181 32054 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:04:55.397007 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:55.383862 31866 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
I20260709 13:04:55.412098 29871 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:55.436856 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47958.3us
I20260709 13:04:55.392081 31944 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:55.442193 31969 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:55.442616 31944 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:55.443109 31944 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:55.444948 29037 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 2.082s (SENT) [suppressed 29 similar messages]
I20260709 13:04:55.447024 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:55.447633 29907 raft_consensus.cc:2376] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9. Re-sending same reply.
W20260709 13:04:55.448662 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 51694.1us
I20260709 13:04:55.450484 32044 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.450598 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:55.450911 31866 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:55.452425 31866 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.453625 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:55.454447 32068 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.454154 31866 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:55.455008 32067 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:55.454933 32068 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.455745 32067 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:55.457068 31607 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:55.455040 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:55.350507 32076 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:55.467756 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55900.5us
I20260709 13:04:55.396093 32073 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:55.469158 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:55.451258 32044 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.437552 31778 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:55.477346 29037 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:55.456998 32067 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.496578 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:55.496721 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:55.028936 29070 rpcz_store.cc:276] 0709 13:04:52.671589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:53.441206 (+769617us) service_pool.cc:224] Handling call
0709 13:04:53.789781 (+348575us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b5800005198. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:53.798311 (+  8530us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:53.835004 (+ 36693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:53.835050 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:53.835774 (+   724us) log.cc:844] Serialized 78 byte log entry
0709 13:04:54.329207 (+493433us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:54.816511 (+487304us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.823967 (+  7456us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.962369 (+138402us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b480027c258. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783ba36aa1 00007f783ba2db37 00007f783ba2d9ca 00007f783bb2526f 00007f783bb24f37 00007f783bb3955c 00007f783bb39508 00007f783bb39488 00007f783bb39443 00007f783bb386ec 00007f783aa62a96
0709 13:04:55.027750 (+ 65381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.027829 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:55.027884 (+    55us) raft_consensus.cc:1685] finished
0709 13:04:55.027901 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.028087 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7034,"prepare.queue_time_us":666522,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":3623,"raft.queue_time_us":80820,"raft.run_cpu_time_us":1545,"raft.run_wall_time_us":137897,"spinlock_wait_cycles":12096640,"thread_start_us":1116838,"threads_started":3,"wal-append.queue_time_us":788138}
I20260709 13:04:55.501381 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:55.505014 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:55.505831 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:55.465770 31924 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:55.363462 32057 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.467684 31803 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:55.472795 32073 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:55.513054 32009 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:55.490033 31830 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:55.518023 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:55.519688 29214 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:55.520767 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:55.496594 32079 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.525431 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.525656 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:55.526274 29215 raft_consensus.cc:2410] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:04:55.527552 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.528167 29213 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:55.502693 31607 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:55.523588 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:55.513870 31801 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:55.531039 31801 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:55.531428 31801 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:55.532596 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.534920 31944 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.513561 31953 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:55.510090 31924 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:55.516327 31999 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:55.514302 32046 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:55.544804 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:55.509516 29906 raft_consensus.cc:2376] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8. Re-sending same reply.
I20260709 13:04:55.519402 32009 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.550884 31803 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:55.553400 32009 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:55.553514 31803 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:55.553982 31803 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:55.571491 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 116513us
I20260709 13:04:55.528858 31858 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:55.504081 31778 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:55.582466 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 64285.6us
I20260709 13:04:55.588881 32036 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:55.589346 32036 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.589756 32036 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:55.539985 32079 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.593156 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:55.593962 32046 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:55.593786 31607 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.594374 32046 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:55.594792 32046 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:55.594806 32084 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:04:55.600018 31573 tablet_bootstrap.cc:492] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:55.600488 32059 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:55.601125 31573 tablet_bootstrap.cc:492] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:04:55.545856 31953 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:55.601961 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.604669 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:55.606571 32065 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:55.607009 32065 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:55.607370 32065 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:55.608177 31621 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.912s	user 0.010s	sys 0.004s
I20260709 13:04:55.609481 31924 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.610787 31924 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:55.607023 32037 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.699s	user 0.006s	sys 0.006s
I20260709 13:04:55.580163 29214 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
W20260709 13:04:55.617165 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.592002 32073 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.621337 29113 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:55.631352 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:55.633008 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:55.624058 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 278123us
I20260709 13:04:55.628468 32059 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.581975 31858 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:55.652406 31869 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:55.653000 31869 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:55.652859 31858 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.654407 31858 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:55.654928 31969 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:04:55.655113 29902 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.548s	user 0.008s	sys 0.000s
I20260709 13:04:55.655537 31969 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:55.656265 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.584395 32045 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:55.656667 31969 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.637534 32073 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:55.658027 31987 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:55.613420 32037 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.636669 32076 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:55.612433 31830 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:55.660794 32037 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:55.660725 31830 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.663084 31830 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:55.663560 31527 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:55.614364 31953 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:55.636973 32079 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:55.646198 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:55.629689 31573 ts_tablet_manager.cc:1403] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 2.383s	user 0.040s	sys 0.035s
I20260709 13:04:55.629760 31866 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.648712 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.635622 31621 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:55.653262 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.654040 31869 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.643738 31778 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:55.644192 32059 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.655139 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.673290 32039 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:55.672123 29222 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:55.681627 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.655612 29902 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:55.685003 29066 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:55.696561 29073 raft_consensus.cc:2410] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:04:55.643436 29037 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:55.700167 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:55.718284 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 74926.9us
I20260709 13:04:55.719375 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:55.702356 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:55.703948 29905 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:55.706326 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.708228 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:55.721128 29909 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:55.715054 29214 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:55.714299 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:55.722819 29217 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:55.712297 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:55.736599 29907 raft_consensus.cc:2410] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:04:55.737998 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.740561 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:55.740978 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.741619 29066 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:55.741827 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:55.742580 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.742842 31778 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:55.743461 29066 raft_consensus.cc:2410] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:04:55.744964 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:55.745191 31778 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:55.756309 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:55.762373 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.763582 29073 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:55.713187 32039 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:55.769382 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:55.770272 29910 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:55.720485 32075 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:55.698132 31830 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:04:55.738948 32037 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:55.751788 31621 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:55.778468 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.787147 29065 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4.
I20260709 13:04:55.716730 32065 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.794876 29910 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:55.796254 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:55.797395 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.789981 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:55.791119 31621 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:55.761782 31796 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:55.799640 29037 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:55.776453 32057 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:55.797853 29073 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:55.800998 29065 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:55.809633 32079 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.810086 31607 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.807790 29909 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:04:55.821513 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:55.893108 29910 raft_consensus.cc:2376] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6. Re-sending same reply.
I20260709 13:04:55.822925 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:55.825098 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:55.828295 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:55.895388 29908 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:55.895514 29906 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:55.828709 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:55.828979 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:04:55.888235 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 75897.8us
I20260709 13:04:55.897078 29911 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:55.892354 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:55.830503 32057 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:55.898846 29065 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:04:55.760281 32076 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:04:55.785007 31944 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:55.788678 31967 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:55.889748 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:55.891520 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:55.918584 31621 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.935678 29146 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:55.937222 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:55.888799 32037 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:55.921969 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:55.900727 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:55.946123 29910 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:55.938385 29909 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:04:55.928910 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1789) took 1118 ms. Trace:
I20260709 13:04:55.939416 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:55.947517 29071 rpcz_store.cc:276] 0709 13:04:54.811102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.818886 (+  7784us) service_pool.cc:224] Handling call
0709 13:04:55.928711 (+1109825us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54002ac954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:55.928845 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1274797312}
W20260709 13:04:55.940598 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 108015us
I20260709 13:04:55.934464 31803 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.950348 29908 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:55.959450 29071 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:04:55.923599 32088 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.322s	user 0.000s	sys 0.002s
I20260709 13:04:55.967334 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:55.968568 29148 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:55.970146 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:55.888775 32080 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 13:04:55.979128 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:55.984289 29186 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:55.925401 31995 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:55.889137 32009 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.952960 32066 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:55.986039 31778 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:55.890468 32040 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:04:55.898375 31607 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:55.919034 31801 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.894562 32079 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:55.904973 32036 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:56.010185 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:56.021772 29908 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:56.022006 32079 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:56.012502 31778 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.023133 32036 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:56.011709 32073 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.013789 32057 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.033550 29217 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:55.912202 32039 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:56.016284 31944 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.018309 31607 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:56.018968 29213 raft_consensus.cc:1217] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[3.3-8.5]   Dedup: 3.3->[6.4-8.5]
I20260709 13:04:56.045763 29213 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: [3.3-8.5]
I20260709 13:04:55.988837 32089 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.046365 32089 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:56.049655 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:56.050457 32089 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:56.050974 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2017) took 1065 ms. Trace:
I20260709 13:04:56.051431 29220 rpcz_store.cc:276] 0709 13:04:54.985545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.993415 (+  7870us) service_pool.cc:224] Handling call
0709 13:04:56.049475 (+1056060us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:56.050878 (+  1403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2632576}
I20260709 13:04:56.018702 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:56.053660 31842 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:56.054250 31842 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:56.054366 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:56.055162 32055 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:56.055316 29909 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:56.055711 32055 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.056167 29219 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:04:56.056449 32055 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:56.058786 31803 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:56.060429 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:56.060565 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:56.061026 31801 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:56.061420 29910 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:55.989113 32059 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:56.032673 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 62643.5us
I20260709 13:04:56.024718 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:55.989537 32067 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:56.039054 32073 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.077792 29141 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:04:56.042163 32097 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:04:56.045084 31573 raft_consensus.cc:359] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:56.045413 31944 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.024422 31866 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:56.097640 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:56.065147 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:55.989009 31924 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:55.989746 32066 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:56.067835 29910 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:56.070539 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:56.079425 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:56.088690 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:56.114698 29148 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:56.095515 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 156173us
I20260709 13:04:56.061882 31607 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:56.116088 29908 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:56.091178 31729 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.089844 32059 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:56.032675 31753 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 13:04:56.065187 32106 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:56.121155 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:56.093849 31573 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:04:56.105101 32039 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:56.096674 32067 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:56.150910 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:56.117435 31924 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.117270 31607 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.127348 32059 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:56.156301 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 48522.6us
I20260709 13:04:56.116669 32066 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:56.157141 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:56.055512 31842 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.121569 32091 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:56.112715 31869 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:04:56.163281 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:56.130861 32106 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.128053 31621 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:56.151366 31618 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:56.218789 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.155248 31924 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:56.221042 29071 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:56.154896 32038 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:56.222923 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.225143 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:56.223373 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.220492 32080 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:56.229836 29069 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
W20260709 13:04:56.216632 31858 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.000s
I20260709 13:04:56.162034 32059 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:56.134071 31801 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:04:55.979401 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:56.158924 31607 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:56.132556 31729 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.162406 32091 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.219379 32089 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:56.221516 32106 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:56.221916 32040 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:56.151878 31953 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:56.251529 32110 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 13:04:56.252386 32113 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:04:56.253026 31729 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:56.236667 31842 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:56.223851 31618 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:56.259613 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:56.262647 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 111893us
I20260709 13:04:56.232234 31621 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:56.265226 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 108142us
I20260709 13:04:56.239668 32091 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:56.274677 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:56.276795 29068 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:56.260303 32035 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:04:56.268731 32046 consensus_meta.cc:287] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.673s	user 0.012s	sys 0.006s
I20260709 13:04:56.286815 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:56.287731 29071 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:04:56.266322 32118 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:56.269440 31618 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:56.291333 31621 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:56.258983 32089 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.282060 32035 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:56.324039 29071 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:04:56.225265 31573 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:56.295277 32009 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:56.311717 32118 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:56.280987 31966 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 13:04:56.330921 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.330459 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:56.338114 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62768.5us
I20260709 13:04:56.280823 32040 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:04:56.349565 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:56.269253 32067 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.356428 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:56.397225 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:56.398053 29071 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:56.357704 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:56.403298 31573 ts_tablet_manager.cc:1434] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.705s	user 0.014s	sys 0.001s
W20260709 13:04:56.403782 31573 ts_tablet_manager.cc:1467] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Tablet startup took 721ms
I20260709 13:04:56.332150 31944 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:56.365965 29147 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:56.367249 29222 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:56.333019 32055 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.368046 29220 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:56.369567 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.336072 32125 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.346089 32035 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:56.370849 32122 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:04:56.388298 31953 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:56.392894 32040 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:56.353915 32073 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:56.395488 32123 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:04:56.401728 32121 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:04:56.403522 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:56.309994 32046 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:56.366858 32118 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.359998 29219 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:56.537178 29141 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
W20260709 13:04:56.369903 32114 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:04:56.334760 32080 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.545576 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:56.393991 32119 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:04:56.565208 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.569510 29910 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:04:56.574393 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 146438us
I20260709 13:04:56.571981 29222 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:56.545536 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:56.538367 32009 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.567406 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:56.579118 29139 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:56.587898 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.589391 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.590104 29139 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:56.591041 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:56.579406 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.579643 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.579473 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.591667 29139 raft_consensus.cc:2376] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6. Re-sending same reply.
I20260709 13:04:56.592746 29142 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:56.618737 29071 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:04:56.632711 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:56.634199 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:56.639282 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.561324 32040 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:56.643426 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:56.563060 32124 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:56.645639 29185 leader_election.cc:336] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.731s (SENT)
W20260709 13:04:56.646786 29148 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.529s	user 0.005s	sys 0.005s
W20260709 13:04:56.649669 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.663875 29148 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:56.664556 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:56.664754 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.665215 29213 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:56.659459 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:56.661163 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.665432 29142 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:56.645465 29902 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:56.668069 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.668854 29142 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:56.669386 32070 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.390s	user 0.000s	sys 0.011s
I20260709 13:04:56.667599 29071 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:04:56.675264 31976 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:56.675458 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:56.677167 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:56.678562 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:56.679201 29071 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:56.682958 29215 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:04:56.687194 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:56.687847 29070 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:04:56.688165 29071 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:04:56.688339 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1797) took 1115 ms. Trace:
I20260709 13:04:56.688709 29070 rpcz_store.cc:276] 0709 13:04:55.572947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.573142 (+   195us) service_pool.cc:224] Handling call
0709 13:04:56.687042 (+1113900us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b5800005198. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:56.688284 (+  1242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34155904}
I20260709 13:04:56.677181 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1798) took 1100 ms. Trace:
I20260709 13:04:56.561796 31944 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:04:56.691841 29070 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:56.689071 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:56.733825 32126 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.389s	user 0.000s	sys 0.000s
I20260709 13:04:56.553772 32125 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:56.689697 29068 rpcz_store.cc:276] 0709 13:04:55.577311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.577472 (+   161us) service_pool.cc:224] Handling call
0709 13:04:56.676989 (+1099517us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b5400069fd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:56.677115 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1549307264}
I20260709 13:04:56.753499 29071 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:56.695174 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:56.740747 32101 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.774s	user 0.000s	sys 0.012s
I20260709 13:04:56.765327 29070 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:56.765555 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.663333 32079 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.784152 29071 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:04:56.751420 32118 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:56.736217 32106 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:56.643362 32009 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.785867 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.785820 32040 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.788417 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1803) took 1156 ms. Trace:
I20260709 13:04:56.791576 29068 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:56.795360 29073 rpcz_store.cc:276] 0709 13:04:55.632228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.887019 (+254791us) service_pool.cc:224] Handling call
0709 13:04:55.945680 (+ 58661us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:55.945958 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:55.945994 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:55.946569 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 13:04:56.152647 (+206078us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:56.152701 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.152793 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.788136 (+635343us) raft_consensus.cc:1685] finished
0709 13:04:56.788165 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.788338 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":174639,"threads_started":1,"wal-append.queue_time_us":301291}
I20260709 13:04:56.798316 29073 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:04:56.632102 32080 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.798682 29073 pending_rounds.cc:85] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 5
I20260709 13:04:56.801337 29073 pending_rounds.cc:107] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:04:56.801625 29073 raft_consensus.cc:2889] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:56.801712 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:56.806661 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.807297 29068 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:56.689050 32124 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:56.718799 31607 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.808379 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.805680 29073 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:04:56.809983 29140 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:56.810881 29073 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:56.790238 32079 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.818964 29141 raft_consensus.cc:2376] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6. Re-sending same reply.
I20260709 13:04:56.825510 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.825922 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.827548 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.828652 29142 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:56.834322 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.834934 31607 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:56.840845 29216 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:56.841346 32055 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:56.842130 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:56.843015 29073 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:56.848839 29064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 293) took 1750 ms (client timeout 1573 ms). Trace:
I20260709 13:04:56.849380 32009 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:56.849314 29064 rpcz_store.cc:269] 0709 13:04:55.098210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.117080 (+ 18870us) service_pool.cc:224] Handling call
0709 13:04:56.848620 (+1731540us) spinlock_profiling.cc:243] Waited 95.2 ms on lock 0x7b54001d8654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:56.848753 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218876672}
I20260709 13:04:56.851542 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:56.831861 32039 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.677s	user 0.005s	sys 0.007s
I20260709 13:04:56.855366 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.856840 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.857993 29069 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:04:56.866740 29073 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:56.855618 32043 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:56.869494 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.877162 32079 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:56.861337 32046 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:56.890446 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:56.893064 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:56.895022 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:56.895532 29140 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:56.877220 31924 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.003s
I20260709 13:04:56.895747 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:56.903260 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:56.911494 31869 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:56.899447 32125 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:56.917287 29910 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:56.917646 29064 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: [3.3-8.5]
I20260709 13:04:56.917755 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:56.918705 32132 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:56.920696 29902 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:56.922103 31953 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:56.923398 32080 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:56.924432 32106 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:56.925180 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.913203 32136 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.929327 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:56.929360 32106 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:04:56.937829 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:56.938664 29221 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:04:56.939066 31618 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.001s
I20260709 13:04:56.939479 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:56.940256 29215 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:56.937855 31715 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:56.939477 31618 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:56.942330 29143 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
W20260709 13:04:56.404238 31573 ts_tablet_manager.cc:1469] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Trace:
0709 13:04:39.694317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.703065 (+  8748us) service_pool.cc:224] Handling call
0709 13:04:41.322887 (+1619822us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e91a2 00007f783c92abb9 00007f783ba5bd88 00007f783bb1fa07 00007f783bb51f4e 00007f783bb36fdd 00007f783bb36f89 00007f783bb36f19 00007f783bb36ee1
0709 13:04:42.089447 (+766560us) ts_tablet_manager.cc:934] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
0709 13:04:47.874817 (+5785370us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:04:49.251812 (+1376995us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:49.264262 (+ 12450us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:04:49.752944 (+488682us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:04:49.786547 (+ 33603us) tablet_replica.cc:206] Creating consensus instance
0709 13:04:49.836857 (+ 50310us) ts_tablet_manager.cc:989] Replica registered.
0709 13:04:49.897011 (+ 60154us) inbound_call.cc:177] Queueing success response
0709 13:04:53.246494 (+3349483us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:04:55.712607 (+2466113us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:04:55.920044 (+207437us) tablet_replica.cc:261] Starting instrumentation
0709 13:04:55.920955 (+   911us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":303,"mutex_wait_us":35970,"raft.queue_time_us":13299,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":11856,"spinlock_wait_cycles":18639232,"tablet-copy.queue_time_us":208618,"tablet-download-pool.queue_time_us":196,"tablet-download-pool.run_cpu_time_us":1083,"tablet-download-pool.run_wall_time_us":1785532,"thread_start_us":2478153,"threads_started":2}
I20260709 13:04:56.941007 31715 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:56.944263 31618 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:56.900311 31924 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:56.946691 29140 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:56.953298 31715 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:56.954538 31715 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:56.947731 32124 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:56.959957 32134 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:56.960150 31924 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:56.961634 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 58471.7us
I20260709 13:04:56.967576 32044 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:56.970927 32069 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:04:56.971374 32059 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.709s	user 0.004s	sys 0.004s
I20260709 13:04:56.971426 32069 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:56.973932 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:56.974591 29908 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:04:56.975803 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:56.977725 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:56.978180 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:56.978962 29073 raft_consensus.cc:2376] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 4. Re-sending same reply.
I20260709 13:04:56.979044 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:56.980377 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:56.982115 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:56.982707 29073 raft_consensus.cc:2368] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5.
I20260709 13:04:56.983670 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:56.984366 29073 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:04:56.931159 32136 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:56.985105 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:56.993656 29068 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:56.979040 32046 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:56.996863 31869 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:56.985216 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:56.990958 32066 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.775s	user 0.000s	sys 0.010s
I20260709 13:04:57.003068 29073 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:56.978180 32124 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:56.978348 32076 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:56.980978 31618 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:56.997864 32046 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:56.982848 32118 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:56.972656 32059 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.007359 29064 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:57.006429 32038 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:04:57.004767 32066 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:57.003950 31842 consensus_meta.cc:287] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.729s	user 0.013s	sys 0.000s
I20260709 13:04:57.010900 32118 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.985330 32080 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:56.992983 32128 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:04:57.013535 32140 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:04:57.015604 32059 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:57.017525 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:57.030326 32068 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:57.030836 29908 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:57.022984 32115 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.953s	user 0.000s	sys 0.010s
I20260709 13:04:57.034255 32066 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:57.034886 32118 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:57.035987 29144 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:57.037505 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 402) took 1205 ms. Trace:
I20260709 13:04:57.038192 29908 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:04:57.038251 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:57.039057 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1799) took 1460 ms (client timeout 1633 ms). Trace:
W20260709 13:04:57.039426 29067 rpcz_store.cc:269] 0709 13:04:55.578301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.578582 (+   281us) service_pool.cc:224] Handling call
0709 13:04:57.038094 (+1459512us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b5800005198. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:04:57.039004 (+   910us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30828928}
I20260709 13:04:57.000108 32125 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:04:57.039697 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.040720 31967 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:56.985555 32069 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.041275 31967 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:57.012267 32076 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.042176 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 461) took 1216 ms. Trace:
I20260709 13:04:57.042214 31967 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.042598 29903 rpcz_store.cc:276] 0709 13:04:55.825816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.829337 (+  3521us) service_pool.cc:224] Handling call
0709 13:04:55.829832 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:55.955416 (+125584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:55.955459 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:55.956026 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:04:56.008188 (+ 52162us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:56.008244 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.008339 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:57.022326 (+1013987us) spinlock_profiling.cc:243] Waited 9.27 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:57.041914 (+ 19588us) raft_consensus.cc:1685] finished
0709 13:04:57.041936 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.042117 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125102,"raft.queue_time_us":126969,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":53,"spinlock_wait_cycles":21318912,"thread_start_us":51668,"threads_started":1,"wal-append.queue_time_us":67246}
I20260709 13:04:57.045333 32136 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:57.037844 29144 rpcz_store.cc:276] 0709 13:04:55.888882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.895096 (+  6214us) service_pool.cc:224] Handling call
0709 13:04:57.037293 (+1142197us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54000429d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:57.037442 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2507260160}
I20260709 13:04:57.052081 29071 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:04:57.054064 29071 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:57.052627 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:57.057917 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.059439 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:57.043713 32076 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:57.062142 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:57.067198 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:57.068567 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:57.012496 32124 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.064772 31842 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:57.068877 32055 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:57.071146 32075 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:57.076717 32124 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:57.071328 32091 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.788s	user 0.013s	sys 0.000s
I20260709 13:04:57.069975 31944 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:57.070230 32066 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:57.082420 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1890) took 1544 ms. Trace:
I20260709 13:04:57.069042 32080 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:04:57.091897 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 93587.7us
I20260709 13:04:57.092388 29217 rpcz_store.cc:276] 0709 13:04:55.538146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.831428 (+293282us) service_pool.cc:224] Handling call
0709 13:04:55.896332 (+ 64904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.033988 (+137656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.082105 (+1048117us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54002841d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:57.082344 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296391040}
I20260709 13:04:57.094380 29217 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:04:57.094640 29218 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:57.095675 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:57.096311 29221 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:57.097134 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:57.098239 31902 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:57.099462 29220 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:04:57.072988 31842 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.100211 31842 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:57.109015 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.074502 32073 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:57.113595 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.113466 29185 leader_election.cc:336] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.538s (SENT)
I20260709 13:04:57.096719 32080 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.110702 32118 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.092682 32091 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.138116 32091 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:57.122155 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:57.139799 31944 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:57.123847 29111 leader_election.cc:400] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5.
I20260709 13:04:57.123077 31715 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:57.140259 31944 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:57.146510 31944 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:57.146289 31715 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.148090 32075 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:57.148720 31715 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:57.126793 32065 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:57.117652 32068 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.092267 32066 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:57.149892 32068 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.150671 32091 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.151278 32091 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.152197 32068 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:57.122869 32073 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.118326 31902 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.154345 31902 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:57.155776 32073 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:57.130175 32089 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:57.156520 31902 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:57.150388 32066 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.157718 32066 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:57.131626 29215 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:57.160616 32065 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:57.132243 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:57.162730 32065 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:57.163568 32065 consensus_queue.cc:1243] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:04:57.132563 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:57.133016 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.164340 29909 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:57.164745 32129 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:57.164963 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.165290 32129 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:57.165763 32129 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:57.166443 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.167434 32089 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:57.167820 32089 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.168182 32089 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:57.170404 32068 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:57.134855 31924 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:57.135468 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:57.136013 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:57.180680 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44823.2us
I20260709 13:04:57.181497 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.179695 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1808) took 1493 ms. Trace:
I20260709 13:04:57.182461 29066 rpcz_store.cc:276] 0709 13:04:55.686731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.767318 (+ 80587us) service_pool.cc:224] Handling call
0709 13:04:57.179502 (+1412184us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b54002d3a54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:57.179632 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782313856}
I20260709 13:04:57.136236 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:57.132247 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:57.140865 29111 leader_election.cc:403] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:04:57.186177 29908 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:57.186664 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 62986.3us
I20260709 13:04:57.186734 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:57.141825 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.187780 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.145273 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.133565 32067 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:57.175851 32140 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:57.176697 29067 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:57.177399 32118 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:57.184713 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:57.189371 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 44248.3us
I20260709 13:04:57.174548 32068 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:57.195681 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.189291 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.197821 29215 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:04:57.195132 32073 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:57.199047 32073 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.192349 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.199455 32073 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:57.199569 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:57.200969 32075 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:57.202337 32055 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:57.202831 32055 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:57.204955 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:57.209506 29222 raft_consensus.cc:2368] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5.
I20260709 13:04:57.214588 29908 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:04:57.182372 32079 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:57.216936 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:57.218331 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:57.220297 29220 raft_consensus.cc:2368] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8.
I20260709 13:04:57.220167 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.221138 29221 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:57.193213 32065 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:57.191462 31995 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:57.225447 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:57.232874 29907 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:57.234023 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:57.195514 32068 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:57.234649 29215 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:57.239606 29071 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:57.252249 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:57.249434 29871 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:57.193969 31604 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:04:57.226475 32091 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:57.229395 31830 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:57.268121 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 78884.1us
I20260709 13:04:57.268559 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:57.236825 32046 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:57.274237 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:57.274567 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 89961.6us
I20260709 13:04:57.271780 29068 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:57.232676 31995 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.204049 32055 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.260615 31987 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:57.252102 31607 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:57.273808 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:57.273432 29221 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:57.270427 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:57.277614 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:57.279357 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:57.279798 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:57.280247 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:57.282998 29068 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:04:57.289526 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:57.270696 32148 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:04:57.285213 29148 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:57.225438 31621 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:57.276182 32157 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:57.288513 29144 raft_consensus.cc:2376] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5. Re-sending same reply.
I20260709 13:04:57.299831 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:57.292903 29871 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:57.236143 32147 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:04:57.294651 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:57.311143 29142 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:57.236510 31969 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:04:57.252645 32038 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:04:57.298597 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:57.299854 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:57.302897 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.311020 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:57.278666 31995 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:57.260401 31842 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.314270 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:57.292610 31858 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:57.328320 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:57.327996 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:57.314585 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:57.322453 29147 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:57.330327 29214 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 5: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:57.323190 29068 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:57.315218 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:57.332207 29143 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:57.299201 32055 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:57.274122 31869 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:04:57.325351 29221 raft_consensus.cc:2368] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:04:57.336174 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:57.298710 31621 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.284335 32091 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:04:57.310063 32076 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:57.378270 32076 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.378806 32076 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:57.361440 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:57.380246 31944 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.380880 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:57.382027 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:57.382421 31944 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:57.382637 29068 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:57.383718 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:57.362027 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59884.7us
I20260709 13:04:57.332846 32156 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:57.335752 31902 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.403352 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:57.298107 31924 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
W20260709 13:04:57.377427 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63239us
I20260709 13:04:57.409194 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:57.394271 29907 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:04:57.403255 29068 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:57.404127 29214 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:57.365262 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.413757 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:04:57.414312 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 49164us
I20260709 13:04:57.415263 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:57.298195 32079 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:04:57.278613 31830 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:57.367486 32144 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:04:57.335641 31987 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:57.364584 31621 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:57.418557 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:57.431910 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.436163 31967 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:57.436864 31967 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:57.437983 31967 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:57.383775 32150 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:04:57.423254 29068 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:57.422355 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:57.421882 29185 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:57.335956 32080 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 13:04:57.424180 31902 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:57.428747 32120 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.149s	user 0.008s	sys 0.003s
I20260709 13:04:57.312770 31753 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:04:57.430298 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:57.443331 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:57.513387 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1817) took 1239 ms. Trace:
I20260709 13:04:57.755982 29069 rpcz_store.cc:276] 0709 13:04:56.273994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.274267 (+   273us) service_pool.cc:224] Handling call
0709 13:04:57.513115 (+1238848us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b54001d8b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:57.513321 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337527552}
W20260709 13:04:57.707957 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.533831 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1811) took 1835 ms. Trace:
I20260709 13:04:57.762949 29065 rpcz_store.cc:276] 0709 13:04:55.698663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.906278 (+207615us) service_pool.cc:224] Handling call
0709 13:04:56.850541 (+944263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.850734 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:56.850758 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:56.850800 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:56.850832 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.850888 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:57.533558 (+682670us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b54001d8650. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:57.533741 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2119586176}
I20260709 13:04:57.648211 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:57.765681 29144 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:57.509557 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:57.767446 29213 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
W20260709 13:04:57.774327 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 352486us
W20260709 13:04:57.753974 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.586916 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.649982 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:04:57.494978 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 196440us
W20260709 13:04:57.838604 28967 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 161) took 21651 ms (client timeout 15000 ms). Trace:
W20260709 13:04:57.420034 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 231086us
I20260709 13:04:57.693996 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:04:57.823567 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 401254us
I20260709 13:04:57.405269 31858 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:57.742995 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:57.648052 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:57.867552 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:57.592325 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:57.840207 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:57.591651 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.869735 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:57.744119 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:57.450500 29147 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:04:57.503038 29214 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:04:57.854478 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:57.866600 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:57.871040 29908 raft_consensus.cc:2368] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8.
I20260709 13:04:57.840777 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:57.868153 29141 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:57.836958 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:57.871627 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:57.872824 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:57.876020 29909 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:57.876614 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:57.866434 29220 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:57.878268 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:57.808961 32035 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:57.745497 29902 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:57.529598 32125 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:57.882066 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:57.506368 32161 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:04:57.810688 32133 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.464203 32154 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:57.594678 31618 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:57.507861 32046 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:04:57.809192 29904 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.005s	sys 0.012s
I20260709 13:04:57.893476 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:04:57.594045 32107 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.640s	user 0.020s	sys 0.000s
I20260709 13:04:57.502627 32128 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.924571 32128 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.927070 32128 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:57.929970 32009 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:57.936393 32169 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:57.937232 32169 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:57.940187 32166 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.940444 29216 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:57.940753 32166 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.943002 32166 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:57.917160 32046 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:57.916570 32133 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.884799 32167 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:57.949000 32133 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:57.884872 32168 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.897238 32125 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:57.952358 32168 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.952773 32125 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.895949 31858 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:57.954792 32168 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:57.954872 31858 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.921829 29905 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEARNER]: Advancing to term 7
W20260709 13:04:57.958118 31995 consensus_meta.cc:287] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.622s	user 0.009s	sys 0.001s
I20260709 13:04:57.958554 31995 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.958895 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:57.959766 29069 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:57.528564 31987 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.960248 29218 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:57.963491 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:57.964336 29157 tablet_copy_service.cc:342] P 730186a8c70f43f2a57d80163bbc37d2: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-79cdfc6a725145348b70034422ef477b received from {username='slave'} at 127.0.0.1:60830
I20260709 13:04:57.894073 29070 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:04:57.841624 31607 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:57.965080 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:57.965792 31607 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:57.966198 31607 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:57.966249 29139 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:04:57.964236 29143 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:57.941614 29216 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:57.968827 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 75453.7us
W20260709 13:04:57.886752 29187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.023s	sys 0.016s
I20260709 13:04:57.440848 32067 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:04:57.891705 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.891366 29065 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:04:57.887552 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:57.972388 29214 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:57.810160 32157 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:04:57.891197 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 54339.3us
I20260709 13:04:57.888425 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:57.650239 32043 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.840142 32091 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:57.892398 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:04:57.882078 32124 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.016s	sys 0.013s
I20260709 13:04:57.527832 31830 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:57.884693 31604 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:57.881628 32040 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:57.907701 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:58.055224 29220 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:58.057359 29112 leader_election.cc:400] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8.
I20260709 13:04:58.057752 29112 leader_election.cc:403] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:04:57.517136 31842 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:57.524078 32076 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:57.884539 32156 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:57.668700 32129 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.918210 32066 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.965137 29157 tablet_copy_service.cc:434] P 730186a8c70f43f2a57d80163bbc37d2: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-79cdfc6a725145348b70034422ef477b on tablet 79cdfc6a725145348b70034422ef477b with peer 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:57.973277 29187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.023s	sys 0.016s
W20260709 13:04:57.974709 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 98156.5us
I20260709 13:04:58.073431 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:57.889914 32162 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:57.901566 32147 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.533823 32073 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.030058 32118 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:04:57.610078 32089 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:57.908757 32148 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:04:58.030082 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 43943us
I20260709 13:04:58.092674 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 412) took 1725 ms. Trace:
W20260709 13:04:57.875159 28967 rpcz_store.cc:269] 0709 13:04:36.095741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.120842 (+ 25101us) service_pool.cc:224] Handling call
0709 13:04:57.746980 (+21626138us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:55.087194 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:55.197973 (+110779us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:55.206105 (+  8132us) write_op.cc:435] Acquired schema lock
0709 13:04:55.209339 (+  3234us) tablet.cc:662] Decoding operations
0709 13:04:55.229079 (+ 19740us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:55.229154 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:55.229184 (+    30us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:04:55.268971 (+ 39787us) tablet.cc:701] Row locks acquired
0709 13:04:55.269003 (+    32us) write_op.cc:260] PREPARE: finished
0709 13:04:55.269202 (+   199us) write_op.cc:270] Start()
0709 13:04:55.269373 (+   171us) write_op.cc:276] Timestamp: P: 1783602295269133 usec, L: 0
0709 13:04:55.269395 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:04:55.270075 (+   680us) log.cc:844] Serialized 4367 byte log entry
0709 13:04:55.648273 (+378198us) op_driver.cc:464] REPLICATION: finished
0709 13:04:55.795886 (+147613us) write_op.cc:301] APPLY: starting
0709 13:04:55.799798 (+  3912us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:55.820167 (+ 20369us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:55.820283 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:55.821643 (+  1360us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:55.821889 (+   246us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:04:55.821943 (+    54us) write_op.cc:312] APPLY: finished
0709 13:04:55.822234 (+   291us) log.cc:844] Serialized 83 byte log entry
0709 13:04:55.822418 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:55.822548 (+   130us) write_op.cc:454] Released schema lock
0709 13:04:55.822591 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:55.822981 (+   390us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:55.823012 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":741640,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":118478,"apply.run_cpu_time_us":3290,"apply.run_wall_time_us":44825,"num_ops":11,"prepare.queue_time_us":276371,"prepare.run_cpu_time_us":21634,"prepare.run_wall_time_us":321307,"raft.queue_time_us":189031,"raft.run_cpu_time_us":6550,"raft.run_wall_time_us":48045,"replication_time_us":378697,"thread_start_us":368963,"threads_started":4,"wal-append.queue_time_us":172521,"wal-append.run_cpu_time_us":32254,"wal-append.run_wall_time_us":1382552}]]}
W20260709 13:04:58.032246 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 49848.4us
I20260709 13:04:58.036552 29221 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:57.910617 32154 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.911123 31618 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:58.038882 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:58.058710 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:58.056733 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:58.095218 29139 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
W20260709 13:04:58.049168 32124 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.017s	sys 0.013s
W20260709 13:04:58.078545 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 200128us
W20260709 13:04:58.064545 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 186348us
I20260709 13:04:58.081523 29140 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:58.036455 32067 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:58.137938 29220 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:04:58.087083 29143 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
W20260709 13:04:58.087807 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.029341 29213 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:58.066321 31729 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8.
I20260709 13:04:58.070091 31527 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:04:58.095173 29141 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:58.052248 31604 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.072774 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:58.096987 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:58.160593 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63678.7us
I20260709 13:04:58.109170 29904 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:58.101815 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:58.093237 29145 rpcz_store.cc:276] 0709 13:04:56.367279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.368052 (+   773us) service_pool.cc:224] Handling call
0709 13:04:56.368475 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:56.813732 (+445257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:56.813752 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:56.819319 (+  5567us) log.cc:844] Serialized 26 byte log entry
0709 13:04:57.134873 (+315554us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b18003f1300. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:04:57.234549 (+ 99676us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:57.234588 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.092348 (+857760us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b5400041354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:58.092395 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:58.092443 (+    48us) raft_consensus.cc:1685] finished
0709 13:04:58.092461 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.092611 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":700,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":14068,"spinlock_wait_cycles":1027965568,"thread_start_us":99503,"threads_started":1,"wal-append.queue_time_us":452752}
I20260709 13:04:58.097589 29113 leader_election.cc:400] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 4 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5.
I20260709 13:04:58.161857 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:58.104624 32176 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:58.129475 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:58.151803 31621 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.716s	user 0.012s	sys 0.000s
I20260709 13:04:58.105979 29217 raft_consensus.cc:1217] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[5.4-8.5]   Dedup: 5.4->[8.5-8.5]
I20260709 13:04:58.093932 32162 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:58.154771 31995 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:58.037706 32043 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.178977 32162 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.180866 32043 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:58.078012 32066 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.182446 31858 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:58.182902 31858 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.183019 32066 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:58.104434 31830 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:58.183298 31858 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:58.183632 31830 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.184055 31830 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:58.184777 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:58.185776 29904 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:58.188329 32067 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.188951 32076 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:58.090144 32147 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.189496 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:58.189895 32118 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:58.190184 29071 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:04:58.098102 32154 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:58.191216 32147 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:58.155604 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:58.163071 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 69233.2us
I20260709 13:04:58.193239 29113 leader_election.cc:400] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8.
I20260709 13:04:58.194293 29113 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:58.195181 31729 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:04:58.096690 29039 connection.cc:256] RPC call timeout handler was delayed by -2.1497161240004061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.225s for a response
W20260709 13:04:58.196384 29039 connection.cc:256] RPC call timeout handler was delayed by -1.8997398820001763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.35440000000000005s for a response
I20260709 13:04:58.165203 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:58.197556 31976 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.198062 31976 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.198464 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 306) took 1001 ms. Trace:
I20260709 13:04:58.198856 29066 rpcz_store.cc:276] 0709 13:04:57.197741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.216222 (+ 18481us) service_pool.cc:224] Handling call
0709 13:04:57.227338 (+ 11116us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:57.253063 (+ 25725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:57.253100 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:57.263012 (+  9912us) log.cc:844] Serialized 26 byte log entry
0709 13:04:57.337343 (+ 74331us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:57.337399 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.337497 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:58.197616 (+860119us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b2400169328. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78338a69c4 00007f78338a4acb 00007f7833a5fdc9 00007f7833a5fa55 00007f7833a5efb0 00007f78366d5488 00007f78366d5be8 00007f78366ad398 00007f78366a876a 00007f78366a7bb6
0709 13:04:58.198161 (+   545us) raft_consensus.cc:1685] finished
0709 13:04:58.198183 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.198392 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":15681,"spinlock_wait_cycles":99930880,"thread_start_us":47562,"threads_started":1,"wal-append.queue_time_us":164063}
I20260709 13:04:58.200304 29216 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:58.197309 29039 connection.cc:256] RPC call timeout handler was delayed by -1.802953341999455s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2723s for a response
W20260709 13:04:58.201661 29039 connection.cc:256] RPC call timeout handler was delayed by -1.5188294879990281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 13:04:58.202013 29039 connection.cc:256] RPC call timeout handler was delayed by -1.5103467979997731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
W20260709 13:04:58.202273 29039 connection.cc:256] RPC call timeout handler was delayed by -1.4959883319988876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
W20260709 13:04:58.202596 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.165565 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:58.165911 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:58.166407 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:58.206197 29146 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:58.166774 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 65072.9us
I20260709 13:04:58.206125 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:58.207602 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:58.208058 29907 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:58.208652 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:58.202523 29039 connection.cc:256] RPC call timeout handler was delayed by -1.3429467979995025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 13:04:58.209435 29039 connection.cc:256] RPC call timeout handler was delayed by -1.3378161669998008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
W20260709 13:04:58.209707 29039 connection.cc:256] RPC call timeout handler was delayed by -1.3348467979994894s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
I20260709 13:04:58.209753 29905 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:04:58.169384 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:58.169350 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:58.171236 32172 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:58.171382 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:04:58.210911 29039 connection.cc:256] RPC call timeout handler was delayed by -1.2862467979994108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2049s for a response
I20260709 13:04:58.158407 31604 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:58.172698 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:58.171723 32091 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:58.222164 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:58.168900 32181 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:04:58.197265 31729 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:58.175872 31618 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.176748 32158 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:58.177063 31924 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:58.211465 32172 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.220474 32167 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:58.191097 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:58.262658 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:04:58.154878 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:58.202486 29112 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:58.167752 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:58.206175 29147 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
W20260709 13:04:58.212116 29039 connection.cc:256] RPC call timeout handler was delayed by -1.2799467979994006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
W20260709 13:04:58.264717 29039 connection.cc:256] RPC call timeout handler was delayed by -1.1773467979992347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
W20260709 13:04:58.233120 32068 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.938s	user 0.011s	sys 0.000s
I20260709 13:04:58.213352 29139 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:58.195640 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:58.216873 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:04:58.217705 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 48411.2us
I20260709 13:04:58.226166 31715 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:58.266585 29066 raft_consensus.cc:2410] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:58.269731 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:58.209815 29188 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:58.168967 29217 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: [5.4-8.5]
I20260709 13:04:58.222831 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:58.271513 29220 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:58.225209 29069 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:58.162746 32179 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.176555 29142 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:58.275874 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:58.263254 29071 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
W20260709 13:04:58.277591 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 82030.9us
I20260709 13:04:58.278599 29113 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:04:58.279397 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 69650.6us
I20260709 13:04:58.282065 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:58.282862 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 80476.4us
I20260709 13:04:58.284214 32185 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:58.284799 32185 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:58.285455 32185 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:58.285101 32147 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:58.288149 32043 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:58.288700 32043 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.289183 32043 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:58.278772 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:58.291879 32148 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:58.292579 32066 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:58.293207 32066 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.293676 32066 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:58.284789 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:58.240257 29907 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:04:58.242393 29216 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:58.246197 29146 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
W20260709 13:04:58.177884 32175 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:04:58.217545 32176 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:58.220463 32076 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:58.301697 32076 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.302213 32076 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:58.150393 29221 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:58.273342 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:58.309535 29039 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:58.229076 32055 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.311501 32055 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:58.274580 32173 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.596s	user 0.010s	sys 0.000s
I20260709 13:04:58.312358 32076 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.274447 32136 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.314356 32136 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.301311 32185 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.316454 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1894) took 1080 ms. Trace:
I20260709 13:04:58.317016 29072 rpcz_store.cc:276] 0709 13:04:57.236007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.236776 (+   769us) service_pool.cc:224] Handling call
0709 13:04:57.264783 (+ 28007us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:57.275191 (+ 10408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:57.275234 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:57.290455 (+ 15221us) log.cc:844] Serialized 26 byte log entry
0709 13:04:57.418248 (+127793us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:57.418293 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.418363 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:58.273809 (+855446us) spinlock_profiling.cc:243] Waited 52.6 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:58.316028 (+ 42219us) raft_consensus.cc:1685] finished
0709 13:04:58.316060 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.316346 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22967,"raft.queue_time_us":11885,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":11673,"spinlock_wait_cycles":120964992,"thread_start_us":78467,"threads_started":1,"wal-append.queue_time_us":354817}
I20260709 13:04:58.320132 32118 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:58.319586 32059 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:58.326421 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:58.328158 32037 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:58.335465 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:58.207182 31621 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.233559 32091 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:58.344868 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:58.108269 32157 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.261816 32172 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:58.261612 31902 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:58.342118 31924 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:58.341086 32170 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:58.568691 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 283966us
I20260709 13:04:58.355248 31995 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:58.574498 29906 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:58.581750 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:58.588375 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 1788) took 1308 ms. Trace:
I20260709 13:04:58.588908 29140 rpcz_store.cc:276] 0709 13:04:57.280422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.293444 (+ 13022us) service_pool.cc:224] Handling call
0709 13:04:58.588165 (+1294721us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b540027d8d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:58.588308 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1727094016}
I20260709 13:04:58.537406 31618 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:58.281320 32068 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.608996 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:58.616592 29222 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:58.572554 31902 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:58.573840 32167 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:58.622768 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:58.596081 31729 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:58.621076 29218 raft_consensus.cc:1217] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Deduplicated request from leader. Original: 2.2->[3.3-8.5]   Dedup: 3.3->[6.4-8.5]
I20260709 13:04:58.341446 32179 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.629457 29220 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:04:58.643512 29215 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:58.601403 32148 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:58.615743 31753 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.573753 32129 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:58.635166 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 300151us
I20260709 13:04:58.642992 29218 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: [3.3-8.5]
I20260709 13:04:58.631767 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:58.648885 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.612665 32089 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:04:58.668864 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1901) took 1220 ms. Trace:
I20260709 13:04:58.664296 29039 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:58.630525 32171 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.630545 31902 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:58.672979 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:58.676692 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:58.646616 32073 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:58.683246 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:58.683415 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:58.625363 32043 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.662060 31729 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:58.695986 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:58.642333 31715 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:04:58.679047 31902 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:58.686136 32182 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:58.612692 32091 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:58.755628 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 123945us
I20260709 13:04:58.663015 32148 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.734606 29218 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
W20260709 13:04:58.759490 32154 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 13:04:58.782603 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:58.677290 31753 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:58.777532 31607 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.807s	user 0.012s	sys 0.000s
I20260709 13:04:58.805008 31607 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.807282 32069 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:58.807435 31607 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:58.808174 32202 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:58.808799 32202 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.811127 32202 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:58.791003 32182 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:58.812376 32182 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:58.812804 32182 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:58.814539 32164 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.675s	user 0.001s	sys 0.001s
W20260709 13:04:58.817408 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.817967 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.818590 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.819561 32046 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.875s	user 0.005s	sys 0.005s
W20260709 13:04:58.817432 32065 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.581s	user 0.005s	sys 0.013s
I20260709 13:04:58.820557 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1893) took 1586 ms. Trace:
I20260709 13:04:58.820699 32065 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 4. Current term is 5.
W20260709 13:04:58.820962 32189 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:04:58.821461 32065 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:58.822595 32193 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:04:58.821146 29064 rpcz_store.cc:276] 0709 13:04:57.234686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.235037 (+   351us) service_pool.cc:224] Handling call
0709 13:04:57.262138 (+ 27101us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:57.274388 (+ 12250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:57.274436 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:57.277810 (+  3374us) log.cc:844] Serialized 26 byte log entry
0709 13:04:57.500287 (+222477us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:57.500328 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.500392 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:58.813861 (+1313469us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:58.820157 (+  6296us) raft_consensus.cc:1685] finished
0709 13:04:58.820187 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.820462 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14009,"raft.queue_time_us":12807,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":7862,"spinlock_wait_cycles":25454848,"thread_start_us":222118,"threads_started":1,"wal-append.queue_time_us":802832}
I20260709 13:04:58.824221 32065 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:58.839277 31830 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.655s	user 0.007s	sys 0.000s
W20260709 13:04:58.825829 32191 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.227s	user 0.000s	sys 0.000s
I20260709 13:04:58.682009 32171 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.828055 32046 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.783731 31944 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:58.838820 29143 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:04:58.841746 32043 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:04:58.844954 31858 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.000s
I20260709 13:04:58.845814 31753 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:58.776006 31621 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:58.825294 32186 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
W20260709 13:04:58.819747 32040 consensus_meta.cc:287] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.756s	user 0.005s	sys 0.002s
I20260709 13:04:58.787267 32197 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:58.857966 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:58.826725 31842 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:04:58.865312 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 287652us
W20260709 13:04:58.859275 32009 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.926s	user 0.005s	sys 0.007s
I20260709 13:04:58.834863 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1876) took 2288 ms. Trace:
I20260709 13:04:58.872195 29145 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:58.876564 29070 rpcz_store.cc:276] 0709 13:04:56.546885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.781417 (+234532us) service_pool.cc:224] Handling call
0709 13:04:57.515017 (+733600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.965645 (+450628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.833573 (+867928us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54001d8b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:58.834603 (+  1030us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b54001d8b50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:58.834783 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548393088}
I20260709 13:04:58.875942 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:58.880604 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:58.881381 29218 raft_consensus.cc:2410] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:04:58.890471 29219 consensus_meta.cc:287] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.542s	user 0.011s	sys 0.000s
I20260709 13:04:58.890940 29219 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:58.791503 32154 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.856667 31830 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.913360 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:58.920737 29139 raft_consensus.cc:2376] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7. Re-sending same reply.
W20260709 13:04:58.922051 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 238865us
I20260709 13:04:58.927094 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:58.927381 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:58.928175 29139 raft_consensus.cc:2368] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9.
I20260709 13:04:58.880192 32009 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:58.882431 32079 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:58.947489 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:58.862612 31858 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.966085 29904 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:04:58.967125 31830 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:58.968940 32066 consensus_meta.cc:287] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.675s	user 0.012s	sys 0.000s
I20260709 13:04:58.927868 29142 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:04:58.883302 31842 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:58.969542 32197 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.969485 32066 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.970136 32197 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.970515 31842 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.870539 31729 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.882441 32068 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:58.926858 32184 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.485s	user 0.001s	sys 0.000s
I20260709 13:04:58.881312 32202 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:58.973212 32182 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.973376 32202 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:58.975499 32202 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:58.959785 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:58.977876 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:58.963709 29144 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:04:58.968885 32154 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:58.996140 29218 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:59.000155 32197 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.001195 32197 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:59.003427 32197 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:59.007474 29147 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:58.967128 31858 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.015724 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1829) took 1723 ms. Trace:
I20260709 13:04:59.019872 29065 rpcz_store.cc:276] 0709 13:04:57.291721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.035908 (+744187us) service_pool.cc:224] Handling call
0709 13:04:58.064743 (+ 28835us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:58.071450 (+  6707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.071494 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:58.072058 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:04:58.214502 (+142444us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:58.214559 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.214651 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.015223 (+800572us) raft_consensus.cc:1685] finished
0709 13:04:59.015251 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.015436 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3465,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":8391,"thread_start_us":107223,"threads_started":1,"wal-append.queue_time_us":189649}
I20260709 13:04:59.017081 29218 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:59.017791 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.022841 29218 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:59.025957 29188 leader_election.cc:400] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9.
I20260709 13:04:59.026481 29188 leader_election.cc:403] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:04:59.027307 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.005472 32179 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:59.029611 29214 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:59.056587 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.013238 32079 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:59.063848 29140 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:59.064424 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.066146 29147 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.069491 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.070732 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.071410 29139 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:04:59.072283 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.021487 32197 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:59.074541 29065 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:59.075404 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1833) took 1295 ms. Trace:
I20260709 13:04:59.033136 29218 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.077555 29218 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:59.075831 29065 rpcz_store.cc:276] 0709 13:04:57.780379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.027216 (+1246837us) service_pool.cc:224] Handling call
0709 13:04:59.074296 (+ 47080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.075101 (+   805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.075332 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:59.028071 31967 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:59.089440 29147 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:59.031529 32065 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.091668 29139 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
W20260709 13:04:59.093881 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 116075us
I20260709 13:04:59.094938 29065 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:59.099221 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1885) took 2179 ms. Trace:
I20260709 13:04:59.079309 32179 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:59.099296 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1903) took 1207 ms. Trace:
I20260709 13:04:59.102154 29065 rpcz_store.cc:276] 0709 13:04:57.891472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.081016 (+1189544us) service_pool.cc:224] Handling call
0709 13:04:59.094710 (+ 13694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.095462 (+   752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.099238 (+  3776us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:59.106256 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1830) took 1688 ms. Trace:
I20260709 13:04:59.092039 32046 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.098349 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:59.099797 29067 rpcz_store.cc:276] 0709 13:04:56.920183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.040367 (+120184us) service_pool.cc:224] Handling call
0709 13:04:57.040687 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:57.194007 (+153320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.098976 (+1904969us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54003edad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:59.099160 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312330496}
I20260709 13:04:59.115307 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:59.116385 32209 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:59.118064 31604 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.889s	user 0.008s	sys 0.005s
I20260709 13:04:59.118407 32209 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:59.118961 32209 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:59.118674 31604 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.119968 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1902) took 1615 ms. Trace:
I20260709 13:04:59.121112 29220 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:04:59.120368 29073 rpcz_store.cc:276] 0709 13:04:57.504850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.586635 (+1081785us) service_pool.cc:224] Handling call
0709 13:04:58.587052 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:58.599316 (+ 12264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.599363 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:58.612712 (+ 13349us) log.cc:844] Serialized 26 byte log entry
0709 13:04:58.870331 (+257619us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:58.870396 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.870470 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.119655 (+249185us) raft_consensus.cc:1685] finished
0709 13:04:59.119680 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.119863 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11440,"raft.queue_time_us":12244,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":17142,"thread_start_us":199389,"threads_started":1,"wal-append.queue_time_us":374164}
I20260709 13:04:59.122040 32221 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:59.122713 29073 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:59.122817 32221 consensus_queue.cc:1243] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:59.123119 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2040) took 1832 ms. Trace:
I20260709 13:04:59.123503 29219 rpcz_store.cc:276] 0709 13:04:57.290761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.840484 (+549723us) service_pool.cc:224] Handling call
0709 13:04:59.122941 (+1282457us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b540023c654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:59.123063 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093869824}
I20260709 13:04:59.105309 32197 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.132257 32221 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:59.132781 32221 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:59.107558 29069 rpcz_store.cc:276] 0709 13:04:57.417648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.343996 (+926348us) service_pool.cc:224] Handling call
0709 13:04:58.344338 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.344546 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:04:58.729741 (+385195us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.105708 (+375967us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:59.105760 (+    52us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:59.105802 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.106079 (+   277us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.106198 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23066,"prepare.queue_time_us":243132,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":33567,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":12872,"thread_start_us":539939,"threads_started":3,"wal-append.queue_time_us":341372}
I20260709 13:04:59.135763 29067 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: [3.3-8.5]
I20260709 13:04:59.138141 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.140291 29147 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:04:59.130932 32172 consensus_meta.cc:287] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.582s	user 0.002s	sys 0.009s
I20260709 13:04:59.151621 29218 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:04:59.130502 32205 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:04:59.156333 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.157145 29139 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
W20260709 13:04:59.137349 32194 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
I20260709 13:04:59.160924 31842 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.162181 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1900) took 1855 ms. Trace:
I20260709 13:04:59.162572 31902 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.172483 29071 rpcz_store.cc:276] 0709 13:04:57.306738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.342468 (+1035730us) service_pool.cc:224] Handling call
0709 13:04:58.342891 (+   423us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:58.343344 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.343381 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:58.356750 (+ 13369us) log.cc:844] Serialized 52 byte log entry
0709 13:04:58.774807 (+418057us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:58.857762 (+ 82955us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.024900 (+167138us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.025287 (+   387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.025395 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.161884 (+136489us) raft_consensus.cc:1685] finished
0709 13:04:59.161907 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.162114 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11887,"prepare.queue_time_us":152579,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":20427,"raft.queue_time_us":139668,"raft.run_cpu_time_us":15128,"raft.run_wall_time_us":258928,"thread_start_us":295848,"threads_started":2,"wal-append.queue_time_us":250515}
I20260709 13:04:59.138943 32171 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.165072 32079 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.166203 32221 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.168692 29139 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:04:59.164016 32172 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.195730 31842 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:59.202335 32156 consensus_meta.cc:287] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.125s	user 0.014s	sys 0.000s
I20260709 13:04:59.209376 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 546) took 1005 ms. Trace:
I20260709 13:04:59.215293 29910 rpcz_store.cc:276] 0709 13:04:58.204217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.204477 (+   260us) service_pool.cc:224] Handling call
0709 13:04:58.204881 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:58.216711 (+ 11830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.216742 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:58.217246 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:04:58.648572 (+431326us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:58.648631 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.648731 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.209133 (+560402us) raft_consensus.cc:1685] finished
0709 13:04:59.209152 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.209307 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":158,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":20552,"thread_start_us":387074,"threads_started":1,"wal-append.queue_time_us":557822}
W20260709 13:04:59.217716 29039 connection.cc:256] RPC call timeout handler was delayed by -1.2340891249996275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.262s for a response
I20260709 13:04:59.212613 32172 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.213346 29148 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:59.213840 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.227098 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.229247 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.230257 29147 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 4.
I20260709 13:04:59.231806 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.234988 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1906) took 1062 ms. Trace:
I20260709 13:04:59.235414 29065 rpcz_store.cc:276] 0709 13:04:58.172312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.104648 (+932336us) service_pool.cc:224] Handling call
0709 13:04:59.105458 (+   810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:59.112178 (+  6720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:59.112212 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:59.117126 (+  4914us) log.cc:844] Serialized 26 byte log entry
0709 13:04:59.233069 (+115943us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:59.233131 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.233212 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.234792 (+  1580us) raft_consensus.cc:1685] finished
0709 13:04:59.234810 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.234933 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3320,"raft.queue_time_us":7534,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":22017,"thread_start_us":104759,"threads_started":1,"wal-append.queue_time_us":105158}
I20260709 13:04:59.241243 32136 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.247714 29140 pending_rounds.cc:85] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 5
I20260709 13:04:59.248508 29140 pending_rounds.cc:107] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:04:59.215060 31902 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.218988 32046 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:59.249907 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.248790 29140 raft_consensus.cc:2889] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:59.250955 29140 pending_rounds.cc:107] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:04:59.251204 29140 raft_consensus.cc:2889] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:59.226141 32156 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:59.231392 32091 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:59.231602 32148 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.283592 29213 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:04:59.247767 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1832) took 1467 ms. Trace:
I20260709 13:04:59.285326 29070 rpcz_store.cc:276] 0709 13:04:57.780097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.877598 (+1097501us) service_pool.cc:224] Handling call
0709 13:04:58.892700 (+ 15102us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:59.072638 (+179938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:59.072685 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:59.073246 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:04:59.240157 (+166911us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:59.240191 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.240265 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.247540 (+  7275us) raft_consensus.cc:1685] finished
0709 13:04:59.247573 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.247706 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28718,"raft.queue_time_us":107871,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":265117,"threads_started":2,"wal-append.queue_time_us":151287}
I20260709 13:04:59.291496 31842 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:59.293888 31618 consensus_meta.cc:287] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.670s	user 0.002s	sys 0.010s
I20260709 13:04:59.307823 29141 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
W20260709 13:04:59.311995 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 93507.4us
I20260709 13:04:59.262147 32046 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.324715 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1910) took 1115 ms. Trace:
I20260709 13:04:59.326030 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.325165 29073 rpcz_store.cc:276] 0709 13:04:58.208626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.123628 (+915002us) service_pool.cc:224] Handling call
0709 13:04:59.123908 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.126429 (+  2521us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:59.127003 (+   574us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.323393 (+196390us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:59.323447 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:59.323495 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.323560 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.323682 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":165,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":433,"thread_start_us":195424,"threads_started":1}
I20260709 13:04:59.283068 32091 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.323843 31902 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:59.334064 31842 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:59.337121 31995 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.018s	sys 0.030s
I20260709 13:04:59.338311 32209 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:04:59.369382 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.370314 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:59.372210 29910 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:59.373518 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2040) took 1208 ms. Trace:
I20260709 13:04:59.374035 29141 rpcz_store.cc:276] 0709 13:04:58.164687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.164927 (+   240us) service_pool.cc:224] Handling call
0709 13:04:59.373268 (+1208341us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003cc3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:59.373368 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2523297664}
I20260709 13:04:59.375810 32124 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.376430 32124 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.378530 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:59.379392 31621 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.369256 32046 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.380321 29903 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:59.384595 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:59.388963 29909 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.389082 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 542) took 1520 ms. Trace:
I20260709 13:04:59.383181 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:59.390733 29908 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:04:59.390887 29186 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:59.378660 32124 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.380676 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2031) took 2094 ms. Trace:
W20260709 13:04:59.358577 31995 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.022s	sys 0.030s
I20260709 13:04:59.392318 29217 rpcz_store.cc:276] 0709 13:04:57.286640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.286774 (+   134us) service_pool.cc:224] Handling call
0709 13:04:58.105736 (+818962us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:58.271119 (+165383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.380478 (+1109359us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b54003cd2d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:59.380618 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1852722432}
W20260709 13:04:59.382922 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.389637 29903 rpcz_store.cc:276] 0709 13:04:57.868799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.868921 (+   122us) service_pool.cc:224] Handling call
0709 13:04:59.379964 (+1511043us) spinlock_profiling.cc:243] Waited 7.72 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:59.380148 (+   184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.388771 (+  8623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.389010 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17757056}
I20260709 13:04:59.385731 32209 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.385916 32069 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:59.385671 32156 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:59.406447 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:59.354818 31902 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.415024 32204 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:59.403210 29141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:04:59.411468 32080 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:59.416463 29141 raft_consensus.cc:606] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:59.416790 32080 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.417658 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.405658 32069 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.392612 32067 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:59.395661 31995 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:59.419770 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:59.404888 32179 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9.
I20260709 13:04:59.362744 31842 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.409798 31944 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.426391 29909 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:04:59.379895 31621 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.415593 32204 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.429239 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:59.428850 29141 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:04:59.431250 32065 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:59.432885 32065 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.435022 31618 raft_consensus.cc:1064] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
W20260709 13:04:59.435523 31618 raft_consensus.cc:1068] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Unable to promote non-voter 7e78ffbcc98242068a874a5d9e341477: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:04:59.437139 31995 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.439882 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.441174 29141 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:59.425191 29139 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:59.421298 32069 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.409919 32046 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:04:59.423997 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:59.449172 29902 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:04:59.449244 32046 raft_consensus.cc:3055] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:59.449765 32046 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:59.450696 32046 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.428610 32040 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:59.411883 32154 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:04:59.444224 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.429170 31944 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.455083 32046 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:59.425377 32156 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:04:59.459048 31944 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:59.430166 32124 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:04:59.464668 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.471956 29139 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:59.446020 32068 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.472528 32068 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.474797 32068 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:59.477671 29112 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:59.445350 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.479259 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.453135 32040 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.459049 32156 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:59.488813 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.484186 32040 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:59.491627 29141 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.490272 32156 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:04:59.496959 32156 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.498603 32156 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.504664 31621 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.505342 32204 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:59.490870 32148 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:59.506392 32148 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:04:59.507700 32148 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.492674 32124 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:59.516530 32124 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.481490 32080 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.502172 31902 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.540297 32124 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.551559 29139 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:04:59.554013 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:59.564164 32215 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:04:59.565208 32235 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.182s	user 0.002s	sys 0.000s
W20260709 13:04:59.567826 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.571436 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.571892 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.572237 29139 raft_consensus.cc:2376] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10. Re-sending same reply.
I20260709 13:04:59.573009 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.573277 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.573825 29139 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
W20260709 13:04:59.574959 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.577560 29217 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:04:59.578687 29142 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:59.579041 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.579839 29217 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:04:59.544493 31944 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:59.544904 32125 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:59.567766 32220 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.367s	user 0.000s	sys 0.000s
I20260709 13:04:59.581981 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.583387 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.583887 29216 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:59.584712 29142 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:59.585570 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.585847 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:59.585593 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:04:59.621553 29038 leader_election.cc:400] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:04:59.621934 29038 leader_election.cc:403] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:59.622676 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.624552 29217 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:04:59.623770 29073 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:59.625541 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:59.626907 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:59.626828 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:59.627445 32055 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.631651 31842 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.606559 29141 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:04:59.548290 32035 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:04:59.553028 32172 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:04:59.575707 32136 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:04:59.659327 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 66875us
I20260709 13:04:59.585626 32225 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:59.664917 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:59.669628 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:59.671020 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:59.575515 32209 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:04:59.621021 32197 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:59.676450 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.673249 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:59.678923 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.642736 31618 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.641307 32037 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.663887 32059 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:04:59.678496 32136 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.686691 32197 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.690478 32059 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:59.692001 32197 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.692096 32136 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.692544 32197 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:59.692891 32197 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:59.693979 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.694650 29142 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:04:59.697620 32040 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.693707 32197 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.700259 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1831) took 1920 ms. Trace:
I20260709 13:04:59.682137 29073 raft_consensus.cc:2410] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:04:59.680519 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 57588.2us
I20260709 13:04:59.704241 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.704973 29217 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:59.621842 31944 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.681425 32221 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:59.706552 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.677618 32219 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:59.710242 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.647121 32080 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:04:59.681596 31902 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:59.718284 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:59.719059 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.700802 29064 rpcz_store.cc:276] 0709 13:04:57.779734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.823634 (+1043900us) service_pool.cc:224] Handling call
0709 13:04:58.847564 (+ 23930us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:58.848041 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.848079 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:58.866137 (+ 18058us) log.cc:844] Serialized 52 byte log entry
0709 13:04:59.030498 (+164361us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:59.117236 (+ 86738us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.117494 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.117588 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.562801 (+445213us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b18004b8f20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f783ba36328 00007f783ba2d901 00007f783ba2d8a7 00007f783bb25288 00007f783bb24f37 00007f783bb3955c 00007f783bb39508
0709 13:04:59.567178 (+  4377us) spinlock_profiling.cc:243] Waited 7.07 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:04:59.700046 (+132868us) raft_consensus.cc:1685] finished
0709 13:04:59.700067 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.700195 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96184,"prepare.queue_time_us":59420,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":54976,"raft.run_cpu_time_us":2211,"raft.run_wall_time_us":447668,"spinlock_wait_cycles":24277760,"thread_start_us":168638,"threads_started":2,"wal-append.queue_time_us":253288}
I20260709 13:04:59.677565 32055 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:04:59.721004 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.721887 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.681300 32171 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:04:59.699873 32040 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.724821 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:59.659111 32210 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:04:59.688498 32118 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:59.719106 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:59.728781 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:59.727224 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:59.744489 29909 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:59.743958 29146 raft_consensus.cc:2393] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:04:59.732753 29874 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:59.687065 31618 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.704154 32215 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:59.686489 32209 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.763664 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.764794 31944 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:04:59.765432 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46392.3us
I20260709 13:04:59.766058 32215 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.768939 32172 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:59.769658 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:04:59.773303 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 67900.1us
I20260709 13:04:59.776427 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:04:59.760110 29909 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:04:59.716156 32221 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:04:59.779685 32221 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:59.702636 32037 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.681002 32225 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:04:59.761265 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 82370.8us
I20260709 13:04:59.753505 31902 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:59.752821 32118 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.782325 32118 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.783063 31902 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.786305 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:59.788347 32219 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:59.788501 32221 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:04:59.754196 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:59.766074 31621 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:04:59.791262 32219 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:59.796630 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:59.797857 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:04:59.749238 32055 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.834933 29146 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:59.769730 32076 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.838011 29038 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:04:59.748962 32171 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.852814 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.804702 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 545) took 1596 ms. Trace:
I20260709 13:04:59.794822 32157 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:04:59.868301 32185 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.869208 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 419) took 1694 ms. Trace:
I20260709 13:04:59.870867 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2045) took 1703 ms. Trace:
I20260709 13:04:59.870514 32069 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:58.673655 29066 rpcz_store.cc:276] 0709 13:04:57.449020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.338195 (+889175us) service_pool.cc:224] Handling call
0709 13:04:58.338554 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.338779 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:04:58.339114 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.656160 (+317046us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.668491 (+ 12331us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:58.668551 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:58.668597 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.668669 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.668806 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12065,"prepare.queue_time_us":2666,"prepare.run_cpu_time_us":5659,"prepare.run_wall_time_us":29265,"raft.queue_time_us":19414,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":6840,"spinlock_wait_cycles":40864000,"thread_start_us":315572,"threads_started":1}
I20260709 13:04:59.840199 32258 raft_consensus.cc:2764] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:04:59.873184 32055 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.878505 32171 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.864260 32124 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:59.884600 29143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2039) took 1728 ms (client timeout 1803 ms). Trace:
I20260709 13:04:59.791365 32118 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:04:59.887847 31604 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.911427 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.911875 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:04:59.912118 31621 consensus_queue.cc:1243] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:04:59.912659 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.914086 29216 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:04:59.891937 32190 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 1.121s	user 0.000s	sys 0.002s
I20260709 13:04:59.892885 32225 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.899803 29146 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:04:59.902004 29902 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:43668
I20260709 13:04:59.919165 29902 raft_consensus.cc:606] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:04:59.920329 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:04:59.920469 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 82545.8us
I20260709 13:04:59.904534 29220 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:04:59.922315 32258 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:59.905047 29145 rpcz_store.cc:276] 0709 13:04:58.175079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.228331 (+ 53252us) service_pool.cc:224] Handling call
0709 13:04:58.257147 (+ 28816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.876743 (+619596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.868960 (+992217us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b5400041354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:59.869140 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1414001024}
I20260709 13:04:59.905167 29148 rpcz_store.cc:276] 0709 13:04:58.167135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.167261 (+   126us) service_pool.cc:224] Handling call
0709 13:04:59.870693 (+1703432us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540029ccd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:59.870803 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2305326336}
I20260709 13:04:59.925109 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:04:59.908336 29143 rpcz_store.cc:269] 0709 13:04:58.156548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.157850 (+  1302us) service_pool.cc:224] Handling call
0709 13:04:59.884350 (+1726500us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b5400190ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:59.884516 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1377505664}
W20260709 13:04:59.927960 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.929770 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:04:59.931211 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:04:59.794976 32162 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:59.907554 32065 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:04:59.932752 32162 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:04:59.907085 32124 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.923833 32258 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:59.933835 32124 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.934260 32136 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:59.933990 32162 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.934757 32136 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:04:59.935947 32136 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:04:59.937141 32091 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:04:59.937327 32136 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.938505 32182 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:04:59.939971 32187 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.295s	user 0.001s	sys 0.001s
I20260709 13:04:59.908852 32069 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.908679 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:04:59.944491 29221 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:04:59.946132 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.937150 29215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2043) took 2645 ms (client timeout 1851 ms). Trace:
I20260709 13:04:59.947624 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:04:59.948367 29143 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:04:59.951112 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:04:59.951216 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:04:59.947536 29215 rpcz_store.cc:269] 0709 13:04:57.291666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.314101 (+ 22435us) service_pool.cc:224] Handling call
0709 13:04:59.936881 (+2622780us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003ccdd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:04:59.937038 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2992083328}
I20260709 13:04:59.951684 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:59.951745 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:04:59.959506 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 547) took 1710 ms. Trace:
I20260709 13:04:59.969252 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:04:59.960141 29905 rpcz_store.cc:276] 0709 13:04:58.248801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.249134 (+   333us) service_pool.cc:224] Handling call
0709 13:04:59.958412 (+1709278us) spinlock_profiling.cc:243] Waited 9.17 ms on lock 0x7b58001a1098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:04:59.958699 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.958990 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:59.959032 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:59.959095 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:59.959163 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.959245 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.959422 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21083520}
I20260709 13:05:00.012763 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:00.010496 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:00.013599 29143 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
W20260709 13:05:00.017724 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 66064.5us
I20260709 13:04:59.910379 32118 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.033131 29222 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:00.028934 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:00.017742 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 66585.8us
I20260709 13:05:00.029202 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:04:59.909782 31976 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.957472 32065 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:59.969175 32171 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:04:59.957901 32039 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:59.909513 32066 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:04:59.968443 32182 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:00.036899 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:00.056247 29907 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:00.013651 32046 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:00.037564 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:00.056845 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 152706us
I20260709 13:05:00.057771 29144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:05:00.040661 29911 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:05:00.058394 29144 raft_consensus.cc:606] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:00.050500 29903 raft_consensus.cc:2368] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:05:00.053409 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:00.059779 29905 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:04:59.904407 29904 rpcz_store.cc:276] 0709 13:04:58.199722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.212960 (+ 13238us) service_pool.cc:224] Handling call
0709 13:04:58.266176 (+ 53216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:58.966593 (+700417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.763785 (+797192us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b5400767fd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:04:59.796431 (+ 32646us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1608929024}
I20260709 13:04:59.968696 32225 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 13:05:00.040328 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:00.076889 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:00.029101 32263 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:00.035727 32260 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:00.053663 32039 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.106317 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:00.057869 31976 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:00.062783 32182 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:00.115777 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 105397us
I20260709 13:05:00.065689 32080 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:00.088901 31944 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:00.100660 32065 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:00.115321 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:00.065291 32157 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.116739 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:00.100502 31842 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:00.069667 32040 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:05:00.116148 32172 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:00.117837 32182 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.072582 32166 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:05:00.117357 32039 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:00.121632 31976 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:00.118599 32253 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:00.118355 32080 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:00.137521 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 216018us
I20260709 13:05:00.119174 32215 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:05:00.117924 32046 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:05:00.146593 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:00.127528 32035 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:00.133433 32040 consensus_queue.cc:1243] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:00.144678 32118 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:00.148051 32263 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:05:00.167810 32274 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:00.168478 32274 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:00.151909 32035 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.169106 32035 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:00.169456 32274 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.133255 31842 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:00.128775 32157 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:00.190549 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 86788.8us
I20260709 13:05:00.163089 32225 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:00.225350 32253 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:00.225391 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 114186us
I20260709 13:05:00.244206 32225 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:00.266341 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 112074us
I20260709 13:05:00.270576 29038 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:00.272147 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:00.192523 32065 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:00.282306 29038 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:00.283576 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 166939us
I20260709 13:05:00.235865 32118 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:05:00.289800 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:00.304426 32118 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:00.305702 32118 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:00.306967 32118 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:00.307365 32118 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:00.254673 32253 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:00.317873 32253 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.296721 32270 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:00.319455 32270 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:00.319903 32270 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.325314 31842 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.242756 31976 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:00.291374 29038 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:00.307742 32118 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.339531 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:00.363386 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.364348 29073 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:05:00.365370 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.366496 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.341320 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.297509 32272 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:00.346082 32287 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:00.346288 32076 raft_consensus.cc:993] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower a936878f59c840a6bd176a1d2d7347b1 to start an election
I20260709 13:05:00.346832 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.349334 32157 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.350162 32285 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:00.356397 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:00.358161 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.359616 29067 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:00.368073 32209 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:00.368525 32287 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:00.369220 29070 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:00.368785 32076 raft_consensus.cc:1081] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Signalling peer a936878f59c840a6bd176a1d2d7347b1 to start an election
W20260709 13:05:00.404287 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 114597us
I20260709 13:05:00.408196 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:00.408602 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.409138 29144 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.409515 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:00.408324 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:00.412774 29142 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:05:00.412856 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:00.419018 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:00.413074 32272 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.434289 29037 leader_election.cc:400] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:05:00.413416 32285 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:00.402253 32035 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:00.440394 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.432199 32039 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:00.414909 31976 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:05:00.449100 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.400033 32225 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.444372 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:00.444036 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:00.445060 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 72901.2us
W20260709 13:05:00.447237 29111 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.757s (SENT) [suppressed 24 similar messages]
I20260709 13:05:00.451723 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:00.454715 29070 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:00.439177 32124 consensus_meta.cc:287] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.000s
I20260709 13:05:00.457360 29072 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:00.439136 32136 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:00.462713 29066 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:00.465358 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:00.465633 32284 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:05:00.467137 32197 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:00.467334 32035 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.423298 32287 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.468998 32287 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.439286 32274 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.473538 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.456207 32065 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:00.437669 32272 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.478410 32293 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:00.515370 32274 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.525722 32293 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:00.526116 32274 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.442232 32297 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:00.527837 32297 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.530123 32297 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.532064 29065 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.533413 32272 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.440280 32285 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.537900 32285 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.538450 32039 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:00.480899 32291 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:05:00.540410 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2056) took 1315 ms. Trace:
I20260709 13:05:00.540795 29147 rpcz_store.cc:276] 0709 13:04:59.224842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.231131 (+  6289us) service_pool.cc:224] Handling call
0709 13:04:59.251043 (+ 19912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.336345 (+ 85302us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:59.338505 (+  2160us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.409097 (+ 70592us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:59.409149 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:59.409191 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.540162 (+1130971us) spinlock_profiling.cc:243] Waited 85.1 ms on lock 0x7b5400290254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:00.540210 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:00.540349 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8178,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":5382,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":4151,"spinlock_wait_cycles":195645312,"thread_start_us":66299,"threads_started":1,"wal-append.queue_time_us":75020,"wal-append.run_cpu_time_us":1030,"wal-append.run_wall_time_us":1002339}
I20260709 13:05:00.482715 32182 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:00.544029 32065 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.545517 31618 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.545332 32118 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.483656 29073 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:00.463088 32124 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.546973 32065 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.464175 32136 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:00.547657 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:00.548147 32124 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.548261 32118 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.548848 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:00.549993 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.551535 29073 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:00.552188 32270 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.553117 32080 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.555254 32270 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.492751 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.497057 29146 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:00.557641 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.557811 29146 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:00.497447 29144 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:00.559850 29071 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:00.560319 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:00.557523 29064 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:00.566349 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:00.558790 29146 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:00.571520 29146 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:00.571684 29067 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:00.573042 29067 raft_consensus.cc:1217] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[5.4-9.6]   Dedup: 5.4->[7.5-9.6]
I20260709 13:05:00.573457 29067 pending_rounds.cc:85] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 4
I20260709 13:05:00.573673 29067 pending_rounds.cc:107] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:05:00.573889 29067 raft_consensus.cc:2889] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:00.579375 29146 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:05:00.579967 29146 pending_rounds.cc:85] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 7
I20260709 13:05:00.582947 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:00.582959 29065 raft_consensus.cc:2410] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:00.547824 32136 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:00.583879 32124 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.584607 29146 pending_rounds.cc:107] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:05:00.590729 29146 raft_consensus.cc:2889] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:00.585464 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:00.526192 32293 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:00.585525 29073 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:00.588438 32182 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:00.592075 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:00.595420 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:00.596760 29037 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:00.597304 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:00.598280 31618 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:00.608299 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:00.611761 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:00.614813 29871 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:00.618060 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:00.618925 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:00.619182 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:00.620100 29144 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:00.598436 32265 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:00.616762 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:00.599164 32297 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:00.621744 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:00.622231 29871 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:00.623291 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:00.652349 29144 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:05:00.654165 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:00.660133 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:00.661932 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:00.603242 31621 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:00.626933 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:00.662915 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:00.613734 32185 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:00.633337 32293 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.616060 32035 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:00.642863 32157 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:00.624459 32162 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:00.662364 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:00.662622 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:00.663556 29142 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:00.592012 29070 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:00.671484 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:00.665088 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:00.602448 32272 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:05:00.615651 31729 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:05:00.720111 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:00.726851 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.662853 32059 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:05:00.722139 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 57125.6us
W20260709 13:05:00.732206 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.625047 32254 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:05:00.658696 32253 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:00.743963 29073 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:00.673960 32172 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:00.734157 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:00.736644 32059 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:00.738497 32297 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:00.733836 32225 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:00.741560 32254 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:00.742532 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:00.757102 29071 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:00.743719 29213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 414) took 2610 ms (client timeout 1797 ms). Trace:
I20260709 13:05:00.757938 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1925) took 2162 ms. Trace:
I20260709 13:05:00.759240 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.759990 29071 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
W20260709 13:05:00.758124 29213 rpcz_store.cc:269] 0709 13:04:58.133575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.172420 (+ 38845us) service_pool.cc:224] Handling call
0709 13:05:00.743544 (+2571124us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000bde54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:00.743651 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2450184960}
I20260709 13:05:00.760913 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:00.758353 29214 rpcz_store.cc:276] 0709 13:04:58.595647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.056508 (+460861us) service_pool.cc:224] Handling call
0709 13:04:59.082356 (+ 25848us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:59.082704 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:59.082738 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:59.090439 (+  7701us) log.cc:844] Serialized 26 byte log entry
0709 13:04:59.230568 (+140129us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:59.230608 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.230689 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:00.661594 (+1430905us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b24001fa378. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78338a69c4 00007f78338a4acb 00007f7833a5fdc9 00007f7833a5fa55 00007f7833a5efb0 00007f78366d5488 00007f78366d5be8 00007f78366ad398 00007f78366a876a 00007f78366a7bb6
0709 13:05:00.750752 (+ 89158us) raft_consensus.cc:1685] finished
0709 13:05:00.750773 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:00.757873 (+  7100us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5705,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":3286,"spinlock_wait_cycles":15586560,"thread_start_us":123606,"threads_started":1,"wal-append.queue_time_us":150653}
I20260709 13:05:00.762598 29071 raft_consensus.cc:2376] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9. Re-sending same reply.
I20260709 13:05:00.763461 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:00.624884 32280 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 13:05:00.746819 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:00.732606 32293 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.724022 32035 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:00.771237 32035 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:00.771668 32035 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:00.752929 32182 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:00.721859 32303 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:05:00.764344 29071 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:00.669026 31842 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:05:00.667692 32215 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:00.747888 32172 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:00.765537 32059 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.768587 32280 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:00.756289 32225 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.753314 32297 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:00.793231 32280 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:00.794391 32280 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:00.796160 32225 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.797047 32037 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:00.800987 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1933) took 1216 ms. Trace:
I20260709 13:05:00.801496 29219 rpcz_store.cc:276] 0709 13:04:59.584425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.584540 (+   115us) service_pool.cc:224] Handling call
0709 13:04:59.584777 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.584967 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:59.640124 (+ 55157us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.731128 (+ 91004us) log.cc:844] Serialized 14 byte log entry
0709 13:05:00.013140 (+282012us) spinlock_profiling.cc:243] Waited 8.8 ms on lock 0x7b18002b5820. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1
0709 13:05:00.800138 (+786998us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b18002b5820. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:05:00.800512 (+   374us) log.cc:844] Serialized 14 byte log entry
0709 13:05:00.800664 (+   152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:00.800704 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:00.800750 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:00.800815 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:00.800927 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12927,"prepare.queue_time_us":59492,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":4587,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":8200,"spinlock_wait_cycles":50094592,"thread_start_us":90434,"threads_started":1,"wal-append.queue_time_us":166594}
I20260709 13:05:00.782538 29071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:47960
I20260709 13:05:00.805804 29071 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:00.806186 29071 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.718231 32162 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:00.782210 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 120369us
I20260709 13:05:00.756661 32254 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:00.786226 29073 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:05:00.810318 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.812651 29871 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:00.822122 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 900582us
I20260709 13:05:00.786437 32172 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.825409 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:00.804908 32280 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.836714 29071 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.817912 32162 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:00.787864 32136 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:00.877662 29071 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:00.884605 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:00.886310 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:00.886929 29219 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:00.843544 32201 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:00.841576 32215 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:00.837316 32091 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:05:00.837455 32067 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:05:00.901335 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:00.899789 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:00.902209 29214 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.842303 32297 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:00.932703 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:00.912639 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:00.881623 32253 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.941632 29217 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:00.945025 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:00.897904 32215 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:00.844359 32254 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.903455 31995 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.903107 31604 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:05:00.952818 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:00.950469 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:00.955633 29213 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:05:00.904536 29904 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:05:00.958827 29214 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
W20260709 13:05:00.950377 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 73402.4us
I20260709 13:05:00.961158 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:00.963217 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:00.964882 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:00.966718 29213 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:00.949872 32215 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.952003 32297 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:00.967670 29185 connection.cc:256] RPC call timeout handler was delayed by -1.6603863670006831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
I20260709 13:05:00.949621 32201 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:00.971971 29185 connection.cc:256] RPC call timeout handler was delayed by -1.6315863670006365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22480000000000003s for a response
W20260709 13:05:00.972455 29185 connection.cc:256] RPC call timeout handler was delayed by -1.3230894989992521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
I20260709 13:05:00.967814 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:00.937204 32312 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:05:00.973091 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:00.960045 31715 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.974143 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:00.976867 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:00.980623 31715 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.981431 32179 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:00.982069 32179 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:05:00.984275 32306 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:05:00.985743 31604 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.986419 31604 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:00.988457 32156 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:00.988960 32156 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:00.989789 32156 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:00.991148 32156 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.993685 31604 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:00.994447 31604 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:00.995056 31604 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:00.996351 32308 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:05:00.980998 29213 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:00.970309 31924 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:00.967466 32254 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.955976 31995 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.014382 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:01.015137 29220 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:05:01.011579 31924 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:01.016624 31924 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.017777 31924 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:01.021258 32172 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.023222 29217 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:01.025238 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1949) took 1106 ms. Trace:
I20260709 13:05:01.028954 29217 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:01.026225 32269 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.751s	user 0.000s	sys 0.003s
I20260709 13:05:01.029069 29218 rpcz_store.cc:276] 0709 13:04:59.919181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.942210 (+ 23029us) service_pool.cc:224] Handling call
0709 13:04:59.942581 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.942823 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:00.160611 (+217788us) log.cc:844] Serialized 14 byte log entry
0709 13:05:00.976125 (+815514us) log.cc:844] Serialized 14 byte log entry
0709 13:05:01.024840 (+ 48715us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:01.024908 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:01.024967 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.025039 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.025173 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":637048,"prepare.queue_time_us":300146,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":67,"raft.queue_time_us":579943,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":2054,"thread_start_us":455615,"threads_started":4,"wal-append.queue_time_us":239024}
I20260709 13:05:01.032840 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:01.033739 31604 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:01.034812 32317 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:01.035256 32317 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.035612 32317 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:01.035408 32035 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.037969 32035 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.037709 28968 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: config changed from index -1 to 5, term changed from 4 to 7, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to <none>, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 7 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:05:00.950197 32253 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.027827 32124 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:01.031225 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 70162.7us
I20260709 13:05:01.026825 32055 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:01.027156 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:01.042219 28968 catalog_manager.cc:5697] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:01.043712 32055 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:01.042119 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:01.046253 28968 catalog_manager.cc:5697] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 4, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 4 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.030153 29217 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.049119 29217 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:01.049712 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 75674.3us
I20260709 13:05:01.050640 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:01.051385 29214 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:01.055009 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:01.055658 29219 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:01.055605 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:01.049446 31944 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:01.039788 32179 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:01.063892 32055 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.052732 32067 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:01.066190 32055 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:01.057381 32221 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.076236 32170 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:01.077346 32170 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:01.077442 29217 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:05:01.084088 29217 pending_rounds.cc:85] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 7
I20260709 13:05:01.063268 31944 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:01.085106 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:01.086180 29906 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:01.058051 31604 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:05:01.064112 31995 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:01.090590 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:01.091562 29140 raft_consensus.cc:2410] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:05:01.092255 32216 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:05:01.092660 29217 pending_rounds.cc:107] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:05:01.092969 29217 raft_consensus.cc:2889] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:01.093340 29147 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:01.094950 29148 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:01.094904 31944 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.097481 31944 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.083819 32055 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:01.064684 32237 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:05:01.085067 32267 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.854s	user 0.002s	sys 0.000s
I20260709 13:05:01.092522 31604 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:01.107983 31604 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:01.109076 31604 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.095089 32080 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:01.110535 31715 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:01.058784 32201 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.117899 32253 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.083287 32136 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:01.078632 32170 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.119056 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:01.119769 32170 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:01.121172 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:01.062343 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:01.092478 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:01.121985 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:01.122364 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60159.5us
I20260709 13:05:01.124312 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 546) took 1742 ms. Trace:
I20260709 13:05:01.127549 32065 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:01.128320 32065 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:01.129483 31729 raft_consensus.cc:993] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:05:01.124768 32216 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.129272 32065 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.129968 31729 raft_consensus.cc:1081] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Signalling peer 6ad9faa1c14444b29c9db8e71f46e2d6 to start an election
I20260709 13:05:01.130146 32216 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:01.131095 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.131340 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.132180 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.132397 32216 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:01.132882 32216 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.133314 32216 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.100962 32055 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.125516 31621 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:01.121997 32176 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:01.125968 32166 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:01.136273 32166 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.127427 29910 rpcz_store.cc:276] 0709 13:04:59.382045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:59.386897 (+  4852us) service_pool.cc:224] Handling call
0709 13:04:59.387247 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:59.439250 (+ 52003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:59.439278 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:59.443940 (+  4662us) log.cc:844] Serialized 26 byte log entry
0709 13:04:59.519803 (+ 75863us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:59.519847 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.115652 (+1595805us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b5400023d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:01.115698 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:01.115740 (+    42us) raft_consensus.cc:1685] finished
0709 13:05:01.115751 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:01.124249 (+  8498us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":480,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":1386,"spinlock_wait_cycles":118791040,"thread_start_us":58114,"threads_started":1,"wal-append.queue_time_us":169404,"wal-append.run_cpu_time_us":1843,"wal-append.run_wall_time_us":1056281}
I20260709 13:05:01.138612 32252 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:01.139175 32252 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.139813 31618 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.140030 32197 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:01.140399 31618 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.141500 32252 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.135771 31621 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.142679 31618 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.144528 31621 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:01.129647 32304 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.497s	user 0.003s	sys 0.000s
I20260709 13:05:01.145697 32166 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.135236 32216 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:01.146224 32301 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.498s	user 0.001s	sys 0.000s
I20260709 13:05:01.146463 31621 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.140406 32197 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:01.148170 32197 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.149304 32197 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:01.154592 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 334146us
I20260709 13:05:01.153020 32055 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.146997 31621 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.158802 32303 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:01.159368 32303 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.162319 32182 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:01.162850 32182 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.163642 32303 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.168540 32055 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:01.169435 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:01.169636 32252 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:01.169721 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:01.170457 32252 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:01.170572 32055 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:01.171049 32055 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:01.172017 32055 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:01.175028 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:01.175048 32136 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:01.175673 32055 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:01.176079 32272 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:01.054093 28968 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 6 to 9, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 9 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:01.186013 29910 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:01.191936 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:01.193953 29146 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:01.194073 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:01.157740 31621 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.170854 32252 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:01.208890 29148 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:01.239244 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:01.213162 29910 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:05:01.217231 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 175220us
I20260709 13:05:01.219759 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:01.182693 28968 catalog_manager.cc:5697] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 2 to 5, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 5 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:01.246531 28968 catalog_manager.cc:5697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 6 to 8, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 8 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.248348 28968 catalog_manager.cc:5697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.224417 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:01.227385 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.250864 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.228510 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:01.252202 29214 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:01.256263 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.258754 29908 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:01.260411 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.263144 29068 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:01.251251 28968 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.270676 28968 catalog_manager.cc:5697] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 3 to 5, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 5 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.272845 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:01.273054 28968 catalog_manager.cc:5697] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.274978 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:01.278875 29217 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:01.279765 29908 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:01.281358 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.281984 29065 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:05:01.284364 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.284714 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.285351 29073 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:01.053709 32225 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.286497 32285 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:05:01.288064 32240 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:05:01.289075 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.290468 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:01.233460 31715 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:01.292629 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.291850 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.294530 29037 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:01.275341 28968 catalog_manager.cc:5697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.054080 32162 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:01.333904 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.409793 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.779743 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:01.559312 29071 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:01.787590 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:01.781085 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:01.921733 29073 raft_consensus.cc:2368] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
I20260709 13:05:01.922333 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.371207 29217 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:01.924522 29219 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:01.922904 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:01.680033 31902 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.004s	sys 0.015s
I20260709 13:05:01.368430 29214 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:01.738461 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:01.934659 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:01.935060 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.935815 29073 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:01.555135 29070 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:01.634217 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:01.880435 28968 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 5 to 7, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 7 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:01.937078 29218 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:01.937539 28968 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:01.474288 32166 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:01.939019 28968 catalog_manager.cc:5697] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:01.785847 32182 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:01.425202 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 205537us
I20260709 13:05:01.786383 32179 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:01.883117 29066 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:01.410766 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.872148 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.888778 31715 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:01.395772 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:01.933885 31902 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.010s	sys 0.018s
I20260709 13:05:01.965390 29910 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:01.780213 29909 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:01.920116 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.966696 29071 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:01.779744 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.326719 29908 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:05:01.920151 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.749571 32125 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.017s	sys 0.005s
I20260709 13:05:01.580391 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:01.980288 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:01.928565 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:01.283238 32297 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:01.932155 29907 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:05:01.426973 32293 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.625s	user 0.008s	sys 0.003s
I20260709 13:05:01.780628 31604 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:01.580022 32080 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.299360 32225 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.574810 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:01.936669 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:01.580437 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.997553 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.942313 28968 catalog_manager.cc:5697] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 2 to 7, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 7 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.944859 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:01.999845 29214 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:01.949779 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:01.882386 29064 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:01.798717 32068 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:01.785079 32172 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:01.950558 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.787056 32176 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:01.889906 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 595436us
I20260709 13:05:01.972266 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:01.967730 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:02.031879 29071 raft_consensus.cc:2410] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:01.974264 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:01.987845 29185 leader_election.cc:400] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
I20260709 13:05:02.032742 29185 leader_election.cc:403] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:05:02.000696 28968 catalog_manager.cc:5697] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 4 to 5, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 5 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:05:02.033823 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 46174.2us
I20260709 13:05:02.034075 28968 catalog_manager.cc:5697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 4 to 5, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 5 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.996503 29073 raft_consensus.cc:2368] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8.
I20260709 13:05:01.835597 32067 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:01.971772 31715 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:01.984932 32125 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.018s	sys 0.010s
I20260709 13:05:01.504988 32252 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:02.011945 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 61976.4us
I20260709 13:05:02.004518 31621 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:02.011070 32166 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.011843 29221 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:02.022305 32265 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:01.970535 32124 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:01.977658 32065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.043s	sys 0.114s
I20260709 13:05:01.503895 32009 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:01.974002 32066 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:01.989490 32182 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:01.969837 32179 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.052817 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:02.006062 32080 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:01.992444 32136 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:02.035776 28968 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:02.112640 28968 catalog_manager.cc:5697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 3 to 6, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 6 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:02.060747 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:02.011263 32272 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.008373 32321 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:01.966483 32237 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:02.002326 32297 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.055840 32156 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:02.023656 32068 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.087353 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.090930 32182 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.062165 31621 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.094336 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 429) took 1072 ms. Trace:
I20260709 13:05:02.141096 29220 rpcz_store.cc:276] 0709 13:05:01.022300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.023081 (+   781us) service_pool.cc:224] Handling call
0709 13:05:01.023360 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.023769 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.023797 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.024298 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 13:05:01.291726 (+267428us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:01.291765 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.291832 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:02.094115 (+802283us) raft_consensus.cc:1685] finished
0709 13:05:02.094136 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.094272 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21070,"raft.queue_time_us":865,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":96,"thread_start_us":266645,"threads_started":1,"wal-append.queue_time_us":1067758}
I20260709 13:05:02.094759 32066 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:02.098335 32317 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.062s	user 0.009s	sys 0.000s
I20260709 13:05:02.081184 32009 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.100879 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2064) took 1215 ms. Trace:
W20260709 13:05:02.103183 32055 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.927s	user 0.008s	sys 0.001s
I20260709 13:05:02.180270 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:02.181222 29221 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:02.081089 32124 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.080197 32166 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.115201 28968 catalog_manager.cc:5697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 2 to 5, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 5 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:02.081315 32065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.044s	sys 0.114s
I20260709 13:05:02.040159 32176 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:02.127841 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 420583us
W20260709 13:05:02.128935 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 431993us
I20260709 13:05:02.143858 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:02.145476 29216 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.897s	user 0.013s	sys 0.040s
W20260709 13:05:02.203236 29216 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.013s	sys 0.041s
W20260709 13:05:02.149626 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.163641 29222 rpcz_store.cc:276] 0709 13:05:00.885593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.888079 (+  2486us) service_pool.cc:224] Handling call
0709 13:05:00.888362 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.042504 (+154142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.042552 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.046283 (+  3731us) log.cc:844] Serialized 26 byte log entry
0709 13:05:01.486195 (+439912us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:01.486236 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:01.486309 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:02.100638 (+614329us) raft_consensus.cc:1685] finished
0709 13:05:02.100666 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.100810 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6813,"raft.queue_time_us":68051,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3893,"thread_start_us":571757,"threads_started":2,"wal-append.queue_time_us":1032203}
I20260709 13:05:02.112293 32293 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:02.166178 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 385297us
I20260709 13:05:02.061199 32067 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:02.073045 32253 consensus_queue.cc:1175] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "730186a8c70f43f2a57d80163bbc37d2" responder_term: 4 status { last_received { term: 4 index: 7 } last_committed_idx: 6 last_received_current_leader { term: 4 index: 7 } }
W20260709 13:05:02.127569 31924 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.109s	user 0.012s	sys 0.001s
I20260709 13:05:02.186543 28968 catalog_manager.cc:5697] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 1 to 6, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 6 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:02.129902 32201 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.122282 32136 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.125157 32272 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.244599 31976 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:02.262612 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.163503 32317 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.205538 32178 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:02.288605 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1842) took 1012 ms. Trace:
I20260709 13:05:02.294442 29070 rpcz_store.cc:276] 0709 13:05:01.276001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.251822 (+975821us) service_pool.cc:224] Handling call
0709 13:05:02.287847 (+ 36025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.288128 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.288170 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:02.288229 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:02.288282 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.288357 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.288518 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:02.285312 31976 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:02.298808 32325 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:05:02.299077 32176 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:02.292492 32179 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:02.261674 32201 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.234565 32065 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:02.234825 31902 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.340173 31902 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.342640 31902 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:02.343235 31902 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.343425 32252 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:02.298156 32067 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:02.343652 31902 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.346189 32201 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.347036 31902 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.349270 29071 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:02.349982 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1845) took 1070 ms. Trace:
I20260709 13:05:02.350301 29071 rpcz_store.cc:276] 0709 13:05:01.279751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.348688 (+1068937us) service_pool.cc:224] Handling call
0709 13:05:02.349070 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.349725 (+   655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.349923 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:02.300539 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 239883us
I20260709 13:05:02.129652 32237 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.352070 32252 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:02.352795 32252 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.354041 32237 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:02.303190 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.355859 29222 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:05:02.355684 32265 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:02.351709 29071 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:02.361654 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1848) took 1081 ms. Trace:
I20260709 13:05:02.362092 29071 rpcz_store.cc:276] 0709 13:05:01.280607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.351065 (+1070458us) service_pool.cc:224] Handling call
0709 13:05:02.351370 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.361356 (+  9986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.361573 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:02.362998 29221 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:02.308838 29073 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:02.313222 29215 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:02.334748 29906 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:02.364787 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1844) took 1086 ms. Trace:
I20260709 13:05:02.365236 29073 rpcz_store.cc:276] 0709 13:05:01.278310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.308065 (+1029755us) service_pool.cc:224] Handling call
0709 13:05:02.308560 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.363858 (+ 55298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.364723 (+   865us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:02.365864 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1843) took 1089 ms. Trace:
I20260709 13:05:02.366287 29070 rpcz_store.cc:276] 0709 13:05:01.278015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.295651 (+1017636us) service_pool.cc:224] Handling call
0709 13:05:02.365259 (+ 69608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.365432 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.365459 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:02.365602 (+   143us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:02.365653 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.365710 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.365814 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:02.364127 29215 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:02.335554 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 560) took 1166 ms. Trace:
I20260709 13:05:02.302821 32065 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:02.337848 32326 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:05:02.339344 32182 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:02.244633 32293 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.305850 32037 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:02.337845 31976 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:02.269098 32162 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:02.373346 29219 raft_consensus.cc:2368] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8.
I20260709 13:05:02.370530 29907 rpcz_store.cc:276] 0709 13:05:01.169360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.169479 (+   119us) service_pool.cc:224] Handling call
0709 13:05:02.334449 (+1164970us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54005e92d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:02.335489 (+  1040us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751389952}
I20260709 13:05:02.368875 29215 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.371663 32252 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:02.381111 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.385237 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 493) took 1161 ms. Trace:
W20260709 13:05:02.276705 32254 consensus_meta.cc:287] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.263s	user 0.010s	sys 0.002s
I20260709 13:05:02.400995 29143 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:02.401296 32215 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 1.425s	user 0.009s	sys 0.003s
I20260709 13:05:02.400705 29906 rpcz_store.cc:276] 0709 13:05:01.223664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.223831 (+   167us) service_pool.cc:224] Handling call
0709 13:05:02.334575 (+1110744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.364237 (+ 29662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.384946 (+ 20709us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54005e92d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:02.385153 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2553494400}
I20260709 13:05:02.394290 29221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:42970
I20260709 13:05:02.375989 32065 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:02.396860 29072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.013s	sys 0.027s
W20260709 13:05:02.431368 29072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.037s	user 0.013s	sys 0.027s
I20260709 13:05:02.373944 32178 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:02.398609 29067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.009s	sys 0.030s
W20260709 13:05:02.442095 29067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.964s	user 0.009s	sys 0.030s
I20260709 13:05:02.399556 29215 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:02.443233 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.443410 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:02.442111 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.399644 29069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.000s	sys 0.011s
I20260709 13:05:02.390069 32237 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.443885 29220 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
W20260709 13:05:02.403344 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.446259 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.444566 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:02.444818 29069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.000s	sys 0.012s
I20260709 13:05:02.411281 29221 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:02.363310 29071 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:02.422971 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.425027 32317 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:02.426000 32009 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:02.404134 32162 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 13:05:02.437132 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.440791 29143 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
W20260709 13:05:02.467177 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 486291us
I20260709 13:05:02.449465 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:02.390944 32179 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.410307 32254 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.445796 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.496169 29214 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:02.496641 29214 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:02.447592 29221 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:02.472147 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:02.500626 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:02.444442 31621 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.506410 32254 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.509048 32254 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.509642 32254 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:02.510162 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.429701 32215 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.497575 29214 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.515066 29214 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:02.443954 29213 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:02.393426 32037 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.512102 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:02.448521 32166 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:02.450318 32287 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:02.531569 29064 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:02.448114 32178 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.515558 32252 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:02.543593 29904 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:02.547236 29219 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:05:02.477267 29215 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:05:02.552147 29215 pending_rounds.cc:85] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 6
I20260709 13:05:02.553839 29215 pending_rounds.cc:107] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:05:02.559373 29215 raft_consensus.cc:2889] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:02.562433 29217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.017s	sys 0.007s
I20260709 13:05:02.516096 32080 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:02.562919 29217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.017s	sys 0.007s
I20260709 13:05:02.515046 32068 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.514091 32334 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:02.450549 32237 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:02.580319 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.530520 32297 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:02.510149 32254 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.592433 29907 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:02.593102 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:02.582629 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.557881 29221 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.513676 32179 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:02.577126 29218 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.034s	sys 0.038s
I20260709 13:05:02.596024 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:02.616297 29218 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.034s	sys 0.038s
I20260709 13:05:02.616590 29215 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:05:02.598600 32253 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:02.619416 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:02.620692 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:02.596772 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:02.600266 29143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2084) took 1408 ms (client timeout 1502 ms). Trace:
W20260709 13:05:02.600495 32333 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:05:02.631422 29907 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:02.631807 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:02.632519 29909 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:02.619542 32253 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:02.636991 32253 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:02.611564 32170 consensus_meta.cc:287] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.491s	user 0.006s	sys 0.006s
I20260709 13:05:02.650626 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2065) took 1764 ms. Trace:
I20260709 13:05:02.588459 32334 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:02.587072 32252 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:02.586350 32009 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:02.588641 32080 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:02.614622 29220 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:02.582158 29185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.026s	sys 0.043s
I20260709 13:05:02.623106 29213 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:02.622783 29143 rpcz_store.cc:269] 0709 13:05:01.191466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.191616 (+   150us) service_pool.cc:224] Handling call
0709 13:05:02.600078 (+1408462us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54003a48d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:02.600201 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2778545024}
W20260709 13:05:02.674067 29185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.026s	sys 0.043s
I20260709 13:05:02.675392 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.625850 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:02.635104 32293 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:02.648926 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:02.593489 32201 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:02.652041 29216 rpcz_store.cc:276] 0709 13:05:00.885846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.885968 (+   122us) service_pool.cc:224] Handling call
0709 13:05:00.886361 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.247700 (+361339us) spinlock_profiling.cc:243] Waited 7.2 ms on lock 0x7b18004732c0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f7836b05060 00007f7836b05bfb 00007f7836b1064b
0709 13:05:01.247860 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.247885 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.248457 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 13:05:02.203801 (+955344us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:02.203865 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.204001 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:02.650304 (+446303us) raft_consensus.cc:1685] finished
0709 13:05:02.650325 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.650563 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25558,"raft.queue_time_us":64752,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":2688,"spinlock_wait_cycles":16600192,"thread_start_us":1023451,"threads_started":2,"wal-append.queue_time_us":1203284}
I20260709 13:05:02.637643 32297 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:05:02.606705 29214 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:05:02.685237 29214 pending_rounds.cc:85] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 7
I20260709 13:05:02.685500 29214 pending_rounds.cc:107] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:05:02.685762 29214 raft_consensus.cc:2889] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:02.646903 32124 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.678805 29215 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
W20260709 13:05:02.679425 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.680569 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:02.687162 29214 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:02.658867 29144 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:02.687793 29216 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:02.690399 29213 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:02.699059 29221 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:02.673985 32037 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:02.673636 32334 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:02.757671 32254 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.685663 32179 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.759361 29215 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:02.684592 32178 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:02.760394 32307 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:02.687318 32197 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:02.696830 31621 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.697199 32215 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:02.774835 32136 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.680527 32009 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:02.778079 29038 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.012s	sys 0.083s
W20260709 13:05:02.780061 29038 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.013s	sys 0.083s
I20260709 13:05:02.689286 31976 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:05:02.680401 32080 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.687628 32201 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.784335 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 488) took 1701 ms. Trace:
W20260709 13:05:02.780440 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 634302us
I20260709 13:05:02.787791 29904 rpcz_store.cc:276] 0709 13:05:01.083432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.084237 (+   805us) service_pool.cc:224] Handling call
0709 13:05:01.084579 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.544258 (+1459679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.781661 (+237403us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54007e0254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:02.784248 (+  2587us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3350093184}
I20260709 13:05:02.785871 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:02.776501 32197 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:02.788602 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:02.788944 29907 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:02.795305 29185 leader_election.cc:400] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8.
I20260709 13:05:02.795701 29185 leader_election.cc:403] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:02.798473 32009 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.800424 32080 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.804268 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 327) took 1643 ms. Trace:
I20260709 13:05:02.804410 32055 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:02.805047 32055 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.805795 32178 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:02.804790 29067 rpcz_store.cc:276] 0709 13:05:01.161324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.272011 (+110687us) service_pool.cc:224] Handling call
0709 13:05:01.272309 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.277521 (+  5212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.277554 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.298056 (+ 20502us) log.cc:844] Serialized 26 byte log entry
0709 13:05:02.442572 (+1144516us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:02.442632 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.442741 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:02.804008 (+361267us) raft_consensus.cc:1685] finished
0709 13:05:02.804034 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.804199 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4813,"raft.queue_time_us":13572,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"thread_start_us":964313,"threads_started":1,"wal-append.queue_time_us":1407530}
I20260709 13:05:02.792560 31976 consensus_queue.cc:1243] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:02.807410 32055 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:02.809465 32037 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:02.810015 32037 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:02.810165 29221 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:05:02.810748 32156 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.681s	user 0.002s	sys 0.009s
I20260709 13:05:02.811519 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1942) took 1559 ms. Trace:
I20260709 13:05:02.811981 29072 rpcz_store.cc:276] 0709 13:05:01.252319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.252496 (+   177us) service_pool.cc:224] Handling call
0709 13:05:01.252913 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.253302 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.253329 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.393504 (+140175us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b2c000014c8. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:05:01.394026 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:05:02.431790 (+1037764us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:02.431844 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.431964 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:02.811179 (+379215us) raft_consensus.cc:1685] finished
0709 13:05:02.811202 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:02.811454 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32779,"raft.queue_time_us":24750,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":19280,"spinlock_wait_cycles":3824896,"thread_start_us":1037398,"threads_started":1,"wal-append.queue_time_us":1395754}
I20260709 13:05:02.813239 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:02.813938 29065 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:02.816521 29874 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.020s	sys 0.020s
W20260709 13:05:02.818586 29874 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.020s	sys 0.020s
W20260709 13:05:02.817637 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.011s	sys 0.029s
I20260709 13:05:02.819511 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.820359 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.821112 29216 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:02.822080 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.822528 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:02.820351 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.011s	sys 0.029s
I20260709 13:05:02.825953 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.837563 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:02.820677 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.838716 29216 raft_consensus.cc:2368] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8.
I20260709 13:05:02.826713 29214 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:02.787482 32237 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:02.844112 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:02.848366 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.777127 32307 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.778956 31621 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:02.853509 29874 leader_election.cc:336] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.797s (SENT)
W20260709 13:05:02.853996 29875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.010s	sys 0.013s
I20260709 13:05:02.849143 29215 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:02.850889 29221 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:02.855592 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.855602 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:02.849992 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.792106 32125 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.071s	sys 0.134s
I20260709 13:05:02.793987 32197 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:02.817175 31715 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.741s	user 0.012s	sys 0.001s
I20260709 13:05:02.854867 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:02.855213 29039 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.015s	sys 0.029s
W20260709 13:05:02.867305 29039 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.015s	sys 0.029s
I20260709 13:05:02.816869 32178 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8.
I20260709 13:05:02.870333 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:02.871002 29070 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:02.794984 32136 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:02.858477 29036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.006s	sys 0.037s
I20260709 13:05:02.848373 32215 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:02.848207 31976 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:02.880450 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:02.861470 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1840) took 1589 ms. Trace:
I20260709 13:05:02.852535 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.864027 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:02.881981 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:02.865592 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:02.881676 29065 rpcz_store.cc:276] 0709 13:05:01.271650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.044654 (+773004us) service_pool.cc:224] Handling call
0709 13:05:02.813774 (+769120us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.814317 (+   543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.861248 (+ 46931us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b5400208ad0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:02.861401 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1768020224}
I20260709 13:05:02.883020 29071 raft_consensus.cc:2410] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:05:02.816702 32156 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7.
W20260709 13:05:02.867769 29144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2086) took 1674 ms (client timeout 1559 ms). Trace:
I20260709 13:05:02.869129 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:02.870110 32254 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:02.858219 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:02.871847 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:02.895496 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.920912 29067 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
W20260709 13:05:02.866786 32125 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.071s	sys 0.134s
I20260709 13:05:02.837095 32265 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.873648 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:02.874495 29036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.006s	sys 0.037s
I20260709 13:05:02.929764 29215 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
W20260709 13:05:02.858712 29875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.011s	sys 0.014s
W20260709 13:05:02.930639 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 723092us
I20260709 13:05:02.875139 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.878345 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.881301 29066 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:02.876744 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:02.889330 29073 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.007s	sys 0.040s
I20260709 13:05:02.922050 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:02.931732 29220 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:05:02.933084 29067 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:05:02.856773 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.854947 32182 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.923707 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:02.924319 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 51589us
I20260709 13:05:02.924578 29221 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:05:02.947554 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:02.811093 32037 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.949538 29070 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:02.949613 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:02.950459 29066 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:02.859292 32009 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
W20260709 13:05:02.926668 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.955147 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.925782 29068 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:02.928817 29113 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:02.785467 32252 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:02.877171 32172 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.026s	sys 0.022s
I20260709 13:05:02.836790 32080 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:02.932277 29213 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
W20260709 13:05:02.883944 29112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.009s	sys 0.013s
I20260709 13:05:02.934300 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:02.880084 32179 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
W20260709 13:05:02.934489 29073 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.009s	sys 0.054s
W20260709 13:05:02.913466 29872 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.009s	sys 0.045s
W20260709 13:05:02.968822 29872 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.009s	sys 0.046s
W20260709 13:05:02.923511 29144 rpcz_store.cc:269] 0709 13:05:01.193730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.193847 (+   117us) service_pool.cc:224] Handling call
0709 13:05:02.867601 (+1673754us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400145354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:02.867699 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3367150976}
I20260709 13:05:02.919826 32345 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:02.879948 32124 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.871363 32237 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:05:02.921175 31976 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:02.882288 29064 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:02.872169 31715 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.963774 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:02.989027 29213 raft_consensus.cc:2376] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9. Re-sending same reply.
W20260709 13:05:02.933625 29222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.000s	sys 0.020s
I20260709 13:05:02.990761 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:02.964555 29065 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:02.961671 29111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.004s	sys 0.016s
W20260709 13:05:02.991217 29222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.000s	sys 0.021s
W20260709 13:05:02.991703 29111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.005s	sys 0.016s
W20260709 13:05:02.995978 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 636541us
W20260709 13:05:02.965432 29112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.011s	sys 0.019s
I20260709 13:05:02.937253 32066 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:02.922252 32136 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.933768 32337 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 13:05:02.961583 32172 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.027s	sys 0.022s
I20260709 13:05:02.980579 29218 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:02.929766 32254 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.965197 32156 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:05:03.013933 32197 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.014501 32178 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:03.014762 32156 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:03.014940 32178 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:03.015179 32156 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:03.015666 32254 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:03.016500 32342 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:05:02.972043 32351 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:03.018893 31621 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.018373 32351 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.973489 32037 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:03.020964 32293 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:03.021378 32293 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:02.965750 32272 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:02.951663 32182 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.022249 32293 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.023353 32197 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.023496 32293 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:02.972296 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 179455us
W20260709 13:05:03.027398 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.028522 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.028981 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.983948 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:02.986917 32347 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:03.008694 32287 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:03.010746 32136 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.011231 29072 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:03.012845 32307 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:03.096717 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 165320us
I20260709 13:05:03.098938 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:03.015904 32178 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:03.019500 32343 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:05:02.987504 31976 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:02.935555 32265 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.967945 32080 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:02.982481 32345 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.007198 29070 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:03.033949 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:03.115232 29907 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:03.038436 32182 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.033663 32156 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:02.986720 32124 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.125092 29909 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:03.054292 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:02.967669 32348 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:05:03.059330 32349 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:03.018473 32336 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
W20260709 13:05:03.072661 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.073272 32344 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.060380 32287 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:03.074589 32337 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:03.018905 32338 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
I20260709 13:05:03.025602 32350 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.089550 31715 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:03.090746 32201 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:03.091239 32347 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:03.034664 32272 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.092811 32351 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.103735 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.060273 32297 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:03.116866 32065 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.003s
I20260709 13:05:03.128019 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.108268 32237 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:03.129451 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.131666 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.131608 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.112088 32357 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.145483 29907 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:03.112924 32215 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:03.149132 32287 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:03.130074 31621 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:03.138197 32178 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.187456 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.193729 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.190631 29910 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:03.195997 29907 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:05:03.189200 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.197927 29904 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:03.199410 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.200140 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.203014 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.188557 32009 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:03.204335 32009 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:03.205626 32009 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.210062 32345 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.162078 32337 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:03.191056 31621 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.212339 31621 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:03.212272 32337 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.215705 32215 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.217476 32265 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.223570 32124 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:03.225697 32335 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.273s	user 0.004s	sys 0.001s
I20260709 13:05:03.137804 32349 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.161815 32297 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.152241 32272 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.136013 32339 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 13:05:03.161832 32352 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:05:03.149514 32344 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.237169 32365 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.198068 32357 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.151010 32350 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.253631 32357 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.255685 32350 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.258481 32344 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.261440 32365 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:03.303444 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 426515us
I20260709 13:05:03.303745 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:03.305930 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:03.309197 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:03.084496 29068 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
W20260709 13:05:03.314527 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.314994 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1856) took 1336 ms. Trace:
W20260709 13:05:03.003587 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.315949 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.126919 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 72711.3us
W20260709 13:05:03.113407 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 155059us
I20260709 13:05:03.315770 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 328) took 2046 ms. Trace:
I20260709 13:05:03.122786 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:03.324973 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 202180us
I20260709 13:05:03.329418 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:03.331085 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 340248us
I20260709 13:05:03.178207 32065 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.336423 32065 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.320531 29069 rpcz_store.cc:276] 0709 13:05:01.269125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.284044 (+ 14919us) service_pool.cc:224] Handling call
0709 13:05:01.298437 (+ 14393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:01.298850 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.298879 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:01.365736 (+ 66857us) log.cc:844] Serialized 52 byte log entry
0709 13:05:02.447538 (+1081802us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:02.448302 (+   764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.448459 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.314408 (+865949us) raft_consensus.cc:1685] finished
0709 13:05:03.314461 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.315473 (+  1012us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":53645,"thread_start_us":935870,"threads_started":1,"wal-append.queue_time_us":1319665}
I20260709 13:05:03.340583 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:03.340866 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:03.323444 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:03.315779 29073 rpcz_store.cc:276] 0709 13:05:01.978165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.366385 (+388220us) service_pool.cc:224] Handling call
0709 13:05:02.366927 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.367169 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:02.968487 (+601318us) log.cc:844] Serialized 14 byte log entry
0709 13:05:03.313333 (+344846us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.313722 (+   389us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:03.313900 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.314109 (+   209us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.314370 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":677045,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"thread_start_us":944846,"threads_started":2,"wal-append.queue_time_us":231017}
I20260709 13:05:03.343384 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:03.344240 32066 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.345257 32066 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:05:03.347239 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.348429 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.348673 32066 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:03.349200 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.349763 32066 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:03.350628 32066 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.351433 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:03.356608 32080 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.357585 32080 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.358357 32080 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.361346 32080 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.361958 32067 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:03.363016 32067 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:03.351783 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:03.402206 32179 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.403021 31715 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:03.403182 32182 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:03.403083 32172 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:03.406297 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.406499 32124 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:03.406745 29186 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:03.407648 32170 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:03.407711 32166 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:03.402206 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:03.409053 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.207754 32009 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:03.410404 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.411252 29218 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:03.412021 32197 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:03.411903 32201 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:05:03.415736 32253 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.740s	user 0.010s	sys 0.001s
I20260709 13:05:03.354076 32066 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:03.417840 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 66635.1us
I20260709 13:05:03.430879 32166 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.437310 32166 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:03.456576 32055 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:03.463147 32297 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.468732 32179 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.469507 32170 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:03.470557 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 1951) took 1571 ms. Trace:
I20260709 13:05:03.471230 32170 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.402241 32067 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.420607 31715 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:03.480188 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 115633us
W20260709 13:05:03.482316 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 170671us
I20260709 13:05:03.485082 32351 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:03.434578 32253 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.482793 32366 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:05:03.484602 32201 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:03.499431 32201 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:03.501781 29907 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:03.501417 32201 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:03.503767 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 101924us
I20260709 13:05:03.504376 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:03.504777 32367 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:03.459432 32156 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:03.508051 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 101483us
I20260709 13:05:03.505079 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:03.516857 32376 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:05:03.517663 32349 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.519486 29908 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:03.520778 32344 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:03.520898 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64662.9us
I20260709 13:05:03.521390 32344 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:03.521919 32344 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:03.526307 29186 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:03.508873 32367 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:03.528081 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 55914.8us
I20260709 13:05:03.528227 32367 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:03.486593 32182 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:03.479085 31715 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.534173 31715 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:03.537122 32066 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:03.537624 29112 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:03.538710 29112 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:03.538574 32124 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:03.539099 32066 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:03.539568 32066 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:03.540002 32066 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:03.540215 32065 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:03.540692 32065 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:03.541177 32065 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:03.542194 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:03.544304 32125 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:03.471869 32179 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.481869 32272 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.477563 32365 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.473829 32067 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.531360 32182 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.567974 32237 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:03.568519 32363 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:03.557210 32366 consensus_queue.cc:1243] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:03.554066 32349 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:03.471027 29217 rpcz_store.cc:276] 0709 13:05:01.899195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.924135 (+ 24940us) service_pool.cc:224] Handling call
0709 13:05:01.924484 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.931778 (+  7294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:01.931823 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:01.932416 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:05:02.579635 (+647219us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:02.579677 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.579747 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.103966 (+524219us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:03.469851 (+365885us) raft_consensus.cc:1685] finished
0709 13:05:03.469908 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.470325 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15253,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":21845,"spinlock_wait_cycles":6294912,"thread_start_us":646761,"threads_started":1,"wal-append.queue_time_us":950392}
I20260709 13:05:03.572628 29903 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:03.571866 32067 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:03.594601 31621 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:03.584713 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 58668.2us
I20260709 13:05:03.596546 32253 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.599660 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:03.600030 32297 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:03.600595 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.600665 32201 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:03.601853 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.586687 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.602661 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:03.574282 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 70135.5us
I20260709 13:05:03.607347 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:03.584852 32272 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:03.609860 32272 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:03.600864 32297 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.611416 32272 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.613219 32297 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.613981 32297 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:03.615592 32136 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:03.617000 32350 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:05:03.613749 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 71676.8us
I20260709 13:05:03.618803 31976 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.620450 29113 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:03.622165 32237 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.622551 31976 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.622754 32067 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:03.622939 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.623039 32237 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.623582 32067 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:03.620692 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 132479us
I20260709 13:05:03.626529 32237 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:03.634886 32067 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:03.636472 32297 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.639292 32297 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:03.640332 32334 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.869s	user 0.017s	sys 0.003s
I20260709 13:05:03.640925 32334 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.635188 32125 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:03.642284 32125 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.642805 32125 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:03.645345 32293 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.613s	user 0.008s	sys 0.008s
I20260709 13:05:03.653607 32124 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:03.635283 32065 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:03.671476 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 62136.2us
W20260709 13:05:03.674002 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 122064us
W20260709 13:05:03.681743 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.654103 32344 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.692335 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:03.692776 32287 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.695111 32357 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:03.695947 32357 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.696205 32345 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:03.698536 29214 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:03.699937 32287 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.701840 32347 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.702615 32347 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.706280 32272 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:03.706880 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.689080 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:03.728572 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.729226 32357 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.730820 31621 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.735153 32334 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.736214 32344 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.741473 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.742166 29143 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:03.742436 29142 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:03.743613 29144 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:03.744079 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.744885 29142 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:03.745801 29113 leader_election.cc:400] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8.
I20260709 13:05:03.746102 29113 leader_election.cc:403] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:05:03.744907 29146 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:03.712982 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:03.749904 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 61053.8us
W20260709 13:05:03.763545 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 76615.4us
I20260709 13:05:03.767831 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:03.767854 29142 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:03.768664 29073 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:05:03.769853 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.770521 29148 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:03.770720 32065 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.762403 32182 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.772056 32357 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:03.772222 29221 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:05:03.772869 32182 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.774137 32347 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.776117 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.777045 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:03.778079 29214 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:03.784703 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:03.786521 32367 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.787292 29147 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:03.788524 29144 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:03.788902 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.789551 32166 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.790529 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:03.791268 29069 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:05:03.791767 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:03.792078 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.794200 29140 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:05:03.794252 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.794621 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:03.795157 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44327.4us
I20260709 13:05:03.793150 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:03.802738 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.803972 29073 raft_consensus.cc:2393] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:03.806156 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.792778 29072 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:05:03.792623 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:03.792855 29218 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:03.812646 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:03.760903 32334 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.053s
W20260709 13:05:03.814529 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 68949.4us
I20260709 13:05:03.815421 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:03.815704 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:03.816653 29069 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:03.817158 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:03.817641 32156 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:03.818058 32287 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:03.818956 32287 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:03.820226 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:03.821165 29909 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:03.820166 32156 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:03.792531 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:03.822063 32156 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:03.776924 32182 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:03.831468 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.829368 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.791867 32166 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.836059 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:03.844280 32337 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.844800 32334 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:03.845275 32334 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.845011 32337 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.845731 32334 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:03.848091 31621 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:03.848527 31621 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.848945 31621 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:03.849753 32381 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:05:03.851666 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:03.852092 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:03.808462 32066 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.852998 32337 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:03.854575 32066 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.842928 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:03.794442 32125 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.842458 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:03.862457 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.864583 29142 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:03.866194 29068 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:03.873903 29214 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:03.875906 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:03.876935 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:03.877725 29065 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:03.883432 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.887588 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:03.887974 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.888141 32253 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:03.888474 29148 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:03.888859 29148 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:03.889988 29148 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:03.891194 29148 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:03.888743 29144 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:03.887173 29065 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:03.888281 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.897102 29143 raft_consensus.cc:2368] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:03.875897 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:03.887539 32357 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8.
I20260709 13:05:03.900471 29142 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 4.
I20260709 13:05:03.905884 29147 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:03.904203 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.905807 32066 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:03.888784 32253 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:05:03.910329 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 78947.5us
I20260709 13:05:03.894860 32055 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:03.911726 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 69258.5us
I20260709 13:05:03.923295 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:03.926488 29141 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:05:03.916848 32055 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.927471 32055 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:03.927970 32386 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:05:03.934249 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.935166 29144 raft_consensus.cc:2368] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9.
I20260709 13:05:03.936674 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:03.940207 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:03.941074 29068 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:03.941196 29065 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:03.942853 29909 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:03.945396 29146 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:03.946547 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:03.946686 29186 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:03.948036 29142 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:03.940099 32066 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:03.908941 32156 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.955091 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.956216 32178 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:03.958235 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.958500 29214 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:03.948961 32367 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:03.959086 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.959328 32383 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:03.964476 29143 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:03.964998 32383 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:03.966173 32383 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:03.975169 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52835.2us
I20260709 13:05:03.949398 29037 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:03.959878 32178 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:03.961969 32337 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:05:03.979969 32287 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:03.980830 32178 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:03.960350 32125 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:03.979242 29148 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:03.954095 32357 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:03.957389 32066 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:04.000710 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.005029 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.005811 29143 raft_consensus.cc:2368] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:04.006845 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:03.993741 32357 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:04.003321 32253 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.029719 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:04.031144 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 82672.9us
I20260709 13:05:04.036350 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:04.039198 32357 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:04.043524 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.044728 29139 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:04.045768 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.046109 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:04.046579 29139 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:04.046725 29140 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:04.047403 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.047978 29139 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:04.048363 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.049829 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.005980 32287 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.051431 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.043589 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:04.052340 32287 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.039440 29143 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:04.053764 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.054226 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.048826 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.055114 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.055431 29139 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:04.056281 29221 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:04.058629 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:04.058725 32337 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:04.060952 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.063041 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:04.063906 29902 raft_consensus.cc:2368] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
I20260709 13:05:04.064277 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:04.064618 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.065222 29144 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:04.065783 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.066038 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.066891 29902 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:04.067113 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.067896 32389 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.068472 32389 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.070325 32272 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:04.071888 32272 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:04.072247 32272 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:04.072633 32272 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:04.075520 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.076560 29148 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:04.075619 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.076594 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:04.078445 29148 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:05:04.067812 29144 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
I20260709 13:05:04.045792 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:03.995352 32182 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:04.084503 29147 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:04.084724 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.089715 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 88273.4us
I20260709 13:05:04.090425 29112 leader_election.cc:400] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
I20260709 13:05:04.090941 29112 leader_election.cc:403] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:04.052558 29145 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:04.075460 32357 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.101581 32125 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:04.103012 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.105525 29140 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
W20260709 13:05:04.106509 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.107061 29907 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:04.116544 29038 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.120007 29140 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:04.127784 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:04.128729 29145 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:05:04.128363 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:04.131402 29907 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
W20260709 13:05:04.135151 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.135038 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:04.102066 32125 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.107875 32351 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:05:04.144107 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:04.148618 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.149382 29140 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:04.149904 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:04.150223 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.150588 29146 raft_consensus.cc:2376] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8. Re-sending same reply.
I20260709 13:05:04.151396 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.152009 29140 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:04.152949 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:04.153303 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:04.153898 29141 raft_consensus.cc:2376] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7. Re-sending same reply.
I20260709 13:05:04.154033 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.154683 29140 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:04.154769 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:04.155498 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.156064 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:04.156392 32389 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:04.156724 29140 raft_consensus.cc:2410] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:04.156996 32125 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.157500 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:04.158511 32395 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.212s	user 0.000s	sys 0.003s
W20260709 13:05:04.161675 32391 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.226s	user 0.002s	sys 0.000s
I20260709 13:05:04.164438 32156 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:04.102357 32334 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.170563 32287 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:04.171041 32334 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:04.171886 29908 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:05:04.176787 32367 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:04.178071 32367 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:04.178476 32367 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:04.180605 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:04.181241 29221 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:04.156481 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:04.186374 29147 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:04.155534 29141 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:04.151576 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:04.194031 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.195046 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.198042 29112 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:04.200744 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 109139us
I20260709 13:05:04.201416 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:04.202826 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.208293 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:04.210348 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.208495 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:04.223704 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:04.181669 32055 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.235621 29065 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:04.236093 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 101189us
I20260709 13:05:04.179584 32367 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.238426 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:04.249791 29068 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:04.261554 29902 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:05:04.265905 32354 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:04.266563 32354 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.267026 32354 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:04.232012 32297 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:04.268443 32297 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:04.269766 32297 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:04.251960 32201 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.233024 32357 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:04.293104 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.302227 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:04.308761 29911 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:05:04.310521 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:04.295887 29142 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:05:04.311314 29070 raft_consensus.cc:2410] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:05:04.311714 29139 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:04.312511 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:04.312899 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:04.313601 29068 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:04.313751 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 92988.2us
I20260709 13:05:04.314108 32055 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:04.314687 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:04.316089 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:04.316881 29218 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:04.315650 29185 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.321973 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:04.321973 29145 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:04.314028 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:04.329600 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:04.334734 29069 raft_consensus.cc:2393] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:04.319049 32337 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:04.314708 32055 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:04.278344 32009 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:04.259814 31621 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:04.348534 32197 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:05:04.357081 32197 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:04.357416 32197 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:04.358052 31621 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:04.358237 32197 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:04.359637 32197 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:04.360260 29143 raft_consensus.cc:2368] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8.
I20260709 13:05:04.355196 29064 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:04.275728 32347 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:05:04.361515 32182 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:04.362223 29910 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:04.362319 32272 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:04.363142 32405 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:05:04.324249 32065 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
I20260709 13:05:04.364707 31715 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:04.365474 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:04.284704 32201 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:04.367890 29185 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:04.368441 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:04.370055 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:04.305146 32066 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:04.400861 29221 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:04.406204 29071 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:04.418104 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:04.419735 32253 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:04.420399 32253 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:04.421572 32253 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.407346 32170 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:04.450517 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 199811us
I20260709 13:05:04.394122 31715 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:04.406842 32080 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:04.409988 32055 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:04.453503 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:04.454372 32080 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:04.467465 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 99664.5us
W20260709 13:05:04.476971 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 108626us
I20260709 13:05:04.455299 31715 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:04.477970 29068 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:04.434535 32009 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:04.480139 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 139721us
I20260709 13:05:04.455960 32389 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:04.455715 32055 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.467669 32337 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:04.481597 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 256510us
I20260709 13:05:04.476815 32182 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:04.502416 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 317738us
I20260709 13:05:04.499876 29218 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:04.510654 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.490389 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:04.514819 32066 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:04.516620 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 76449.3us
I20260709 13:05:04.517776 32080 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.520788 32201 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:04.521763 32272 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:04.527045 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:04.530090 32354 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:04.536720 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50308.6us
I20260709 13:05:04.497272 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.546728 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.502978 32009 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:04.592741 29221 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 5.
W20260709 13:05:04.601888 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 95883.3us
I20260709 13:05:04.593730 29064 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:04.543475 32182 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.540134 32055 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:04.503636 32170 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:04.596915 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:04.601943 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 55277.9us
I20260709 13:05:04.504271 32178 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.089s
I20260709 13:05:04.595103 32389 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.607908 32418 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:04.609676 32416 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:04.609982 29871 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:04.610123 32416 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:04.605767 32182 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:04.592216 32383 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:05:04.611394 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 121100us
I20260709 13:05:04.611177 32416 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:04.614346 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 127216us
I20260709 13:05:04.616478 29112 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:04.612179 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:04.617971 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.618367 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.610006 32389 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:04.603946 32272 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:04.622980 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:04.606060 32170 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:04.619657 32009 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:04.614916 32201 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:04.621125 32416 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:04.619858 32156 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:04.620864 32419 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:04.633944 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:04.637467 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:04.638447 29910 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:04.639237 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:04.605312 32125 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:04.642037 32418 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:05:04.642427 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.642496 32383 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:05:04.642967 32411 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.002s
I20260709 13:05:04.643278 32352 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:05:04.624071 32178 consensus_queue.cc:1243] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:04.622969 32357 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:04.654682 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 44812.4us
I20260709 13:05:04.648967 32367 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:04.664721 32287 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:04.663692 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:04.664534 29911 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:04.630738 32253 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:04.649132 32201 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.632665 32272 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:04.640612 32009 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:04.669899 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 57793.7us
I20260709 13:05:04.672363 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:04.673074 32415 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:05:04.673410 32178 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:04.710608 32178 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:04.673480 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:04.674477 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:04.675396 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.678650 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:04.674002 32287 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.713546 29908 raft_consensus.cc:2410] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:05:04.684418 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:04.685892 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.686343 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.686770 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:04.714371 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 99922.3us
I20260709 13:05:04.715070 29112 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:04.688076 32351 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:04.688645 29902 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:04.673089 32357 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:04.695302 32080 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:04.699751 32201 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:04.702723 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 71611us
I20260709 13:05:04.671079 32418 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:04.716580 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.720482 29904 raft_consensus.cc:2410] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:04.679328 32419 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:05:04.733790 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 94624.9us
I20260709 13:05:04.738051 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.709656 32009 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.738734 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.739087 29217 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:04.719452 32287 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.738965 29214 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:04.711237 32178 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:04.740211 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.723503 32352 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:04.724067 32065 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:04.745042 29221 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:04.762813 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:04.762935 32066 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:04.770920 32066 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.770807 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:04.771528 32406 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.171s	user 0.004s	sys 0.000s
I20260709 13:05:04.772087 29222 raft_consensus.cc:2376] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7. Re-sending same reply.
I20260709 13:05:04.772607 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.771520 32066 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:04.773677 32383 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:04.775007 32418 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.742292 32419 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:04.776396 32419 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:04.781719 32182 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:04.779094 32055 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.786669 32055 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:04.787501 32367 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:04.788003 32354 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:04.788115 32065 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:04.788800 32354 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:04.789044 32066 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:04.789279 32065 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.789988 32352 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:04.791594 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:04.776423 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:04.772308 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.772666 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 88335.4us
I20260709 13:05:04.791646 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.794092 29218 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:04.789402 32354 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:04.796169 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:04.738224 32197 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:04.798939 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 120021us
I20260709 13:05:04.808631 29222 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:05:04.809279 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 96471.1us
I20260709 13:05:04.847950 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.855477 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.797685 32182 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:04.862833 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 127010us
W20260709 13:05:04.864560 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 131383us
I20260709 13:05:04.872321 29068 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [3.3-9.5]
I20260709 13:05:04.773620 32272 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.876449 29218 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:04.876534 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:04.881089 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.883188 29071 raft_consensus.cc:2368] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9.
W20260709 13:05:04.883997 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 145818us
I20260709 13:05:04.884105 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:04.888885 29064 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:05:04.890969 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.891901 29218 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:05:04.889365 29064 pending_rounds.cc:85] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 3
I20260709 13:05:04.871994 32125 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:04.893985 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.895744 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:04.859301 32424 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:05:04.900411 29072 raft_consensus.cc:2410] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:05:04.897835 32354 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:04.904057 32426 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:05:04.893296 29064 pending_rounds.cc:107] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:05:04.908802 29064 raft_consensus.cc:2889] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:04.909659 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:04.912708 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 113799us
I20260709 13:05:04.941802 32437 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:05:04.944037 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 51531.1us
W20260709 13:05:04.945849 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 141111us
I20260709 13:05:04.854561 32352 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.958871 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:04.946707 29111 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.962967 32436 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:04.963505 32436 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.963853 32436 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:04.966113 32287 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:04.963016 29072 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:04.972065 29147 raft_consensus.cc:2410] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:05:04.976166 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.945403 29185 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:04.947028 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.979168 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.980422 29185 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.949210 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:04.982102 29218 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:04.950801 32425 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:05:04.982779 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.982898 32425 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:04.991158 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:04.984054 32425 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:04.992121 32297 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:04.992692 32297 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.993186 32297 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:04.974434 32426 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:04.962643 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:05.000881 32357 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:05.001530 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:05.004616 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:05.005548 29068 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:05.000449 32426 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.983933 32430 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:05.018124 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 147956us
I20260709 13:05:05.036332 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:05.036477 29221 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:05.036608 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:04.960155 32421 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:05.042893 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:05.046799 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:05.047130 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:05.018443 32426 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:05.070888 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:05.034399 32178 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.045750 32421 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:05.063380 32347 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:05.060834 32287 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:05.037045 32197 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.033099 32430 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.081430 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:04.975554 31715 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:05.081796 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 222407us
I20260709 13:05:05.103152 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:05.104341 29142 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:05.053900 32416 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:05.110767 29039 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:05.082125 29221 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:05:05.109555 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:05.115448 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:05.116366 29065 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:05.121107 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:05.121325 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:05.121773 29143 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:05.122133 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:05.122694 29064 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:05:05.123190 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:05.124290 29911 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:05.124125 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:05.126307 29072 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:05.123394 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:05.127631 29070 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:05.127799 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:05.054951 32440 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:05.062634 32009 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.130154 29139 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:05:05.130734 29139 pending_rounds.cc:85] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 7
I20260709 13:05:05.130270 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:05.130982 29139 pending_rounds.cc:107] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:05:05.131553 29139 raft_consensus.cc:2889] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:05.050717 31621 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:05:05.088917 32425 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:05.132540 29217 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:05.133311 29218 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:05:05.134446 29221 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:05.088546 32357 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:05.101711 32430 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:05.137521 29065 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:05:05.136720 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 267818us
I20260709 13:05:05.138897 29068 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 5: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:05.110451 32416 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.110047 32347 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:05.075857 32421 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.126734 32434 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.134382 32440 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:05.159710 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 49078.9us
I20260709 13:05:05.159160 29068 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:05.194765 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:05.113981 32197 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:05.138823 32272 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:05.158363 29221 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:05.148082 32416 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:05.202739 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.148909 31621 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:05:05.151909 32170 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.004s
I20260709 13:05:05.199766 32347 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:05.205843 32442 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:05.213214 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.213508 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.214763 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.212010 29064 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:05.215286 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.216463 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.216775 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.158169 32357 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:05.232451 32421 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:05.232333 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:05.236943 32009 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:05.238004 31715 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:05.240516 32419 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:05.242452 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:05.242973 32201 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.244690 29070 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:05.245536 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:05.246949 29067 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:05:05.238018 32178 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:05.250689 32297 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.256498 32448 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:05.293643 32440 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:05.295305 32272 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:05.183667 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.279366 29071 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:05.304061 29213 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:05:05.305904 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.307451 29073 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:05.308583 29214 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:05.309916 29219 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:05.310243 29220 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:05.316579 32009 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:05.257570 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 179413us
I20260709 13:05:05.298555 31621 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:05.317545 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:05.326330 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60879.2us
I20260709 13:05:05.325824 29066 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 6: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:05.334007 29144 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:05.336982 29222 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:05.344583 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44970.9us
I20260709 13:05:05.332319 32272 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:05.326558 32442 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:05.337597 32178 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:05.304203 32170 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:05.331172 32009 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:05.341641 31715 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:05.350240 32436 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:05.360302 31715 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:05.361475 32347 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.362818 29213 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:05.300608 32434 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.363845 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.366901 29213 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:05.359122 29215 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:05.363683 32009 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:05.364346 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:05.372577 29216 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:05.374635 29216 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:05.357509 29146 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:05.381269 29214 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:05.382409 32419 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:05.397610 29219 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:05.398241 29220 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:05.401199 29213 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:05:05.339474 32448 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:05.407061 29067 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:05.404739 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:05.414302 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62763.4us
I20260709 13:05:05.365127 32447 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.375448 32426 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.417537 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:05.384744 32178 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:05.418540 29213 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:05.419234 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:05.421695 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:05.423647 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 228971us
I20260709 13:05:05.424779 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:05.422397 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:05.355062 32337 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:05:05.428139 29216 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:05.429550 29221 raft_consensus.cc:2368] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6.
I20260709 13:05:05.431054 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.432327 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:05.433763 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.435392 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:05.437433 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.438902 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:05.439612 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.439764 29213 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:05.417901 32389 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:05.379801 32430 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.443434 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:05.423039 32431 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:05.420732 32178 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:05.454854 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:05.455497 29073 raft_consensus.cc:2410] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:05:05.456112 29216 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:05.456408 29214 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:05.460312 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:05.461530 29220 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:05.463019 29220 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:05:05.469223 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:05.477672 32434 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:05.484534 32201 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:05.485328 32201 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.485800 32201 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.486195 32197 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:05.445487 32287 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:05.427606 31621 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.486896 32442 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.487869 31621 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:05.488718 32297 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:05.442808 32389 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.489426 32389 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:05.427354 32357 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.493150 32426 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:05.493827 32426 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:05.494158 32426 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:05.494527 32426 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:05.510303 29220 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:05.514664 32448 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.420979 32447 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:05.518894 32009 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:05.525286 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:05.528081 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.529374 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.529980 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:05.530264 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.530686 29219 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:05.530010 29218 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:05:05.532760 29146 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:05.534842 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.535537 29219 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:05.536033 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 179985us
I20260709 13:05:05.537384 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:05.538193 29146 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:05.539180 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:05.539187 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:05.539803 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:05.539391 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:05.540903 29141 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:05.540138 29142 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:05.542447 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:05.542455 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:05.544186 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:05.548211 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:05.548988 29067 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:05.552944 29065 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:05.524948 32416 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:05.537464 32297 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.537597 32351 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.576715 29142 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:05:05.578120 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:05.573706 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:05.584808 29219 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:05.576300 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:05.592734 32201 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:05.545354 32430 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:05.549456 32354 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:05.618539 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 171946us
I20260709 13:05:05.619930 29188 leader_election.cc:400] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9.
I20260709 13:05:05.620265 29188 leader_election.cc:403] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:05.551743 32287 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.626595 29141 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:05.638123 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:05.630633 29067 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:05.551301 32080 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:05:05.603878 31715 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:05.668947 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:05.678932 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:05.683387 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:05.685806 29218 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:05.649956 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:05.607004 32125 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:05.639086 32459 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.090s	user 0.000s	sys 0.002s
W20260709 13:05:05.698159 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 78439.2us
I20260709 13:05:05.616928 32354 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:05.680392 32424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.007s	sys 0.009s
I20260709 13:05:05.668838 32430 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:05.607129 32297 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.685369 32080 consensus_queue.cc:1243] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:05.622208 32347 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:05.686970 32421 raft_consensus.cc:2764] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:05:05.708316 32125 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:05.717165 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:05.709077 32424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.007s	sys 0.009s
I20260709 13:05:05.639552 32425 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:05:05.639433 32337 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
W20260709 13:05:05.721351 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 298991us
I20260709 13:05:05.708014 32354 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:05.722811 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 178684us
I20260709 13:05:05.613174 32351 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:05.650130 32364 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:05:05.689863 31715 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 13:05:05.726966 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.726449 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:05.696165 32287 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:05.755070 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.752452 29220 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:05.756848 29111 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:05.762243 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:05.718251 32416 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:05.773592 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:05.774214 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 248978us
W20260709 13:05:05.776504 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 170215us
I20260709 13:05:05.731292 32009 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:05.754276 31715 consensus_queue.cc:1243] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:05.772701 32424 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:05.766464 32430 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9.
I20260709 13:05:05.717276 32438 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:05:05.800812 29113 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:05.772812 32462 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:05.774894 32347 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:05:05.805740 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 97823.7us
I20260709 13:05:05.798328 32464 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.804318 32416 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:05.821115 32439 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.845952 29064 pending_rounds.cc:85] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 3
I20260709 13:05:05.851361 29064 pending_rounds.cc:107] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:05:05.851681 29064 raft_consensus.cc:2889] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:05.798555 32431 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:05:05.802752 32272 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.867053 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:05.869556 29905 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:05.822953 32440 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:05.877833 32474 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:05.878520 32474 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:05.849298 32464 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:05.881897 32446 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.813s	user 0.000s	sys 0.002s
I20260709 13:05:05.830868 32197 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 13:05:05.867388 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 82039.1us
I20260709 13:05:05.887742 29911 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:05.889048 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:05.892589 29065 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:05.887404 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 178684us
I20260709 13:05:05.899578 32287 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:05.906709 29905 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:05.909724 29073 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:05.881922 32464 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:05.899734 32474 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:05.900229 32467 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:05.916496 32440 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:05.906996 32469 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:05:05.832540 32462 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:05.898960 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:05.900080 32463 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:05:05.863086 32472 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:05.870432 32416 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:05.833174 32125 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:05.973806 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 74914.7us
I20260709 13:05:05.983709 29188 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:05.919394 32462 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:05.984839 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:05.965822 32357 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.972508 32440 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:06.011042 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.862561 32439 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.022332 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:05.976848 32297 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.029235 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:05.965508 32438 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:05:06.032277 29146 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.001791 32287 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:06.047472 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 79624us
I20260709 13:05:06.049146 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.049993 29143 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:06.049851 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.049379 29144 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:06.018412 32431 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:06.050406 29139 raft_consensus.cc:2376] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10. Re-sending same reply.
I20260709 13:05:06.021474 32462 raft_consensus.cc:2764] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:05:06.060868 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:06.064453 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.031497 32351 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.102836 29065 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:06.104912 29188 leader_election.cc:400] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:06.106109 29146 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:06.107230 29188 leader_election.cc:403] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:05:06.108774 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 308948us
I20260709 13:05:06.110594 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:06.112344 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.113255 29141 raft_consensus.cc:2410] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:06.113662 32485 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.114253 32485 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.116940 32478 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:06.117429 32478 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:06.118256 32478 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.119369 32462 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:06.120285 32482 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.103s
W20260709 13:05:06.121222 32178 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.000s
I20260709 13:05:06.121658 32178 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.054548 32440 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.124228 29213 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:06.124316 32334 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:06.124902 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:06.124970 32334 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.125871 32440 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:06.126360 32178 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.126423 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:06.127887 32178 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:06.128283 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:06.133149 32272 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.023439 32357 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:06.135238 32471 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:05:06.135620 32357 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.110901 32416 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:06.060333 32431 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:06.145529 32431 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:06.147213 32426 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.652s	user 0.005s	sys 0.005s
I20260709 13:05:06.149547 32354 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:06.153785 32389 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.664s	user 0.008s	sys 0.003s
I20260709 13:05:06.154352 32389 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.156759 32354 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.164544 32462 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:06.165169 32462 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:06.165567 32462 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:06.167548 32485 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:06.164664 32334 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.126451 32440 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:06.123901 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:06.172431 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44221.7us
I20260709 13:05:06.157559 32439 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:06.174932 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.189098 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:06.192456 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 68622.3us
I20260709 13:05:06.193436 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:06.147655 32426 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.174168 32197 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:06.200739 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 455) took 1584 ms. Trace:
W20260709 13:05:06.202059 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 92141.1us
I20260709 13:05:06.203143 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:06.208333 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.201292 29147 rpcz_store.cc:276] 0709 13:05:04.613131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.613343 (+   212us) service_pool.cc:224] Handling call
0709 13:05:06.189211 (+1575868us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400191754. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:06.197529 (+  8318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791743360}
I20260709 13:05:06.216967 32354 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:06.218161 32287 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:06.197105 32439 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:06.193384 32351 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.225028 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:06.219805 32467 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:06.234557 32354 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:06.239292 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 50343.1us
I20260709 13:05:06.219358 32389 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:06.241916 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64393.9us
I20260709 13:05:06.241061 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:06.235486 32351 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.239013 32482 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:06.248169 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:06.246834 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:06.237771 32467 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:06.250574 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:06.249459 32482 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:06.250787 32468 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:05:06.252852 32351 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.255492 32467 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:06.256877 32482 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:06.256628 32347 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:06.246948 32439 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:06.271817 32439 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:06.274119 29038 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:06.272369 32439 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:06.276870 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:06.280547 29064 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.282701 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:06.285190 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:06.286266 29065 raft_consensus.cc:2410] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:06.254733 32354 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:06.287686 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.289345 29146 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:06.291817 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.292528 29141 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:06.296856 29073 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:06.300549 29065 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:06.302579 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.250083 32125 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:05:06.286523 32437 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:06.311580 29113 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:06.306813 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:06.306555 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:06.312155 29147 raft_consensus.cc:2368] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:05:06.282924 32347 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:05:06.299015 32389 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:06.324239 32491 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:06.324434 32389 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:06.324780 32491 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.325223 32491 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:06.325802 32389 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.329027 32486 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:05:06.292106 32351 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:05:06.330691 32351 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:06.331135 32351 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:06.369199 29143 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:06.297991 32182 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:06.287235 32462 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.379698 32297 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:06.378108 32437 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:06.401799 32437 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.403667 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2007) took 1188 ms. Trace:
I20260709 13:05:06.404326 29221 rpcz_store.cc:276] 0709 13:05:05.215534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.305609 (+ 90075us) service_pool.cc:224] Handling call
0709 13:05:06.403420 (+1097811us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54000be854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:06.403573 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231917312}
I20260709 13:05:06.397790 32491 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:06.412314 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 105867us
I20260709 13:05:06.393208 29141 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:06.314127 32426 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:06.426534 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 135124us
I20260709 13:05:06.427727 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:06.432919 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:06.433024 29064 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:06.410706 32416 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:06.406942 32435 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:06.440312 32182 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:06.408126 32351 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.461800 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2108) took 1030 ms. Trace:
W20260709 13:05:06.463210 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 49694.1us
I20260709 13:05:06.447806 32426 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:06.446585 32435 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:06.450703 32416 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:06.484719 32491 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.484687 32435 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.484800 32416 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.486338 32426 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.486928 32426 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.476501 29216 rpcz_store.cc:276] 0709 13:05:05.431214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.471443 (+ 40229us) service_pool.cc:224] Handling call
0709 13:05:05.472397 (+   954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.511991 (+ 39594us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:05.535463 (+ 23472us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.036020 (+500557us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:06.058447 (+ 22427us) log.cc:844] Serialized 26 byte log entry
0709 13:05:06.058598 (+   151us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:06.461176 (+402578us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.461430 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.461535 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.461577 (+    42us) raft_consensus.cc:1685] finished
0709 13:05:06.461589 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.461734 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25084,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":2740,"raft.queue_time_us":33091,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":9515,"spinlock_wait_cycles":8361856,"thread_start_us":794150,"threads_started":2,"wal-append.queue_time_us":585531}
I20260709 13:05:06.489280 32426 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.491153 32426 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:06.493642 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 182103us
I20260709 13:05:06.482177 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:06.494599 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:06.498239 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:06.474575 32182 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:06.439312 32482 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.500128 32287 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:06.503660 32178 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.509443 32178 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:06.510025 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.504908 32500 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:06.510803 32500 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:06.511585 32178 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.505077 32182 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:06.511868 32500 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.513581 32462 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:06.516812 29217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1984) took 1758 ms (client timeout 2140 ms). Trace:
W20260709 13:05:06.524372 29217 rpcz_store.cc:269] 0709 13:05:04.758282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.764371 (+  6089us) service_pool.cc:224] Handling call
0709 13:05:06.516582 (+1752211us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54002f5dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:06.516746 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771157888}
I20260709 13:05:06.524921 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:06.523721 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:06.527791 31621 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:06.528263 31621 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.528697 31621 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.528992 29113 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:06.492103 32426 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.528167 32287 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:06.542932 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:06.532876 32448 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.014s	user 0.007s	sys 0.012s
W20260709 13:05:06.544376 32448 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.008s	sys 0.012s
I20260709 13:05:06.547390 32507 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:06.548211 32433 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:06.548725 32433 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.550720 32433 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:06.556687 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:06.536901 32347 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.538471 32496 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:06.559587 32482 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.560269 32287 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:06.560846 32448 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.524196 32467 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.567715 32496 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:06.568224 32287 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:06.568869 32496 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.569305 32287 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.569861 32467 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.570569 29065 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:06.558054 31621 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.541865 32491 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:06.541621 32501 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:06.575052 32501 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.576325 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:06.589509 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:06.575875 32426 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.572418 32347 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.591166 31621 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.600265 32347 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.601843 32491 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:06.602489 32480 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.602977 32447 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.603060 32480 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.604274 32507 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:06.604753 32439 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.605996 32480 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:06.606813 32501 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.608145 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2008) took 1392 ms. Trace:
I20260709 13:05:06.611405 29215 rpcz_store.cc:276] 0709 13:05:05.215870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.369338 (+153468us) service_pool.cc:224] Handling call
0709 13:05:05.369654 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.369928 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:05.405173 (+ 35245us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.597971 (+1192798us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b18004733e0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a
0709 13:05:06.607775 (+  9804us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:06.607826 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:06.607871 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.607965 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.608085 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":69759,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":2534,"raft.queue_time_us":10235,"spinlock_wait_cycles":7198592,"thread_start_us":80403,"threads_started":2,"wal-append.queue_time_us":65038,"wal-append.run_cpu_time_us":1243,"wal-append.run_wall_time_us":1001492}
I20260709 13:05:06.612896 32507 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.613037 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:06.613386 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:06.613495 32507 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.613858 29902 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:06.620147 32491 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.620884 32491 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:06.622488 32495 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:05:06.623772 32491 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.592021 32389 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:06.631377 29141 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:06.634757 31621 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:06.636624 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.642760 31621 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.643347 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.644873 32491 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.645028 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.645728 32491 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.646407 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.648378 31621 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.574071 32448 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.648726 32491 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.650146 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.650332 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:06.650761 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:06.651351 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.651463 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:06.652096 29142 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:06.651952 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:06.652778 29071 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:06.653681 32448 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:06.654647 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:06.655248 29067 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:06.580200 32482 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:05:06.658344 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.659320 29148 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:06.660624 29148 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:05:06.642442 32357 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.662006 32357 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.662519 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:06.662834 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:06.663273 29073 raft_consensus.cc:2368] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9.
I20260709 13:05:06.663453 29070 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:06.652997 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:06.664072 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:06.664721 29073 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:06.665369 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:06.665535 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:06.666641 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:06.668279 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:06.681758 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:06.682802 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:06.683261 29066 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:06.668455 29185 leader_election.cc:400] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9.
I20260709 13:05:06.669103 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:06.669780 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:06.670707 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:06.670850 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:06.683907 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:06.685242 29904 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:05:06.685575 29066 raft_consensus.cc:2376] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8. Re-sending same reply.
I20260709 13:05:06.671353 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:06.686473 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:06.686904 32498 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:05:06.687021 29066 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:05:06.672245 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:06.673671 29140 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:06.674042 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:06.669782 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:06.684832 29902 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.688828 29903 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:06.688902 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:06.674540 29222 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 1987) took 1829 ms (client timeout 1530 ms). Trace:
I20260709 13:05:06.684991 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:06.674935 29214 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:05:06.685006 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:06.675809 32451 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.171s	user 0.007s	sys 0.004s
W20260709 13:05:06.689662 29222 rpcz_store.cc:269] 0709 13:05:04.845051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.847641 (+  2590us) service_pool.cc:224] Handling call
0709 13:05:06.674328 (+1826687us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b5400339fd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:06.674467 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1112479488}
I20260709 13:05:06.693703 29148 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:06.674261 29071 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:06.680158 32364 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:06.686786 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:06.672935 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:06.690269 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:06.690910 29068 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:05:06.691331 29141 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:06.648938 31621 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.684609 29072 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:06.697417 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:06.697515 31621 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.678232 32182 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.606828 32439 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.702178 29067 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:05:06.702728 29070 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
W20260709 13:05:06.658280 29871 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.590s (SENT) [suppressed 31 similar messages]
I20260709 13:05:06.718104 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:06.718393 29902 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:06.703406 29112 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:06.705354 32065 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:06.705544 29073 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:06.719146 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:06.696535 32364 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:06.695816 32416 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:06.704515 29905 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:06.724864 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:06.727609 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:06.707428 32182 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.705581 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.708695 32436 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.709296 32066 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.695464 32201 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:06.703560 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.711679 31621 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.719591 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:06.718607 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.695111 29148 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:05:06.738577 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.703755 32448 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:06.732343 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:06.733616 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:06.737147 29142 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
W20260709 13:05:06.753969 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.713806 32357 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.750566 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 1983) took 1653 ms. Trace:
W20260709 13:05:06.757694 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.755864 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:06.758155 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.755885 29186 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:06.757704 29069 rpcz_store.cc:276] 0709 13:05:05.096973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.109104 (+ 12131us) service_pool.cc:224] Handling call
0709 13:05:05.137483 (+ 28379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.140184 (+  2701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:05.140211 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:05.149459 (+  9248us) log.cc:844] Serialized 26 byte log entry
0709 13:05:05.342341 (+192882us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:05.342382 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.342447 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.675099 (+1332652us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:06.728657 (+ 53558us) raft_consensus.cc:1685] finished
0709 13:05:06.728684 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.750450 (+ 21766us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31560960,"thread_start_us":136017,"threads_started":1,"wal-append.queue_time_us":225008}
I20260709 13:05:06.729698 32458 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:06.730652 32439 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:06.726615 32080 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.765000 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.766578 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:06.766283 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 77335.9us
I20260709 13:05:06.750025 32462 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:06.768291 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:06.774452 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.751581 32201 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.776149 29140 raft_consensus.cc:2368] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8.
W20260709 13:05:06.782691 29113 leader_election.cc:336] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 2.140s (SENT)
I20260709 13:05:06.760792 32448 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:06.755488 32437 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:06.783308 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:06.784403 32352 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:06.784757 32452 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:05:06.782466 32156 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:06.782568 32182 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:05:06.735579 32432 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:05:06.766572 32178 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:05:06.814651 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 156390us
I20260709 13:05:06.815256 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.802027 32448 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:06.784480 32201 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.822790 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.783864 32439 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:06.817000 32156 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.766644 32426 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
W20260709 13:05:06.827870 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 59673.6us
I20260709 13:05:06.799414 32437 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.831969 29140 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:06.845760 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:06.750070 29222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:05:06.846876 29222 raft_consensus.cc:606] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:06.847500 29222 raft_consensus.cc:614] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:06.827072 32389 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:06.867700 32389 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:06.868882 32389 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.870570 32389 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:06.827072 32504 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:05:06.874667 29145 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:06.783464 32080 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:06.876412 32448 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.877862 32080 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:06.847594 32437 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:06.878647 32448 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.884593 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.885458 29144 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:06.892582 32507 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:06.896632 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 128056us
I20260709 13:05:06.898764 32478 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:06.900578 32435 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.837966 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:06.904853 29219 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.838451 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:06.909008 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:06.910111 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:06.910781 29218 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:06.814270 32352 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:06.838037 29186 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:06.912166 32352 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:06.912456 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 74534.9us
I20260709 13:05:06.839584 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:06.917299 29217 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:06.918479 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:06.920504 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.921398 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.921733 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.783267 32462 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.846882 32337 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:05:06.850363 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:06.952666 29216 raft_consensus.cc:2368] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:05:06.856778 32480 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:06.850190 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:06.827780 32447 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:06.859127 32439 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.860114 29140 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:05:06.958120 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.958858 29140 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:06.860424 32518 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.861768 32452 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.845515 32170 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:06.836911 32156 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.835541 32178 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:06.865857 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:06.827311 32065 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:06.907238 32364 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:06.916549 32517 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:06.922286 29141 raft_consensus.cc:2410] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:05:06.923056 29038 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:06.980288 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:06.981034 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 58078us
I20260709 13:05:06.982200 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:06.982379 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:06.983040 29216 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:06.983498 29038 leader_election.cc:400] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:05:06.849824 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:06.925990 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.984747 29146 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:06.927039 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.984875 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:06.985437 29147 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:06.928104 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.840988 29221 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:05:06.986443 29910 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:05:06.987016 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:06.930006 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:06.952169 32462 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.931368 32418 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:06.956897 32480 consensus_queue.cc:1243] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:06.911582 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:06.991444 29218 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:06.991209 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:06.992370 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:06.993618 29142 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:06.932621 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:06.994910 29143 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:05:06.942267 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.000029 32272 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:07.000689 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2109) took 1569 ms. Trace:
I20260709 13:05:07.001042 32507 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:07.001703 32507 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:07.002425 32507 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:07.003753 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:07.004382 32351 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:06.990420 32418 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:07.007064 29139 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:05:07.007483 29139 pending_rounds.cc:85] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 6
I20260709 13:05:07.007732 29139 pending_rounds.cc:107] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:05:07.010680 29139 raft_consensus.cc:2889] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:06.953624 29222 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:06.954085 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.939163 29219 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:07.012940 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:07.013518 29219 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:06.961169 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:07.018282 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:06.951637 29145 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:06.962795 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:07.018919 29216 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.964026 32357 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:06.976279 29217 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:06.934873 32383 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:06.983840 29038 leader_election.cc:403] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:05:07.004456 29220 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:07.008823 29141 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:05:07.021407 29219 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:06.939551 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:07.159437 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 324863us
I20260709 13:05:07.044909 32458 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:07.001242 29214 rpcz_store.cc:276] 0709 13:05:05.431518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.470829 (+ 39311us) service_pool.cc:224] Handling call
0709 13:05:06.674708 (+1203879us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:06.690123 (+ 15415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.000413 (+310290us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400339fd0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:07.000606 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2699942272}
I20260709 13:05:06.956938 32470 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:05:06.969265 32518 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.022074 32468 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:06.976460 32452 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.169759 29216 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:06.962239 32080 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:05:07.063486 32178 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:07.192261 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.154400 29140 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:07.156746 29222 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:07.157425 29218 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:07.045178 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.961612 29144 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
W20260709 13:05:07.160863 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 177480us
I20260709 13:05:07.174690 29221 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:07.157327 32436 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:07.059052 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 96327us
I20260709 13:05:07.174253 32297 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:05:07.185913 32352 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.186749 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:07.196197 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 256703us
I20260709 13:05:07.205210 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:07.210906 29038 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:07.175705 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:07.165524 32426 consensus_queue.cc:1243] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:07.160466 32383 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:07.157938 32440 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:07.231854 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:07.234645 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:07.236464 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60858.9us
I20260709 13:05:07.243137 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:07.181780 32468 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:07.252422 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:07.241791 32197 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:07.185686 32462 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.231169 32416 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:07.220707 32337 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:07.279035 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:07.280166 29871 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:07.280870 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:07.281067 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:07.281617 29220 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:07.285669 29216 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c02e27bf25948329604215b11034f44"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:05:07.286859 29216 raft_consensus.cc:606] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:07.287228 29216 raft_consensus.cc:614] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:07.293983 29218 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:07.297392 29216 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:07.268692 32447 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:07.301168 32468 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.303550 32462 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:07.312888 32452 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:07.251189 32287 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:05:07.276197 32197 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.316560 32521 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:07.317095 32352 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:07.250391 32432 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:05:07.195948 32458 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.342057 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:07.367403 29220 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:07.316555 32518 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:07.344444 32383 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:07.346607 32201 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.501s	user 0.007s	sys 0.008s
I20260709 13:05:07.347885 29213 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:07.271832 32440 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.350764 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:07.274539 32470 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:05:07.317353 32520 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.353750 32448 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:07.356773 32478 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:07.295867 32357 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:07.343084 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 94179us
I20260709 13:05:07.364753 32197 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:07.291755 32354 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
W20260709 13:05:07.373060 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 138190us
I20260709 13:05:07.386672 29220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:05:07.390599 29220 raft_consensus.cc:606] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:07.375660 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 163272us
I20260709 13:05:07.288358 32337 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.407028 29222 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:07.380297 29216 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:05:07.342041 32522 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:05:07.412242 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 178052us
I20260709 13:05:07.416376 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:07.420680 29065 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:07.388471 32519 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.387672 32448 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:07.425930 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:07.426734 29140 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:07.412817 32347 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.436082 32347 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.438031 32440 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:07.438619 32347 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:07.406003 32352 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:05:07.442575 32337 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:07.444370 32347 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:07.444854 32347 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.445225 32347 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:07.390233 32520 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.447938 32520 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:07.377512 32507 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.409981 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:07.414633 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.414803 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:07.454035 29070 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:07.377942 32201 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.411067 32426 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:07.369318 32521 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.454097 29073 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:07.448989 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:07.462775 29140 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:07.413704 29216 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:05:07.474432 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.474298 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:07.480767 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70935.5us
W20260709 13:05:07.488286 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 243369us
I20260709 13:05:07.429356 32519 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.450575 32156 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:07.510226 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.507130 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.516253 29070 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:07.512485 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.513836 29039 leader_election.cc:400] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:05:07.517858 29067 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:07.518748 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:07.523288 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:07.505141 29073 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:07.494979 32517 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:07.520308 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:07.495874 32529 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:05:07.571157 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50936.2us
I20260709 13:05:07.467964 32448 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.455127 32447 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.580703 32517 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.509711 32351 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:05:07.486703 32426 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.585407 32426 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:07.575188 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:07.587147 32527 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:07.587886 32527 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.588397 32527 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:07.593122 32507 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:07.598706 32347 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.577901 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.623721 29065 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:07.580070 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.624966 29070 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:05:07.577975 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:07.626202 32510 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.794s	user 0.009s	sys 0.000s
I20260709 13:05:07.609126 32201 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:07.629746 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:07.630591 29217 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:07.644397 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 255155us
I20260709 13:05:07.645187 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:07.646641 29067 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:05:07.613255 32521 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:07.615144 32156 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:07.561285 32523 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:05:07.652837 29067 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:07.561537 32272 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:07.616765 32482 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:07.618973 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 43850.5us
I20260709 13:05:07.619643 32426 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:07.619316 32533 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:07.620359 32529 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:07.620639 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:07.622747 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:07.675787 32521 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:07.676532 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55998.1us
I20260709 13:05:07.676723 32178 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.677570 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:07.677825 32440 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:07.679132 32440 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:07.679157 29188 leader_election.cc:400] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8.
I20260709 13:05:07.677599 32178 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.681834 29217 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:07.572225 32436 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:07.652935 29871 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:07.686189 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:07.686548 32527 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.659240 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.660490 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:07.673954 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:07.680593 29188 leader_election.cc:403] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:05:07.690093 29065 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:07.690120 29214 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:07.691118 29871 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:07.610157 32197 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.618844 32520 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:07.676541 32521 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.684140 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.682783 32178 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:07.689049 29070 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:07.694458 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.695600 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:07.694360 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.796368 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:07.855157 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:07.697506 32337 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:07.671861 32437 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.793s	user 0.013s	sys 0.000s
I20260709 13:05:07.732287 32197 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:07.851547 32182 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:07.786942 32389 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:07.694599 32426 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:07.733880 32529 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:07.793346 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 175088us
I20260709 13:05:07.656824 32432 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:07.809918 29904 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:07.780750 29903 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:07.838220 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:07.869516 29902 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:07.865541 29071 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:07.803634 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:07.883174 29908 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:05:07.863565 29067 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:07.877401 29144 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:07.688185 32540 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:07.787093 29214 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:07.868251 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:07.890148 29068 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:05:07.775758 32507 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.878671 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:07.872756 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:07.895515 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.875211 29139 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:05:07.865001 29907 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:07.785190 29910 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:07.832382 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 168596us
I20260709 13:05:07.877439 29141 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:07.897066 29070 raft_consensus.cc:2410] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:05:07.898291 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.705967 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.811000 29065 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:07.900861 29065 pending_rounds.cc:85] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 6
I20260709 13:05:07.901139 29065 pending_rounds.cc:107] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:05:07.901369 29065 raft_consensus.cc:2889] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:07.874473 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:07.866792 29073 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:07.885289 29146 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:05:07.873348 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 177824us
I20260709 13:05:07.748077 32351 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:07.886709 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:07.904158 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:07.904687 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:07.890396 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:07.905371 29144 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:07.866345 32448 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:07.781188 32347 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:07.862623 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:07.870805 32542 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.696748 32352 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:07.815580 32357 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:07.734997 32156 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8.
I20260709 13:05:07.734584 32272 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:07.893759 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.866565 32182 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.869654 32519 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:07.879813 32447 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:07.865399 32437 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.864691 32337 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:07.906416 29185 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:07.869151 32389 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.911255 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:07.911906 29214 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:05:07.903821 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:07.937091 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2023) took 1073 ms. Trace:
I20260709 13:05:07.937620 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:07.870790 32354 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.690009 32482 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.915716 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:07.888830 32540 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:07.910156 32537 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:05:07.925365 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:07.955024 29215 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:07.908243 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:07.815328 32436 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.904192 32433 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.910472 29217 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:07.922060 32178 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:07.904460 32351 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:07.909183 32197 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:07.871706 32529 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:07.866987 32520 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.919889 32352 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:07.935400 29073 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:07.937515 29214 rpcz_store.cc:276] 0709 13:05:06.863201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.911330 (+1048129us) service_pool.cc:224] Handling call
0709 13:05:07.911673 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:07.936743 (+ 25070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.936971 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:07.923326 32253 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.943013 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:07.910074 32501 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.024s
W20260709 13:05:07.932827 32541 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:05:07.909721 32523 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:05:07.947518 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:07.911245 32507 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.909950 32548 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:07.869035 32426 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.953665 32535 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:07.968526 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 60377us
I20260709 13:05:08.053848 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:08.055097 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:07.994552 29071 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:05:07.952185 32447 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:07.995110 29217 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:07.921264 32542 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.929558 32543 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 13:05:07.999666 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62892.4us
I20260709 13:05:07.996208 32433 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.960773 32527 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:08.023469 32352 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:07.969177 32534 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:05:08.084165 29071 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:07.990309 32536 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:05:07.993705 32178 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:07.957109 32354 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:08.027653 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 84768.7us
W20260709 13:05:08.078955 32544 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:05:07.969532 32458 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:07.954487 32540 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:07.970958 32347 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:08.056741 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.058245 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:08.034448 32517 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:08.088941 29071 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:05:08.124874 29071 pending_rounds.cc:85] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 9
W20260709 13:05:08.108664 32545 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.195s	user 0.001s	sys 0.001s
I20260709 13:05:08.026549 32197 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:07.956646 32357 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:08.088183 32548 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:08.032425 32253 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.093284 32178 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:08.105811 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:08.059626 32549 raft_consensus.cc:2764] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:05:08.061411 32156 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:08.061494 32447 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:08.081108 32519 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:08.122751 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64619.2us
I20260709 13:05:08.155522 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:08.156103 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 72444.4us
I20260709 13:05:08.156872 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.158114 29188 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.125154 29071 pending_rounds.cc:107] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:05:08.158812 29071 raft_consensus.cc:2889] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:08.086982 32521 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:05:08.061669 29217 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:08.160399 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2025) took 1296 ms. Trace:
I20260709 13:05:08.160902 29217 rpcz_store.cc:276] 0709 13:05:06.863729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.061013 (+1197284us) service_pool.cc:224] Handling call
0709 13:05:08.061418 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.160107 (+ 98689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.160333 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.125767 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:08.162349 29904 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:08.126511 29145 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:08.052130 32535 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.031961 32272 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:08.131866 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 184426us
I20260709 13:05:08.138199 29064 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:08.139092 32552 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 13:05:08.139417 32546 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:05:08.139941 32557 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:05:08.143637 32520 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:08.145660 32354 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:08.145598 32178 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.061650 32337 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.177440 32535 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:08.059607 32351 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.176144 29064 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:08.180564 32458 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:08.130200 32517 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:08.129935 32357 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:08.159973 32523 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:08.159754 32447 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:08.126099 32347 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:08.149499 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55576.2us
W20260709 13:05:08.212368 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.169911 32182 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:08.177161 32440 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:08.202764 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:08.177750 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:08.199832 32437 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:08.160009 32548 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.210868 32523 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.208684 32357 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:08.214802 32351 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:08.230144 32551 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:05:08.230264 32539 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:08.230758 32565 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:08.221752 32521 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:08.231568 32416 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:08.230753 32542 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:08.232988 32554 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:08.234819 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2022) took 1371 ms. Trace:
I20260709 13:05:08.214978 32482 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:08.201514 32354 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:05:08.196674 32528 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:05:08.294723 32507 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:08.250303 32562 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:05:08.236223 32523 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:08.249926 29216 rpcz_store.cc:276] 0709 13:05:06.862942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.487665 (+624723us) service_pool.cc:224] Handling call
0709 13:05:07.489524 (+  1859us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:07.496294 (+  6770us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:07.496345 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:07.509950 (+ 13605us) log.cc:844] Serialized 26 byte log entry
0709 13:05:07.816130 (+306180us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:07.816188 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.816283 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.234558 (+418275us) raft_consensus.cc:1685] finished
0709 13:05:08.234583 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.234751 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":129,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":14082,"thread_start_us":235982,"threads_started":1,"wal-append.queue_time_us":280371}
I20260709 13:05:08.298400 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:08.250829 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 78000.6us
I20260709 13:05:08.299809 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:08.245092 32416 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:08.300982 32367 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:08.301555 32416 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.302392 32367 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:08.302822 32416 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:08.303069 32367 consensus_queue.cc:1243] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:05:08.303591 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.304041 32367 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:08.304169 32436 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:08.304488 32367 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:08.305270 32367 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.238660 32437 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:05:08.226840 32529 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:08.244081 32565 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.307163 32565 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:08.285367 29218 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:05:08.308197 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2128) took 1465 ms. Trace:
I20260709 13:05:08.310882 29216 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:08.308591 29218 rpcz_store.cc:276] 0709 13:05:06.843056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.294869 (+451813us) service_pool.cc:224] Handling call
0709 13:05:08.265483 (+970614us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:05:08.265717 (+   234us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.307877 (+ 42160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.308137 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9132288}
I20260709 13:05:08.330057 29218 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:08.289427 32517 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:08.275650 32547 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.423s	user 0.003s	sys 0.000s
I20260709 13:05:08.337634 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2027) took 1473 ms. Trace:
I20260709 13:05:08.295025 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:08.305667 32436 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:08.239151 32440 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:05:08.250977 32554 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:08.338227 29219 rpcz_store.cc:276] 0709 13:05:06.864314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.336517 (+1472203us) service_pool.cc:224] Handling call
0709 13:05:08.336934 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.337183 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.337221 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.337281 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:08.337329 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.337400 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.337551 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.360157 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2026) took 1496 ms. Trace:
I20260709 13:05:08.360669 29218 rpcz_store.cc:276] 0709 13:05:06.864006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.314429 (+1450423us) service_pool.cc:224] Handling call
0709 13:05:08.314760 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.334931 (+ 20171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.360070 (+ 25139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.361194 29219 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:08.361960 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2034) took 1495 ms. Trace:
I20260709 13:05:08.362339 29219 rpcz_store.cc:276] 0709 13:05:06.866264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.360402 (+1494138us) service_pool.cc:224] Handling call
0709 13:05:08.360766 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.361668 (+   902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.361896 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.364235 29219 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:08.365043 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 483) took 1497 ms. Trace:
I20260709 13:05:08.365518 29219 rpcz_store.cc:276] 0709 13:05:06.867442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.363359 (+1495917us) service_pool.cc:224] Handling call
0709 13:05:08.364007 (+   648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.364755 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.364979 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:08.366557 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 152184us
I20260709 13:05:08.299525 32389 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:08.358824 29216 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:08.367834 29217 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:08.250826 32527 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:08.368206 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2033) took 1502 ms. Trace:
I20260709 13:05:08.368556 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2032) took 1502 ms. Trace:
I20260709 13:05:08.369066 29217 rpcz_store.cc:276] 0709 13:05:06.865782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.342211 (+1476429us) service_pool.cc:224] Handling call
0709 13:05:08.367650 (+ 25439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.368322 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.368502 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.370173 29216 rpcz_store.cc:276] 0709 13:05:06.866027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.346304 (+1480277us) service_pool.cc:224] Handling call
0709 13:05:08.346728 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.367786 (+ 21058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.368103 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.373875 32418 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.375389 32418 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:08.358518 32436 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:08.380851 29217 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:08.389027 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 486) took 1514 ms. Trace:
I20260709 13:05:08.389506 29217 rpcz_store.cc:276] 0709 13:05:06.874427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.369956 (+1495529us) service_pool.cc:224] Handling call
0709 13:05:08.380627 (+ 10671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.388633 (+  8006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.388948 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.249838 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2024) took 1386 ms. Trace:
I20260709 13:05:08.400013 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 484) took 1532 ms. Trace:
I20260709 13:05:08.400820 29219 rpcz_store.cc:276] 0709 13:05:06.867733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.366470 (+1498737us) service_pool.cc:224] Handling call
0709 13:05:08.366742 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.370001 (+  3259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.370839 (+   838us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.371225 (+   386us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:08.371454 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.371517 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.399856 (+ 28339us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:08.406940 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 117206us
I20260709 13:05:08.390720 29214 rpcz_store.cc:276] 0709 13:05:06.863476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.053712 (+1190236us) service_pool.cc:224] Handling call
0709 13:05:08.054297 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.054638 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:08.201818 (+147180us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.249191 (+ 47373us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.249286 (+    95us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:08.249356 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.249484 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.249727 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1309,"prepare.queue_time_us":180469,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":2235,"raft.queue_time_us":3311,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":108,"thread_start_us":158738,"threads_started":2}
I20260709 13:05:08.409762 29214 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:08.410473 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2037) took 1505 ms. Trace:
W20260709 13:05:08.408162 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 67564.1us
I20260709 13:05:08.412181 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:08.412488 29148 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:05:08.412590 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.410977 29214 rpcz_store.cc:276] 0709 13:05:06.905414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.409258 (+1503844us) service_pool.cc:224] Handling call
0709 13:05:08.409583 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.410236 (+   653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.410421 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.414335 29214 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:08.415221 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2133) took 1481 ms. Trace:
I20260709 13:05:08.415555 29214 rpcz_store.cc:276] 0709 13:05:06.933887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.413872 (+1479985us) service_pool.cc:224] Handling call
0709 13:05:08.414167 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.414837 (+   670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.415147 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.417047 29214 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:08.417629 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2134) took 1211 ms. Trace:
I20260709 13:05:08.417929 29214 rpcz_store.cc:276] 0709 13:05:07.206822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.416455 (+1209633us) service_pool.cc:224] Handling call
0709 13:05:08.416708 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.417379 (+   671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.417581 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.420886 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.410224 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:08.421830 29145 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:08.424332 32572 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:08.432008 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.432857 29214 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:08.434100 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2135) took 1040 ms. Trace:
I20260709 13:05:08.434507 29214 rpcz_store.cc:276] 0709 13:05:07.393723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.418677 (+1024954us) service_pool.cc:224] Handling call
0709 13:05:08.421595 (+  2918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.433830 (+ 12235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.434046 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.440809 29217 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:08.444677 29218 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:05:08.422699 32367 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:08.448908 29214 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:05:08.451277 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.451874 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.448988 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 488) took 1574 ms. Trace:
I20260709 13:05:08.454100 29145 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:08.456629 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2041) took 1013 ms. Trace:
I20260709 13:05:08.457367 29217 rpcz_store.cc:276] 0709 13:05:06.875836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.390455 (+1514619us) service_pool.cc:224] Handling call
0709 13:05:08.440558 (+ 50103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.441543 (+   985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.448920 (+  7377us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.457543 29214 rpcz_store.cc:276] 0709 13:05:07.443370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.435278 (+991908us) service_pool.cc:224] Handling call
0709 13:05:08.435619 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.456175 (+ 20556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.456487 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8960}
I20260709 13:05:08.454879 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:08.459676 29148 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:05:08.431080 32478 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:08.467351 29219 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:08.473209 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 489) took 1596 ms. Trace:
I20260709 13:05:08.473587 29219 rpcz_store.cc:276] 0709 13:05:06.876420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.401987 (+1525567us) service_pool.cc:224] Handling call
0709 13:05:08.467007 (+ 65020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.472856 (+  5849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.473119 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.430094 32496 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:08.419399 32354 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:08.420626 32432 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:08.454763 32197 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:08.455686 32536 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:05:08.468355 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 243959us
I20260709 13:05:08.447213 32584 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:05:08.477725 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2042) took 1033 ms. Trace:
I20260709 13:05:08.480567 29217 rpcz_store.cc:276] 0709 13:05:07.444085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.467968 (+1023883us) service_pool.cc:224] Handling call
0709 13:05:08.477072 (+  9104us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.477309 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.477333 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.477381 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:08.477420 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.477485 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.477645 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.481282 29188 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.483556 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:08.471985 32253 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:08.487447 29219 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:08.493067 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.484230 32539 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:08.497740 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:08.491698 32548 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:08.498577 29145 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:08.499729 32352 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.500383 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:08.490649 32536 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:08.503516 32536 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:08.504267 32577 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:08.506598 32539 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:08.507375 32539 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:08.510411 32578 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:08.512653 29221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2029) took 1647 ms (client timeout 1871 ms). Trace:
I20260709 13:05:08.513080 32579 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:08.513162 29221 rpcz_store.cc:269] 0709 13:05:06.864947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.986767 (+121820us) service_pool.cc:224] Handling call
0709 13:05:08.512431 (+1525664us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540033aed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:08.512572 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430079360}
I20260709 13:05:08.514616 29216 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:08.515180 29221 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:08.482544 32432 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:08.520987 32582 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:05:08.526464 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:08.527503 29148 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:08.534675 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:08.494808 32347 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:08.473585 32436 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.521050 32170 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:08.568418 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:08.570380 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:08.490038 32354 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.572723 32170 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:08.576746 32170 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.578126 32170 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:08.585024 29213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 485) took 1717 ms (client timeout 1853 ms). Trace:
W20260709 13:05:08.585564 29213 rpcz_store.cc:269] 0709 13:05:06.868037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.984625 (+116588us) service_pool.cc:224] Handling call
0709 13:05:08.584800 (+1600175us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54002f62d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:08.584959 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":833169280}
I20260709 13:05:08.586756 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:08.504562 32536 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.590462 32364 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:08.593961 32468 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:08.575018 32352 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:08.612738 32080 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.614507 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:08.619330 32436 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:08.622927 32590 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:05:08.628129 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2118) took 1789 ms. Trace:
I20260709 13:05:08.628645 29215 rpcz_store.cc:276] 0709 13:05:06.838934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.839075 (+   141us) service_pool.cc:224] Handling call
0709 13:05:07.448856 (+609781us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8eee 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:05:08.627778 (+1178922us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54000bd454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:08.628048 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120787456}
I20260709 13:05:08.634074 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:08.635040 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 482) took 1767 ms. Trace:
W20260709 13:05:08.635468 29222 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2036) took 1768 ms (client timeout 2146 ms). Trace:
I20260709 13:05:08.635617 29218 rpcz_store.cc:276] 0709 13:05:06.867192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.361529 (+1494337us) service_pool.cc:224] Handling call
0709 13:05:08.361894 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.445207 (+ 83313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.634610 (+189403us) spinlock_profiling.cc:243] Waited 82.5 ms on lock 0x7b54003a98d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:08.634877 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189720832}
W20260709 13:05:08.636449 29222 rpcz_store.cc:269] 0709 13:05:06.866905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.242754 (+375849us) service_pool.cc:224] Handling call
0709 13:05:08.635256 (+1392502us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54003557d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:08.635394 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361463168}
I20260709 13:05:08.638897 29222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:05:08.638988 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:08.639531 29222 raft_consensus.cc:606] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:08.640161 29222 raft_consensus.cc:614] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:08.640261 29213 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:08.642405 29215 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 9: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:08.660436 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:05:08.670300 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.644600 32578 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.687592 32578 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.695703 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 487) took 1821 ms. Trace:
I20260709 13:05:08.696235 29216 rpcz_store.cc:276] 0709 13:05:06.874712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.371107 (+1496395us) service_pool.cc:224] Handling call
0709 13:05:08.512697 (+141590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.515007 (+  2310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.694088 (+179081us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b540033aed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:08.695515 (+  1427us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b540033aed0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:08.695639 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328206336}
W20260709 13:05:08.701754 32545 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.570s	user 0.002s	sys 0.000s
W20260709 13:05:08.707954 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 171863us
I20260709 13:05:08.712558 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:08.720465 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 268639us
I20260709 13:05:08.721478 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.728613 29143 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:05:08.727746 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 270815us
I20260709 13:05:08.729224 29144 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:08.731531 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:08.750548 29216 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
W20260709 13:05:08.743870 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 87195us
I20260709 13:05:08.613484 32080 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.746266 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:08.761085 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.738899 32178 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:08.772372 32448 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.855s	user 0.010s	sys 0.019s
I20260709 13:05:08.773093 32080 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:08.727819 32520 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:08.780325 32520 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.780504 32354 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:08.780915 32520 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:08.785055 32535 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.578s	user 0.031s	sys 0.002s
I20260709 13:05:08.785562 32535 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:08.764314 32357 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.533s	user 0.013s	sys 0.021s
I20260709 13:05:08.788305 32357 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.790604 32357 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:08.792495 32594 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:08.792055 32554 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.793071 32594 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:08.793313 32554 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:08.794034 32594 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.797057 32432 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:08.797258 32351 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.558s	user 0.016s	sys 0.008s
W20260709 13:05:08.764657 32540 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.635s	user 0.014s	sys 0.013s
I20260709 13:05:08.798041 32540 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.797092 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:08.799501 32567 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:08.799708 32536 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:08.799988 32567 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:08.800251 32540 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:08.801043 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.730050 32433 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:08.731992 32578 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:08.804791 29145 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
W20260709 13:05:08.799535 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 53322.1us
W20260709 13:05:08.824558 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 57280.8us
I20260709 13:05:08.835680 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.836540 29144 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:08.800936 32567 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.829066 32470 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.830024 28969 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:08.800354 32536 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:08.808030 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:08.836867 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:08.841984 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.846946 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:08.849109 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:08.859889 32536 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.801790 32542 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:08.869594 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:08.735347 32565 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.832566 32535 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:08.845882 32383 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:08.880920 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:08.882759 29903 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:08.883464 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:08.883667 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:08.890326 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:08.891309 29905 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:08.889364 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:08.895118 29216 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:05:08.853897 32587 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:08.892158 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 735080us
I20260709 13:05:08.893079 29908 raft_consensus.cc:2410] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:08.860721 32532 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:05:08.893357 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:08.862068 32448 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:08.899353 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.886245 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:08.900348 32587 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:08.905022 32586 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.876780 32572 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:08.847368 32539 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.843808 32527 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:05:08.848496 32554 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:05:08.854244 32433 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:05:08.908460 32539 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:08.909444 32565 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:08.871383 32440 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
W20260709 13:05:08.911309 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.873517 32540 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:08.911868 32540 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.892408 32542 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:08.912410 32540 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:08.859993 32534 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:05:08.913383 32542 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.914847 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:08.915498 29909 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:08.876073 32567 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:08.916548 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:08.917703 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:08.918563 29902 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEARNER]: Advancing to term 9
I20260709 13:05:08.920035 32589 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:05:08.920964 29908 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:08.922781 32554 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:08.901520 29147 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:05:08.923782 32433 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:08.924193 32433 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:05:08.861408 32337 consensus_queue.cc:1175] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "730186a8c70f43f2a57d80163bbc37d2" responder_term: 9 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: [8.7-8.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:08.863852 32253 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:08.925050 32433 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:08.925642 32253 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:08.926250 32433 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:08.905915 32572 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.923187 32554 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:08.905098 32587 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:08.926818 29222 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:08.869838 32522 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:05:08.872476 32439 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:05:08.838747 32178 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:05:08.902355 29907 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:05:08.937458 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:08.989703 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2027) took 1066 ms. Trace:
I20260709 13:05:08.989938 29220 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:08.938138 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:08.939002 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 357212us
I20260709 13:05:08.939613 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:08.909284 32539 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.990080 29071 rpcz_store.cc:276] 0709 13:05:07.922834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.177163 (+254329us) service_pool.cc:224] Handling call
0709 13:05:08.177476 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.177715 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:08.391165 (+213450us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.989384 (+598219us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.989434 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:08.989477 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.989547 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.989651 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":524363,"prepare.queue_time_us":111188,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"thread_start_us":260869,"threads_started":2,"wal-append.queue_time_us":103669}
I20260709 13:05:08.920972 32589 consensus_queue.cc:1243] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:08.894933 32543 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.979539 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:08.979650 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:08.980346 29214 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:08.957412 32572 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:09.010435 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:08.986349 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:08.987159 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:08.989120 29222 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:08.926537 32569 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.963868 29902 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:05:08.994333 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70953.2us
W20260709 13:05:08.994935 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55813.5us
I20260709 13:05:08.990576 32532 consensus_queue.cc:1243] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:09.001585 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:08.862344 32470 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:08.983193 32182 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:08.969329 32535 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:08.993978 32523 consensus_meta.cc:287] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.693s	user 0.020s	sys 0.010s
I20260709 13:05:09.018486 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:08.980535 32448 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:08.892180 32578 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:05:09.021450 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1909) took 1209 ms. Trace:
I20260709 13:05:09.019027 29113 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:09.039620 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:09.028703 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 131339us
I20260709 13:05:09.040372 29906 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:09.030539 29905 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 595) took 2178 ms (client timeout 1817 ms). Trace:
I20260709 13:05:08.905907 32586 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.042297 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:09.004323 32432 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.047765 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:09.040904 29905 rpcz_store.cc:269] 0709 13:05:06.844931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.845485 (+   554us) service_pool.cc:224] Handling call
0709 13:05:09.023411 (+2177926us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54002a08d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:09.023559 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4639321472}
I20260709 13:05:09.003796 32066 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:05:09.011197 32065 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:05:09.039861 29066 rpcz_store.cc:276] 0709 13:05:07.812414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.812565 (+   151us) service_pool.cc:224] Handling call
0709 13:05:07.862560 (+ 49995us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:07.864218 (+  1658us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:07.864553 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.007748 (+143195us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.082620 (+ 74872us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:08.115470 (+ 32850us) log.cc:844] Serialized 52 byte log entry
0709 13:05:08.115640 (+   170us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:08.115880 (+   240us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.279265 (+163385us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b180042fca0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20522 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:08.842773 (+563508us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b180042fca0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:05:08.981802 (+139029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.981875 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.981918 (+    43us) raft_consensus.cc:1685] finished
0709 13:05:08.981943 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.021279 (+ 39336us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b54002aa650. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:09.021393 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":854849,"prepare.queue_time_us":3999,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":1443,"raft.queue_time_us":883318,"raft.run_cpu_time_us":542,"raft.run_wall_time_us":35235,"spinlock_wait_cycles":118783104,"thread_start_us":354503,"threads_started":4,"wal-append.queue_time_us":89674}
I20260709 13:05:08.926614 32253 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.031028 32351 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:09.004056 32539 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.031237 32561 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.084823 32561 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.086082 32539 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:09.086948 32561 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:09.061776 29216 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:09.089274 32470 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:09.091558 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2137) took 1469 ms. Trace:
I20260709 13:05:09.092056 29219 rpcz_store.cc:276] 0709 13:05:07.622137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.488845 (+866708us) service_pool.cc:224] Handling call
0709 13:05:08.494974 (+  6129us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.509588 (+ 14614us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:05:08.509629 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.537177 (+ 27548us) log.cc:844] Serialized 26 byte log entry
0709 13:05:08.839951 (+302774us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:05:08.840002 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.840073 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.091306 (+251233us) raft_consensus.cc:1685] finished
0709 13:05:09.091328 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.091492 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":9661,"thread_start_us":267680,"threads_started":1,"wal-append.queue_time_us":430535}
I20260709 13:05:09.094714 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:09.096529 29146 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:09.064450 29903 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
W20260709 13:05:09.049870 32418 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.019s
I20260709 13:05:09.035269 32523 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.035080 32182 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.059046 32432 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.069876 32602 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:09.070128 32517 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:09.070763 32436 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:09.076201 32559 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:09.077078 32556 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:09.059718 32287 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:05:09.078871 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:09.039639 32535 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:09.079360 32478 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.006s
I20260709 13:05:09.132692 32080 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:09.136812 32478 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:09.137920 32602 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:09.138367 32523 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:09.138336 32602 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:09.139493 29221 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:09.079535 32533 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:09.140309 32533 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:09.124514 32517 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:09.142530 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 63711.4us
I20260709 13:05:09.143474 29038 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:09.075759 32555 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:05:09.080117 32548 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:09.080262 32426 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.084076 32197 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:09.014387 32543 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.102450 32351 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.149224 32448 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.067340 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:09.154129 29905 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:09.155205 32351 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:09.159070 32367 consensus_meta.cc:287] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.697s	user 0.013s	sys 0.026s
I20260709 13:05:09.097312 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:09.121461 32436 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.099273 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:09.105191 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:09.056917 32557 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:09.065352 32447 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.844s	user 0.020s	sys 0.009s
I20260709 13:05:09.059420 32297 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:09.116034 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.120254 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:09.087345 32540 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.228593 32182 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:09.229319 32540 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:09.062706 32416 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:09.123832 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:09.122036 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:09.128369 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:09.146987 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:09.102795 32561 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:09.231390 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:09.231721 32561 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.234153 32561 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:09.109683 32594 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.234865 32594 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.235721 32517 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:09.160562 32556 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:09.152715 32197 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.125473 32253 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:09.238575 32197 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:09.153220 32468 consensus_meta.cc:287] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.559s	user 0.011s	sys 0.011s
I20260709 13:05:09.096551 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:09.150863 32602 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:09.240374 32601 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:05:09.241045 32602 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:09.231411 29219 raft_consensus.cc:2410] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:05:09.242573 32587 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.043210 32554 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.110275 32580 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.202631 29147 raft_consensus.cc:2368] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:09.021173 32569 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:09.267874 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 176532us
I20260709 13:05:09.281903 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:09.198722 32436 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:09.204283 32482 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:09.285321 32197 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.286288 32197 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.199054 29905 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:09.206350 29906 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:09.141443 32533 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.236472 32517 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.244024 32555 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:09.244385 32447 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.245025 32432 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:09.245535 29140 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
I20260709 13:05:09.198226 29142 raft_consensus.cc:2410] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:05:09.212148 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 115406us
I20260709 13:05:09.305910 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:09.161235 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:09.228627 29216 pending_rounds.cc:85] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 2
W20260709 13:05:09.293267 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.306679 29216 pending_rounds.cc:107] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:05:09.306959 29216 raft_consensus.cc:2889] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:09.298727 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:09.300316 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:09.247872 32557 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:09.283466 32561 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:09.205998 32539 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
I20260709 13:05:09.309675 29147 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 6: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:05:09.302922 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 159544us
I20260709 13:05:09.160820 32426 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.275689 32556 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:09.347431 29217 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:09.300758 32482 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:09.298027 32352 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:09.304890 32357 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.410925 32357 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.315428 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:09.380410 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:09.415730 32610 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:05:09.416878 32352 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:09.417743 32352 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.306960 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:09.420509 29905 raft_consensus.cc:2376] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9. Re-sending same reply.
I20260709 13:05:09.283476 32580 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.301616 32517 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.386039 29910 raft_consensus.cc:2410] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:05:09.266908 32297 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:09.311491 32561 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.401985 29909 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:09.405129 29219 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:09.409034 32556 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.297870 32555 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:09.380249 32389 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:05:09.413329 32482 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:09.526193 32606 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.301s	user 0.000s	sys 0.001s
I20260709 13:05:09.394191 32559 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:09.527223 32618 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:09.527786 32618 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.529306 32426 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:09.531131 32297 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:09.531325 32383 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.633s	user 0.007s	sys 0.004s
I20260709 13:05:09.536211 29069 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:09.536580 29144 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:09.527446 32559 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.531862 32297 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:05:09.506347 32548 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.516711 32611 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:09.400568 32287 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.133s
I20260709 13:05:09.561570 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:09.550056 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.523793 32561 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:09.537599 32383 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:05:09.633212 32611 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.680291 32611 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.684434 32197 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:09.685544 32609 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.686965 32609 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:09.692214 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 376833us
I20260709 13:05:09.584862 32548 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:09.693830 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:09.536856 32272 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.694869 32272 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.695993 32619 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:09.696616 32619 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:09.696897 32548 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:09.697069 32619 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:09.698294 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 136839us
I20260709 13:05:09.689430 32611 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:09.603621 32572 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.575s	user 0.006s	sys 0.012s
I20260709 13:05:09.701433 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:09.704636 29066 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:09.705200 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:09.701936 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:09.703316 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.706166 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:09.702713 29037 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:09.500737 32557 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.539906 32586 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:09.708693 29071 raft_consensus.cc:2410] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:09.709491 29067 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:09.670202 32351 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:09.716714 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:09.710551 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:09.776510 29064 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:09.776429 29070 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:09.724407 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:09.590574 32569 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:09.668733 32527 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.777676 29068 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
W20260709 13:05:09.791059 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 88438.7us
I20260709 13:05:09.698797 32352 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:09.818184 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:09.703508 32287 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:09.820232 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:09.819661 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:09.827841 29064 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:09.718297 32447 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:09.829452 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:09.832947 29068 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:09.833072 29070 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:09.831740 32297 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:09.835892 32297 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:09.836539 32620 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 13:05:09.820842 32548 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:09.831893 32611 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.838065 32297 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:09.775511 32351 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:09.796239 32587 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:09.828207 32569 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.838961 32611 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.852483 32569 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.854382 32569 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:09.822371 32527 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:09.838099 32622 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 13:05:09.867864 29064 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:09.718487 32572 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:09.870469 29186 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.041s	sys 0.048s
W20260709 13:05:09.889371 29186 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.041s	sys 0.048s
W20260709 13:05:09.889784 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 610635us
I20260709 13:05:09.890347 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:09.849668 32520 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.068s	user 0.009s	sys 0.006s
W20260709 13:05:09.887530 29218 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.009s	sys 0.077s
I20260709 13:05:09.891516 29064 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:05:09.850425 32447 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:09.895673 29218 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.010s	sys 0.077s
I20260709 13:05:09.897840 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:09.898772 29064 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:09.845451 32618 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:09.844456 32623 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:09.867816 32527 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:09.916810 29064 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:09.854394 32611 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:09.922473 32517 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:09.943327 32628 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.918823 32520 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.945319 32623 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.958453 32628 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.921155 32447 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:09.981491 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:09.982510 29064 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:09.989843 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:09.990854 29902 raft_consensus.cc:2410] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:09.994566 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:09.995327 29906 raft_consensus.cc:2368] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:05:09.957762 32559 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:09.926477 32351 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:10.000908 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:10.001384 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:10.001976 29905 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:10.000952 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:10.004302 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:09.983785 32609 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:10.016150 29037 leader_election.cc:400] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:05:10.024461 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:10.027329 29037 leader_election.cc:403] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:10.027516 29073 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:10.026876 29064 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:05:10.029999 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:10.030031 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:10.034817 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.074337 29066 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
W20260709 13:05:10.080751 29187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.013s	sys 0.017s
I20260709 13:05:10.026078 32580 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:10.035080 32559 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:05:10.035301 32383 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:10.094045 29073 raft_consensus.cc:2410] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:05:10.081156 29187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.013s	sys 0.017s
I20260709 13:05:10.091171 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:10.065550 32612 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:05:10.066232 32351 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:10.104295 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:10.105777 32628 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:10.109800 32632 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.144251 32632 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:10.149454 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:10.114418 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 490) took 2550 ms. Trace:
I20260709 13:05:10.113160 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:10.152832 29217 rpcz_store.cc:276] 0709 13:05:07.564322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.485600 (+921278us) service_pool.cc:224] Handling call
0709 13:05:08.493890 (+  8290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.402885 (+908995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.114126 (+711241us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b540033a4d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:10.114352 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1962294016}
I20260709 13:05:10.118758 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:10.120977 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.125010 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:10.095937 32527 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:10.129668 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:10.130672 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1900) took 2737 ms. Trace:
W20260709 13:05:10.132441 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.133432 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:10.112933 32615 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.880s	user 0.005s	sys 0.004s
I20260709 13:05:10.319967 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:10.318105 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2156) took 1224 ms. Trace:
I20260709 13:05:10.325557 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:10.168799 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 64549.4us
I20260709 13:05:10.318710 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2046) took 2439 ms. Trace:
I20260709 13:05:10.515472 29221 rpcz_store.cc:276] 0709 13:05:07.879085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.522226 (+643141us) service_pool.cc:224] Handling call
0709 13:05:08.522536 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.140019 (+617483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.318470 (+1178451us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b540026c0d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:10.318647 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1418350464}
I20260709 13:05:10.516422 29186 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:10.323789 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:10.316771 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:10.177086 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.094239 32383 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.093833 32580 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.090869 32253 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.153723 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:10.522629 29141 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:10.320542 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2162) took 1219 ms. Trace:
I20260709 13:05:10.396595 28969 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:10.512066 29142 rpcz_store.cc:276] 0709 13:05:09.094136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.094276 (+   140us) service_pool.cc:224] Handling call
0709 13:05:10.317854 (+1223578us) spinlock_profiling.cc:243] Waited 65 ms on lock 0x7b5400190d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:10.318033 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149611392}
I20260709 13:05:10.396709 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1896) took 1284 ms. Trace:
I20260709 13:05:10.513827 29067 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:10.511770 29139 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:10.532162 29140 rpcz_store.cc:276] 0709 13:05:09.100755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.100890 (+   135us) service_pool.cc:224] Handling call
0709 13:05:10.320374 (+1219484us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b54002fab54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:10.320485 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34518912}
I20260709 13:05:10.513844 29069 rpcz_store.cc:276] 0709 13:05:07.393040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.393928 (+   888us) service_pool.cc:224] Handling call
0709 13:05:07.411067 (+ 17139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.536712 (+2125645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.130327 (+593615us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b5400185e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:10.130552 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4879321856}
W20260709 13:05:10.523329 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.537569 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.330307 32627 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.408s	user 0.000s	sys 0.001s
W20260709 13:05:10.331700 32624 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.336s	user 0.000s	sys 0.001s
W20260709 13:05:10.327105 32613 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.789s	user 0.000s	sys 0.001s
I20260709 13:05:10.539752 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:10.261580 32447 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:10.254202 32640 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:10.456820 32507 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.539561 32448 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:10.549006 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:10.549343 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:10.226014 32572 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:10.539816 32439 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:10.105835 32080 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:10.550486 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:10.598716 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:10.600040 29141 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:05:10.531457 32527 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:10.551448 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:10.333269 32626 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.365s	user 0.000s	sys 0.002s
I20260709 13:05:10.555275 32354 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:10.556674 32536 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:10.557178 32607 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.320s	user 0.002s	sys 0.000s
I20260709 13:05:10.558346 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2163) took 1457 ms. Trace:
I20260709 13:05:10.262362 32520 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:10.560740 32287 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:10.263222 32634 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.582660 29068 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:10.323704 32436 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.024s	user 0.009s	sys 0.000s
I20260709 13:05:10.314994 32496 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:10.542158 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:10.334748 32565 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:10.225178 32628 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:10.537019 32383 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:10.520414 32639 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:10.588778 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:10.567878 32619 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.865s	user 0.013s	sys 0.000s
W20260709 13:05:10.588864 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 332305us
I20260709 13:05:10.328661 32623 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:10.537335 29148 rpcz_store.cc:276] 0709 13:05:09.064862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.065464 (+   602us) service_pool.cc:224] Handling call
0709 13:05:09.065732 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.080320 (+ 14588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.080348 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:09.080898 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:05:09.285392 (+204494us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:09.285452 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.285557 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:10.349325 (+1063768us) raft_consensus.cc:1685] finished
0709 13:05:10.349346 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:10.349522 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14195,"raft.queue_time_us":14819,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"thread_start_us":204085,"threads_started":1,"wal-append.queue_time_us":635626}
I20260709 13:05:10.588079 29067 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:10.534782 32629 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:10.581952 32594 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:10.582821 32632 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:10.604029 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:10.606878 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:10.612784 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2057) took 1525 ms. Trace:
I20260709 13:05:10.613893 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2164) took 1467 ms. Trace:
I20260709 13:05:10.552525 32253 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:10.552414 32580 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:10.339711 32555 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:10.592777 32507 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:10.827292 32638 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:05:10.602859 32080 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:10.827663 32272 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:10.826455 32646 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.717517 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:10.755975 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 496) took 1668 ms. Trace:
I20260709 13:05:10.621704 29144 rpcz_store.cc:276] 0709 13:05:09.101077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.128104 (+ 27027us) service_pool.cc:224] Handling call
0709 13:05:10.558138 (+1430034us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b540029e354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:10.558274 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702986112}
I20260709 13:05:10.830817 29219 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:10.603480 32439 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:10.768229 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:10.860280 29213 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:10.859323 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:10.860127 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1925) took 1118 ms. Trace:
I20260709 13:05:10.722391 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:10.862470 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:10.830762 29218 rpcz_store.cc:276] 0709 13:05:09.087436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.088016 (+   580us) service_pool.cc:224] Handling call
0709 13:05:09.088298 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.088760 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.088789 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:09.089286 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:05:09.248537 (+159251us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:09.896580 (+648043us) log.cc:844] Serialized 14 byte log entry
0709 13:05:10.090609 (+194029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.091867 (+  1258us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:10.326298 (+234431us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:10.744749 (+418451us) raft_consensus.cc:1685] finished
0709 13:05:10.755653 (+ 10904us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:10.755883 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27507,"prepare.queue_time_us":449089,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":153241,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":18823,"spinlock_wait_cycles":4779392,"thread_start_us":909198,"threads_started":3,"wal-append.queue_time_us":302378}
I20260709 13:05:10.863543 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:10.863840 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:10.864529 29222 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:10.877081 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:10.833727 29140 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:10.860965 29067 rpcz_store.cc:276] 0709 13:05:09.711959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.720674 (+  8715us) service_pool.cc:224] Handling call
0709 13:05:10.325361 (+604687us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b5800005198. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:05:10.830116 (+504755us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8054528}
I20260709 13:05:10.878126 29217 raft_consensus.cc:2376] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12. Re-sending same reply.
I20260709 13:05:10.597174 32448 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:10.873127 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:10.882455 29071 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:10.873723 29187 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.27.71.190:37537, user_credentials={real_user=slave}} blocked reactor thread for 95019.2us
W20260709 13:05:10.686681 32624 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:05:10.860203 29215 rpcz_store.cc:276] 0709 13:05:09.087092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.087585 (+   493us) service_pool.cc:224] Handling call
0709 13:05:09.089795 (+  2210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.090153 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.090184 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:09.090640 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:05:09.148064 (+ 57424us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:09.148118 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.183447 (+ 35329us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:10.112267 (+928820us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:10.611083 (+498816us) raft_consensus.cc:1685] finished
0709 13:05:10.611115 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:10.612693 (+  1578us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16956,"raft.queue_time_us":33340,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":117751168,"thread_start_us":54822,"threads_started":1,"wal-append.queue_time_us":142077}
I20260709 13:05:10.865314 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:10.717816 32542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.478s	user 0.034s	sys 0.037s
I20260709 13:05:10.828887 32646 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:10.886284 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:10.650949 32645 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.891841 29219 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:10.832099 29146 rpcz_store.cc:276] 0709 13:05:09.146158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.317463 (+171305us) service_pool.cc:224] Handling call
0709 13:05:09.379157 (+ 61694us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:09.379620 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.379658 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:09.380275 (+   617us) log.cc:844] Serialized 52 byte log entry
0709 13:05:09.547243 (+166968us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:09.547305 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.547417 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:10.332535 (+785118us) spinlock_profiling.cc:243] Waited 56.6 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:10.613524 (+280989us) raft_consensus.cc:1685] finished
0709 13:05:10.613553 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:10.613820 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1112,"raft.queue_time_us":3979,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":4179,"spinlock_wait_cycles":130224512,"thread_start_us":163947,"threads_started":1,"wal-append.queue_time_us":565570}
I20260709 13:05:10.623596 32482 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.867861 32496 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:10.778106 32357 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:10.859880 32565 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:10.936131 29068 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:10.914155 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:10.944635 29069 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:10.868861 32572 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.917785 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:10.859709 32628 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:10.926448 32651 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:10.926716 32646 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:10.881426 32555 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:10.891641 32542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.679s	user 0.036s	sys 0.048s
I20260709 13:05:10.941828 29188 leader_election.cc:400] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:10.969261 29188 leader_election.cc:403] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:05:10.945658 29909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 534) took 1973 ms (client timeout 2008 ms). Trace:
I20260709 13:05:10.945961 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2044) took 1075 ms. Trace:
I20260709 13:05:10.947804 29067 raft_consensus.cc:2410] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:05:10.923029 29213 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:05:10.906771 29071 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:10.923760 29222 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
W20260709 13:05:10.949858 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 244209us
W20260709 13:05:10.952006 32561 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.265s	user 0.013s	sys 0.000s
I20260709 13:05:10.639565 32536 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:10.868036 32436 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:10.861913 32629 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:10.624221 32354 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:10.948939 32357 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:10.868467 32619 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:10.928165 32439 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:10.967185 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 833803us
I20260709 13:05:10.869287 32639 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:10.976279 29213 pending_rounds.cc:85] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 3
I20260709 13:05:11.001266 29213 pending_rounds.cc:107] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:05:11.001658 29213 raft_consensus.cc:2889] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:10.971679 29068 rpcz_store.cc:276] 0709 13:05:09.870215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.531093 (+660878us) service_pool.cc:224] Handling call
0709 13:05:10.945881 (+414788us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.003588 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:11.004635 29068 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 5.
I20260709 13:05:10.968834 32594 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.981978 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.011440 29214 raft_consensus.cc:2410] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:10.983966 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:10.955987 32178 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.975660 32653 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.994943 32080 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.019711 29066 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:10.970865 29909 rpcz_store.cc:269] 0709 13:05:08.958728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.958840 (+   112us) service_pool.cc:224] Handling call
0709 13:05:10.931549 (+1972709us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b540061fdd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:10.931710 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893095168}
I20260709 13:05:11.020319 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:10.998764 32602 consensus_meta.cc:287] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.757s	user 0.014s	sys 0.002s
I20260709 13:05:10.981457 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:11.021025 29068 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:11.021257 29067 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:11.007694 29222 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:10.928095 32482 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:11.030180 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.996042 32496 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.012941 29145 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:05:11.014669 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:11.031822 32496 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:11.033171 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:11.037174 32354 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.037778 32354 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.038254 32354 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:10.982579 32579 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:10.872126 32634 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:10.998253 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.018608 32650 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:05:11.028699 32580 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:11.030098 32647 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:05:11.016080 29221 raft_consensus.cc:2410] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:05:10.958114 32572 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:10.917421 32645 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.078083 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:11.079277 29069 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:05:11.062152 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.080251 29222 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:10.990610 32561 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.044056 32272 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:11.087111 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.087713 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.087944 29214 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:11.091156 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.092069 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.092587 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.092921 29222 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:05:11.093264 29218 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:11.094461 29222 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:10.967857 32651 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.053404 32556 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.008867 32357 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:11.105532 29185 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.000s	sys 0.012s
W20260709 13:05:11.106983 29185 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.001s	sys 0.013s
W20260709 13:05:11.107347 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 594922us
W20260709 13:05:11.109854 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.100946 29067 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:11.073104 32527 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:11.102759 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.044509 32178 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:11.106223 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 73111.7us
I20260709 13:05:11.020759 32594 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.104768 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:11.126060 29069 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:11.121727 29072 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:11.124115 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:11.126693 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.124424 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:11.095886 32542 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.128643 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.119390 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.012528 32448 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.130147 29147 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:11.128873 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.131693 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:11.129500 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:11.132758 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:11.134080 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.133991 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.134775 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.116456 32556 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.056703 32653 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.158239 29067 raft_consensus.cc:2410] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:11.019543 32565 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.118750 32602 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.139365 29139 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:05:11.120307 32354 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.154680 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.137733 29146 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:11.201694 29213 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:11.138686 29144 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:11.161127 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.120304 32640 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.198997 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.166489 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:11.199791 29069 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:11.227667 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.248291 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2056) took 2143 ms. Trace:
I20260709 13:05:11.130270 32507 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.134941 32439 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.100325 32272 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.248204 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:11.251938 29142 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:11.247077 29139 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:11.209676 29214 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:11.131951 32662 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:11.203178 29140 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:11.213614 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1921) took 1502 ms. Trace:
I20260709 13:05:11.248873 29220 rpcz_store.cc:276] 0709 13:05:09.061040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.061145 (+   105us) service_pool.cc:224] Handling call
0709 13:05:09.070069 (+  8924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.141590 (+ 71521us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:09.388636 (+247046us) log.cc:844] Serialized 14 byte log entry
0709 13:05:09.954378 (+565742us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:09.954454 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:09.954534 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.329505 (+374971us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:11.204224 (+874719us) spinlock_profiling.cc:243] Waited 71.3 ms on lock 0x7b54003a93d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:11.204285 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.204415 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19662,"prepare.queue_time_us":449189,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":68,"raft.queue_time_us":272359,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":35428,"spinlock_wait_cycles":399760640,"thread_start_us":811618,"threads_started":3,"wal-append.queue_time_us":490606}
I20260709 13:05:11.261359 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:11.261734 29038 leader_election.cc:336] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.525s (SENT)
I20260709 13:05:11.265391 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:11.265930 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.262894 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.266935 29147 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
W20260709 13:05:11.267426 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 176921us
I20260709 13:05:11.267853 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.155961 32639 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:11.263160 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.157140 32556 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:11.209422 32539 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:11.159296 32527 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.120s
I20260709 13:05:11.280601 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2059) took 2053 ms. Trace:
I20260709 13:05:11.157464 32594 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.157512 32572 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.273268 29069 rpcz_store.cc:276] 0709 13:05:09.710823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.159991 (+1449168us) service_pool.cc:224] Handling call
0709 13:05:11.199249 (+ 39258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.207803 (+  8554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.213489 (+  5686us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.272675 29148 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:05:11.275707 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 109415us
I20260709 13:05:11.276589 29139 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:05:11.286548 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.274741 32352 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:05:11.296272 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.334949 32653 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.335279 32572 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:11.335745 32572 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:11.238714 32602 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.161060 32542 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.336688 32572 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.264108 32565 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:11.264340 32507 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.296795 32354 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:05:11.204063 32661 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:11.310631 32178 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:11.312132 32658 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:05:11.404779 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:11.251408 32640 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.312696 32603 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:11.315225 32535 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:11.316401 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2168) took 1191 ms. Trace:
I20260709 13:05:11.317911 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1913) took 1613 ms. Trace:
W20260709 13:05:11.319532 29066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 366) took 1490 ms (client timeout 1891 ms). Trace:
I20260709 13:05:11.321168 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.322659 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:11.331291 29910 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 601) took 2459 ms (client timeout 1928 ms). Trace:
I20260709 13:05:11.334026 32536 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.334545 29215 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:05:11.335106 29903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 603) took 2441 ms (client timeout 2752 ms). Trace:
I20260709 13:05:11.261063 32632 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:11.419008 29903 rpcz_store.cc:269] 0709 13:05:08.894261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.894587 (+   326us) service_pool.cc:224] Handling call
0709 13:05:11.334950 (+2440363us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54003fdc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:11.335053 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335980800}
I20260709 13:05:11.447294 29146 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:11.335114 29140 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:11.336339 29904 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:11.288692 32629 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:11.353770 32542 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:11.342819 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 67519.2us
I20260709 13:05:11.454145 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:11.353855 32507 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.455981 32632 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.457211 32660 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.457418 32507 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:11.457779 32660 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.458276 32632 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:11.459342 32673 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.460024 32660 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:11.459971 32673 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.462289 32673 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.464007 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:11.464646 29067 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:05:11.465431 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:11.466369 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:11.467293 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:11.351696 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.305938 32436 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.312376 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:11.406561 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 265310us
W20260709 13:05:11.415549 29910 rpcz_store.cc:269] 0709 13:05:08.874462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.881315 (+  6853us) service_pool.cc:224] Handling call
0709 13:05:11.330983 (+2449668us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54007c01d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:11.331192 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907809536}
W20260709 13:05:11.415275 29066 rpcz_store.cc:269] 0709 13:05:09.829550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.149100 (+319550us) service_pool.cc:224] Handling call
0709 13:05:11.319329 (+1170229us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54002ad354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:11.319461 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993568128}
I20260709 13:05:11.353667 32645 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.415802 29145 rpcz_store.cc:276] 0709 13:05:10.124975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.128419 (+  3444us) service_pool.cc:224] Handling call
0709 13:05:10.128787 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.032136 (+903349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.316100 (+283964us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b540027ced4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:11.316309 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2032783104}
I20260709 13:05:11.485152 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.416884 29072 rpcz_store.cc:276] 0709 13:05:09.704842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.704974 (+   132us) service_pool.cc:224] Handling call
0709 13:05:11.317704 (+1612730us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54001d8654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:11.317845 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3243942656}
I20260709 13:05:11.485885 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:11.486315 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:11.421267 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.286899 32614 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:11.290799 29216 rpcz_store.cc:276] 0709 13:05:09.227554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.227880 (+   326us) service_pool.cc:224] Handling call
0709 13:05:09.228331 (+   451us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:09.313279 (+ 84948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.313330 (+    51us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:09.346734 (+ 33404us) log.cc:844] Serialized 78 byte log entry
0709 13:05:10.116531 (+769797us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b18004735c0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:05:10.590691 (+474160us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:10.590735 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.590801 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.280265 (+689464us) raft_consensus.cc:1685] finished
0709 13:05:11.280288 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.280535 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":75284,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":11618,"spinlock_wait_cycles":2803968,"thread_start_us":473446,"threads_started":1,"wal-append.queue_time_us":1377930}
I20260709 13:05:11.298753 32253 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.417827 32603 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:11.492462 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:11.467809 29067 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 5.
I20260709 13:05:11.278999 32671 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.149s
I20260709 13:05:11.489655 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:11.493166 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:11.471791 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 49160.4us
I20260709 13:05:11.482973 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:11.310822 32656 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 13:05:11.486682 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 174405us
I20260709 13:05:11.500952 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:11.501852 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:11.269574 32439 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.488445 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:11.485795 29145 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
W20260709 13:05:11.504480 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.279124 32539 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:11.505164 29145 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:05:11.505505 32539 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:11.491640 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.493716 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:11.506240 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.506743 29067 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:11.507189 29146 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:11.447600 32080 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:11.498620 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:11.486735 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.512467 29215 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:11.502544 29185 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:11.514468 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:11.486146 32389 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:11.520332 29067 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: [3.3-10.6]
I20260709 13:05:11.521304 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2031) took 1706 ms. Trace:
I20260709 13:05:11.521721 29067 rpcz_store.cc:276] 0709 13:05:09.814875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.519386 (+1704511us) service_pool.cc:224] Handling call
0709 13:05:11.519904 (+   518us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:05:11.520941 (+  1037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.521231 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.523780 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.524669 29215 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
W20260709 13:05:11.512022 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 374810us
I20260709 13:05:11.492067 32632 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:11.355463 32672 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:11.535754 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.538949 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.535707 29145 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:05:11.536283 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:11.539989 29145 pending_rounds.cc:85] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 4
I20260709 13:05:11.540261 29145 pending_rounds.cc:107] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:05:11.405064 32661 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.540174 29148 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:11.540576 29145 raft_consensus.cc:2889] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:11.540761 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.539319 29146 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
W20260709 13:05:11.532217 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 262511us
I20260709 13:05:11.542546 29067 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [4.7-5.8]
I20260709 13:05:11.539532 29215 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:11.543735 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2179) took 1412 ms. Trace:
W20260709 13:05:11.545301 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 46816.9us
I20260709 13:05:11.508302 32673 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:11.550843 29066 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:11.545836 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:11.506006 32561 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:11.509966 32178 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:11.536630 29112 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:11.550812 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2032) took 1735 ms. Trace:
I20260709 13:05:11.582283 29067 rpcz_store.cc:276] 0709 13:05:09.815166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.539805 (+1724639us) service_pool.cc:224] Handling call
0709 13:05:11.542325 (+  2520us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:11.550524 (+  8199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.550760 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.583823 29067 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:11.584518 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2033) took 1769 ms. Trace:
I20260709 13:05:11.584913 29067 rpcz_store.cc:276] 0709 13:05:09.815460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.583202 (+1767742us) service_pool.cc:224] Handling call
0709 13:05:11.583501 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.584204 (+   703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.584437 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.542896 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:11.581655 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:11.547405 29141 rpcz_store.cc:276] 0709 13:05:10.131372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.601272 (+469900us) service_pool.cc:224] Handling call
0709 13:05:10.601583 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.601831 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:10.971110 (+369279us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.080123 (+109013us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.080464 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.457010 (+376546us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b1800036580. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f783a9e158f 00007f7833a20571 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd
0709 13:05:11.485420 (+ 28410us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b1800036580. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:11.543342 (+ 57922us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.543402 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:11.543447 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.543520 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.543664 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":214728,"prepare.queue_time_us":264444,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":8248,"raft.queue_time_us":508530,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":29088,"spinlock_wait_cycles":62255360,"thread_start_us":914021,"threads_started":3,"wal-append.queue_time_us":128262}
I20260709 13:05:11.533088 32253 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:11.509341 32605 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:05:11.535228 32389 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:11.534706 32426 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:11.523106 32629 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:05:11.485736 32668 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:05:11.537662 32625 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:11.588034 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 45236.7us
I20260709 13:05:11.601037 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:11.554916 32651 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:11.603543 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:11.605362 29038 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:11.559628 29215 raft_consensus.cc:2376] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11. Re-sending same reply.
I20260709 13:05:11.544031 32632 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:11.543977 32580 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:11.543879 32535 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:05:11.591948 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 55382.1us
I20260709 13:05:11.580722 32357 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:11.591774 29113 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:11.549744 32542 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:11.551630 32080 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:11.599066 29067 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:05:11.616058 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:11.580617 32496 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:05:11.666687 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2034) took 1851 ms. Trace:
I20260709 13:05:11.667147 29067 rpcz_store.cc:276] 0709 13:05:09.815752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.585746 (+1769994us) service_pool.cc:224] Handling call
0709 13:05:11.598830 (+ 13084us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.666379 (+ 67549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.666624 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:11.667795 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.598372 32645 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:11.595021 32389 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:11.669003 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 77295.2us
I20260709 13:05:11.595882 32594 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:11.665474 32557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.584s	user 0.004s	sys 0.012s
I20260709 13:05:11.671285 29112 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:11.673113 32677 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:11.673447 32651 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:11.657548 32426 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:11.596786 32178 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:11.656204 32685 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:05:11.604908 32253 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:11.602463 32625 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.673626 32357 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:11.594190 32561 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 13:05:11.726524 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:11.665928 32665 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.378s	user 0.000s	sys 0.001s
I20260709 13:05:11.659830 32632 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:11.676122 32645 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.662119 32535 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.879086 32652 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:05:11.683899 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 150363us
I20260709 13:05:11.837560 32555 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:11.684392 32679 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:11.729521 32557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.664s	user 0.008s	sys 0.026s
I20260709 13:05:11.678550 32533 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:11.799685 29186 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 1.908s (SENT) [suppressed 23 similar messages]
I20260709 13:05:11.854946 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2037) took 2033 ms. Trace:
I20260709 13:05:11.776109 32677 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:11.725224 32672 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:11.786795 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 181496us
W20260709 13:05:11.923058 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.923345 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:11.878907 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2149) took 1103 ms. Trace:
I20260709 13:05:11.873176 32645 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:11.898767 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.897068 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2152) took 1120 ms. Trace:
I20260709 13:05:11.929505 29221 rpcz_store.cc:276] 0709 13:05:10.776460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.859040 (+ 82580us) service_pool.cc:224] Handling call
0709 13:05:11.896796 (+1037756us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b540007add4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:11.896985 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355884288}
I20260709 13:05:11.842039 32625 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:11.676437 32542 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.850037 32272 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.905893 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:11.937213 29216 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 9 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:05:11.726079 32436 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.186s
I20260709 13:05:11.914736 29070 rpcz_store.cc:276] 0709 13:05:09.821351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.840978 (+2019627us) service_pool.cc:224] Handling call
0709 13:05:11.854406 (+ 13428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.854602 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:11.854631 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:11.854671 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:11.854700 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.854746 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.854884 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:11.914506 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 114876us
I20260709 13:05:11.912755 32671 consensus_queue.cc:1243] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:11.587579 32603 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.924602 29217 rpcz_store.cc:276] 0709 13:05:10.775468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.879138 (+103670us) service_pool.cc:224] Handling call
0709 13:05:10.890985 (+ 11847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:10.919121 (+ 28136us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:10.919165 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:10.938897 (+ 19732us) log.cc:844] Serialized 26 byte log entry
0709 13:05:11.098848 (+159951us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:11.098887 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.098956 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.878635 (+779679us) raft_consensus.cc:1685] finished
0709 13:05:11.878662 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.878834 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27656,"raft.queue_time_us":27981,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":10335,"thread_start_us":117649,"threads_started":1,"wal-append.queue_time_us":200096}
I20260709 13:05:11.841974 32662 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:05:11.876096 32594 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:11.899860 32535 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:11.929090 29215 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:11.889779 32652 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:11.881747 32688 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:11.913203 32679 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:11.934268 32561 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:11.879951 32683 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:05:11.921743 32677 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:11.913374 32543 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:11.959509 29070 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: []
W20260709 13:05:12.009739 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 338536us
I20260709 13:05:12.008720 32182 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:12.009063 32253 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:11.995527 32679 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:11.955276 32640 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:11.995672 32561 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:11.954428 32178 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:11.947142 32439 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:12.021407 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 362) took 4151 ms. Trace:
I20260709 13:05:12.025914 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 363) took 2197 ms. Trace:
I20260709 13:05:12.026297 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:12.086809 29070 rpcz_store.cc:276] 0709 13:05:09.828751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.946334 (+2117583us) service_pool.cc:224] Handling call
0709 13:05:11.959262 (+ 12928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.025648 (+ 66386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.025852 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:12.087600 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 61400.7us
W20260709 13:05:12.030795 32676 log.cc:927] Time spent T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.433s	user 0.001s	sys 0.000s
I20260709 13:05:11.931457 32533 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:12.043589 32681 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:05:12.043882 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 542) took 2767 ms. Trace:
W20260709 13:05:12.048357 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.114202 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.049098 32662 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:12.049149 29071 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2043) took 2181 ms (client timeout 1771 ms). Trace:
I20260709 13:05:12.052270 32645 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:12.071256 32559 consensus_meta.cc:287] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.977s	user 0.012s	sys 0.003s
I20260709 13:05:12.054381 32561 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:12.057284 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 813901us
W20260709 13:05:12.057888 32383 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.187s	user 0.005s	sys 0.006s
W20260709 13:05:12.061066 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.131621 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:12.064318 32628 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:12.065828 32660 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:11.995321 32603 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:12.080041 32482 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:12.081244 32554 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:11.951841 32672 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:12.107304 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 421219us
I20260709 13:05:12.028909 32661 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:12.112392 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 364) took 2283 ms. Trace:
I20260709 13:05:12.157258 29070 rpcz_store.cc:276] 0709 13:05:09.829043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.087899 (+2258856us) service_pool.cc:224] Handling call
0709 13:05:12.111829 (+ 23930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.112055 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:12.112088 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:12.112129 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:12.112166 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.112221 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.112337 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:12.114797 29071 rpcz_store.cc:269] 0709 13:05:09.867377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.118441 (+251064us) service_pool.cc:224] Handling call
0709 13:05:12.048950 (+1930509us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b5400208ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:12.049081 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":855358848}
I20260709 13:05:12.112886 29904 rpcz_store.cc:276] 0709 13:05:09.276253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.276458 (+   205us) service_pool.cc:224] Handling call
0709 13:05:11.336006 (+2059548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:11.448677 (+112671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.043556 (+594879us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54003fdc50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:12.043763 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4687199488}
I20260709 13:05:12.086740 29065 rpcz_store.cc:276] 0709 13:05:07.870289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.900291 (+ 30002us) service_pool.cc:224] Handling call
0709 13:05:07.900673 (+   382us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:07.994936 (+ 94263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:07.994966 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:07.996702 (+  1736us) log.cc:844] Serialized 52 byte log entry
0709 13:05:08.128401 (+131699us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:08.204618 (+ 76217us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.858514 (+3653896us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b1800421480. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287
0709 13:05:11.871788 (+ 13274us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.995237 (+123449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.995344 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:12.016185 (+ 20841us) raft_consensus.cc:1685] finished
0709 13:05:12.016224 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.021341 (+  5117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":811572,"prepare.queue_time_us":167829,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":9281,"raft.queue_time_us":930487,"raft.run_cpu_time_us":507,"raft.run_wall_time_us":39618,"spinlock_wait_cycles":47208832,"thread_start_us":645084,"threads_started":5,"wal-append.queue_time_us":348624,"wal-append.run_cpu_time_us":2866,"wal-append.run_wall_time_us":1098308}
I20260709 13:05:12.053164 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2030) took 2238 ms. Trace:
I20260709 13:05:12.113109 32580 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:12.132795 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.193063 29142 raft_consensus.cc:2410] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:12.134157 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.195255 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2160) took 1066 ms. Trace:
I20260709 13:05:12.134779 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.196125 29071 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:05:12.134846 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.135526 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.196975 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2039) took 2365 ms. Trace:
I20260709 13:05:12.195796 29213 rpcz_store.cc:276] 0709 13:05:11.129032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.202430 (+ 73398us) service_pool.cc:224] Handling call
0709 13:05:11.260605 (+ 58175us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.275592 (+ 14987us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:11.275621 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:11.286195 (+ 10574us) log.cc:844] Serialized 26 byte log entry
0709 13:05:11.546339 (+260144us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:11.546394 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.546477 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:12.194927 (+648450us) raft_consensus.cc:1685] finished
0709 13:05:12.194965 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.195169 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14564,"raft.queue_time_us":24816,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":98,"thread_start_us":248928,"threads_started":1,"wal-append.queue_time_us":362931}
I20260709 13:05:12.197218 29143 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:12.136412 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:12.138615 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 215370us
I20260709 13:05:12.136492 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.198096 29144 raft_consensus.cc:2368] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:05:12.121058 32559 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:05:12.198632 29038 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:12.150128 29140 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:05:12.155514 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.197355 29071 rpcz_store.cc:276] 0709 13:05:09.831815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.159506 (+2327691us) service_pool.cc:224] Handling call
0709 13:05:12.159939 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.196725 (+ 36786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.196921 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:12.160745 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.118486 32677 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.141651 32660 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:12.154942 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:12.147151 32383 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.169400 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.177564 32533 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:12.156263 32672 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.179872 32645 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:12.180850 32542 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:12.141782 32618 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:12.118161 32662 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.148324 32634 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:12.192848 32580 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:12.061414 32670 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:12.264638 32670 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.267112 32670 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:11.994617 32426 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.161150 29069 rpcz_store.cc:276] 0709 13:05:09.814519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.309699 (+1495180us) service_pool.cc:224] Handling call
0709 13:05:11.310084 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:11.354706 (+ 44622us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:11.629449 (+274743us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.827451 (+198002us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:11.844900 (+ 17449us) log.cc:844] Serialized 26 byte log entry
0709 13:05:11.845384 (+   484us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:11.845503 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.845657 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:12.052840 (+207183us) raft_consensus.cc:1685] finished
0709 13:05:12.052861 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.053049 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44117,"prepare.queue_time_us":382049,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":141315,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":38397,"spinlock_wait_cycles":15942528,"thread_start_us":377754,"threads_started":2,"wal-append.queue_time_us":321871}
W20260709 13:05:12.202706 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47858.3us
I20260709 13:05:12.273401 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:12.143450 32561 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:12.247845 32539 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.739s	user 0.009s	sys 0.003s
I20260709 13:05:12.204924 29038 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:12.206769 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 367) took 2375 ms. Trace:
I20260709 13:05:12.275825 29039 leader_election.cc:400] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:05:12.277781 29070 rpcz_store.cc:276] 0709 13:05:09.831571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.158807 (+2327236us) service_pool.cc:224] Handling call
0709 13:05:12.206191 (+ 47384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.206401 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:12.206433 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:12.206479 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:12.206529 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.206590 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.206705 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.278621 29039 leader_election.cc:403] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:12.209506 29147 raft_consensus.cc:2410] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:12.217599 29065 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:05:12.218575 29073 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:12.224555 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:12.279999 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 368) took 2444 ms. Trace:
W20260709 13:05:12.279975 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55499us
I20260709 13:05:12.280649 29065 rpcz_store.cc:276] 0709 13:05:09.835796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.160928 (+2325132us) service_pool.cc:224] Handling call
0709 13:05:12.217254 (+ 56326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:12.279487 (+ 62233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.279927 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.224018 32660 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.224934 32677 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:12.233767 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.281042 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:12.194057 32703 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 13:05:12.205919 32700 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.283615 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:12.244704 29064 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:12.209895 32482 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:12.260336 29071 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:12.255788 32618 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.216444 32666 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:12.261643 32383 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:12.262805 32389 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.526s	user 0.003s	sys 0.007s
W20260709 13:05:12.249660 32565 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.912s	user 0.011s	sys 0.001s
I20260709 13:05:12.260591 32448 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:12.263418 32652 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.253926 32672 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:12.291045 29148 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:12.293370 29070 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:12.294266 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 372) took 2457 ms. Trace:
I20260709 13:05:12.290849 29113 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:12.258853 32628 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.308053 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2045) took 2436 ms. Trace:
I20260709 13:05:12.296442 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:12.310487 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.311487 32565 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:12.310166 29070 rpcz_store.cc:276] 0709 13:05:09.871155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.279004 (+2407849us) service_pool.cc:224] Handling call
0709 13:05:12.284290 (+  5286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.307773 (+ 23483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.307983 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.307543 32666 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:12.314098 29070 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:12.315009 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2048) took 2367 ms. Trace:
I20260709 13:05:12.315459 29070 rpcz_store.cc:276] 0709 13:05:09.947983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.313468 (+2365485us) service_pool.cc:224] Handling call
0709 13:05:12.313874 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.314702 (+   828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.314937 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.316527 32673 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.317059 32673 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.282503 32655 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:05:12.299374 32688 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:05:12.320742 32688 consensus_queue.cc:1243] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:12.320823 32673 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:12.297849 29065 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:05:12.301066 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 96274.2us
I20260709 13:05:12.282680 32623 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:12.295220 32357 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:12.306221 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:12.307866 29071 rpcz_store.cc:276] 0709 13:05:09.836963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.207141 (+2370178us) service_pool.cc:224] Handling call
0709 13:05:12.260150 (+ 53009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.293892 (+ 33742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.294189 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.306537 32654 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:05:12.293509 32482 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.289762 32645 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:12.293953 32670 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:12.295881 32618 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:12.324101 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.341979 29908 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:12.342429 29214 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:12.344331 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:12.350979 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2047) took 2427 ms. Trace:
I20260709 13:05:12.353854 29065 rpcz_store.cc:276] 0709 13:05:09.923534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.282197 (+2358663us) service_pool.cc:224] Handling call
0709 13:05:12.297635 (+ 15438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:12.343520 (+ 45885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.350862 (+  7342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.306752 32660 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:12.339049 32623 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.287019 32539 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.341817 32652 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:12.341519 32357 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.299685 32389 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.335680 32628 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:12.464129 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.295364 32700 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:12.490374 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.390592 32625 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.360889 32645 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.361373 32482 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:12.368191 32351 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:12.512841 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:12.468084 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.390398 32623 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:12.352980 32586 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.399242 32357 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:12.495009 29071 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:05:12.501694 32679 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.306946 32448 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:12.503484 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 159286us
I20260709 13:05:12.499986 32670 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:12.511435 32535 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.000s
W20260709 13:05:12.504451 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 181591us
I20260709 13:05:12.519120 29071 pending_rounds.cc:85] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 4
I20260709 13:05:12.529412 29071 pending_rounds.cc:107] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:05:12.529665 29071 raft_consensus.cc:2889] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:12.506695 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 115019us
W20260709 13:05:12.531077 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.508309 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:12.513504 29215 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:12.518972 29072 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:12.519838 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:12.391141 29065 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:12.508329 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:12.521911 29066 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:12.532509 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1929) took 1573 ms. Trace:
I20260709 13:05:12.532944 29065 rpcz_store.cc:276] 0709 13:05:10.959000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.355419 (+1396419us) service_pool.cc:224] Handling call
0709 13:05:12.390879 (+ 35460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.532274 (+141395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.532459 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.529826 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:12.519995 29220 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:12.523104 32679 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:12.534610 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:12.524993 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:12.526556 29064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 374) took 2689 ms (client timeout 1750 ms). Trace:
I20260709 13:05:12.527590 32572 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:12.530437 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2051) took 2398 ms. Trace:
I20260709 13:05:12.523042 32448 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:12.514974 32351 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.544169 29071 rpcz_store.cc:276] 0709 13:05:10.132319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.339399 (+2207080us) service_pool.cc:224] Handling call
0709 13:05:12.494565 (+155166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.530154 (+ 35589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.530378 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:12.536940 29064 rpcz_store.cc:269] 0709 13:05:09.837468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.082613 (+245145us) service_pool.cc:224] Handling call
0709 13:05:12.526316 (+2443703us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54002ac454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:12.526464 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3981903744}
I20260709 13:05:12.512629 32645 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:12.545573 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:12.445597 32580 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.577550 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:12.529024 32670 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:12.581343 29065 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:12.583884 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:12.580253 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:12.586621 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2054) took 1253 ms. Trace:
I20260709 13:05:12.587069 29065 rpcz_store.cc:276] 0709 13:05:11.333602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.547467 (+1213865us) service_pool.cc:224] Handling call
0709 13:05:12.581224 (+ 33757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.581712 (+   488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.586551 (+  4839us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:12.550290 32351 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:12.547609 32426 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:12.555529 32629 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:12.535981 32535 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.530112 32586 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.584134 32641 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:12.586776 32554 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:12.583613 32662 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:12.614878 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.616156 32709 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 13:05:12.618152 32700 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:12.621660 32619 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:12.623823 32539 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:12.626195 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:12.627408 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:12.627578 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2036) took 2809 ms. Trace:
I20260709 13:05:12.627987 29067 rpcz_store.cc:276] 0709 13:05:09.818070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.690288 (+1872218us) service_pool.cc:224] Handling call
0709 13:05:11.690643 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.128960 (+438317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:12.129010 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:12.129072 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:12.129125 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.627357 (+498232us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54002ad854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:12.627401 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.627522 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007312640}
I20260709 13:05:12.629846 32628 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:12.631127 29908 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 599) took 2642 ms (client timeout 2171 ms). Trace:
W20260709 13:05:12.631501 29908 rpcz_store.cc:269] 0709 13:05:09.989151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.990210 (+  1059us) service_pool.cc:224] Handling call
0709 13:05:12.630955 (+2640745us) spinlock_profiling.cc:243] Waited 2.34 s on lock 0x7b54004141d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:12.631057 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5374629760}
I20260709 13:05:12.634413 32542 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:12.634940 32542 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.607272 32629 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.636195 32629 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:12.637022 32542 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:12.637562 32542 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.637943 32542 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.640095 32253 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.640587 32253 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.640945 32542 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:12.642623 32253 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:12.646649 32675 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:12.647130 32675 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:12.650369 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2049) took 2691 ms. Trace:
W20260709 13:05:12.652992 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 133243us
W20260709 13:05:12.653133 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.654592 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:12.597441 32426 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.650826 29070 rpcz_store.cc:276] 0709 13:05:09.959100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.316619 (+2357519us) service_pool.cc:224] Handling call
0709 13:05:12.316894 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.317099 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:12.317377 (+   278us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.512122 (+194745us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.649741 (+137619us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b180046f840. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:05:12.649956 (+   215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:12.650023 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:12.650071 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.650165 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.650301 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1611,"prepare.queue_time_us":2727,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":662,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3573,"spinlock_wait_cycles":27151104,"thread_start_us":194174,"threads_started":1,"wal-append.queue_time_us":298449}
I20260709 13:05:12.655799 29038 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:12.664137 29139 raft_consensus.cc:2368] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10.
I20260709 13:05:12.663861 32619 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:12.611099 32634 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:12.610960 32641 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.614877 32554 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:12.669090 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:12.670780 29146 raft_consensus.cc:2410] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:05:12.673149 32603 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.001s
I20260709 13:05:12.680123 32705 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:12.680116 29067 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:12.689133 29070 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:12.690991 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:12.695798 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:12.696940 29904 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:12.701862 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:12.702836 29908 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:12.704270 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:12.710855 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:12.711201 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:12.711488 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:12.711804 29071 raft_consensus.cc:2376] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11. Re-sending same reply.
I20260709 13:05:12.713145 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:12.713555 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:12.713838 29071 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:12.714931 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:12.716945 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:12.717176 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:12.717537 29071 raft_consensus.cc:2376] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9. Re-sending same reply.
I20260709 13:05:12.717712 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:12.731029 32565 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:12.720008 32625 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:12.737283 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:12.751807 29221 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:12.752934 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:12.754333 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:12.739280 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.756297 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:12.759121 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:12.740513 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:12.763520 29216 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:05:12.745334 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.731148 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 418296us
I20260709 13:05:12.765066 32586 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:12.806975 29904 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:12.765196 32697 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:12.765874 32436 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:12.811137 32352 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:05:12.746940 32565 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.733958 32602 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:12.769091 32696 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.809s	user 0.010s	sys 0.000s
I20260709 13:05:12.770543 32080 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:05:12.773144 32682 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.746s	user 0.002s	sys 0.002s
I20260709 13:05:12.662022 32426 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:12.783702 32625 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:05:12.784624 32713 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
W20260709 13:05:12.785383 32718 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:05:12.786972 32253 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:12.787416 29875 connection.cc:256] RPC call timeout handler was delayed by -2.5419141579995994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 13:05:12.920188 32352 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.919852 32253 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.920859 32352 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.922379 32253 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:12.922875 32352 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:12.923449 29875 connection.cc:256] RPC call timeout handler was delayed by -1.6394724909996512s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
W20260709 13:05:12.923964 29875 connection.cc:256] RPC call timeout handler was delayed by -1.5311919529995066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
I20260709 13:05:12.925836 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:12.926740 29902 raft_consensus.cc:2410] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:12.661348 32679 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:12.927386 32654 consensus_meta.cc:287] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.566s	user 0.008s	sys 0.000s
I20260709 13:05:12.927755 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:12.927645 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:12.666154 32675 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.817008 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:12.669085 32634 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:12.821698 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.826632 32080 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:12.818629 32697 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:12.822783 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:12.813764 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:12.677768 32641 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:12.810521 32632 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.922s	user 0.009s	sys 0.003s
I20260709 13:05:12.794144 32554 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:12.834969 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:12.850032 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.855738 29871 leader_election.cc:336] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.515s (SENT)
W20260709 13:05:12.952332 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 96654us
W20260709 13:05:12.953269 29871 leader_election.cc:336] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.527s (SENT)
I20260709 13:05:12.953807 29871 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:12.955271 29871 leader_election.cc:336] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.608s (SENT)
I20260709 13:05:12.893185 29070 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:12.900734 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:12.812911 32706 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.821696 32618 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.914198 32614 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:12.817318 32389 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:12.926208 32623 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:12.927265 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:12.672797 32619 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:12.944795 32580 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:12.933946 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:12.979630 29139 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:12.933975 29144 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:12.981061 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.934288 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.934222 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.982575 29068 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:12.934506 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.934463 29909 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:05:12.984107 29909 pending_rounds.cc:85] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 6
I20260709 13:05:12.984404 29909 pending_rounds.cc:107] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:05:12.984710 29909 raft_consensus.cc:2889] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:12.935251 29904 raft_consensus.cc:2410] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:05:12.936261 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:12.937915 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 357703us
I20260709 13:05:12.982124 29145 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:05:12.989297 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.989655 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:12.941303 29037 connection.cc:256] RPC call timeout handler was delayed by -1.0001017109998429s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
I20260709 13:05:12.932229 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:12.928828 29902 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:12.966171 32539 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:12.935055 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 167059us
I20260709 13:05:12.957579 29903 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:12.958307 29216 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:12.955790 29871 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:13.004288 29874 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:12.968303 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:12.971081 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:12.933658 32697 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:12.927889 32654 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 6. Current term is 7.
I20260709 13:05:12.929657 32634 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:12.973834 29907 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:12.789109 32535 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:12.960682 32660 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.003s
I20260709 13:05:12.932897 32352 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:12.974575 32651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.167s	user 0.012s	sys 0.012s
I20260709 13:05:12.991554 29142 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 5.
I20260709 13:05:12.929491 29908 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:12.990036 29145 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:12.998196 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:13.034693 29905 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:13.000748 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:13.035791 29141 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:13.001561 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:12.960391 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:12.982097 32580 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.008687 29222 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:05:13.009543 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:13.010807 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:13.013626 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:13.051584 29909 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:12.931406 32645 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.964324 32542 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:05:13.023690 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.072445 29070 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:13.073742 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.075196 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:12.948282 32632 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.930403 32080 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:13.041728 32651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.226s	user 0.012s	sys 0.012s
I20260709 13:05:12.972712 32614 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.014432 32389 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.050233 29904 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:13.060443 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:13.092655 29145 raft_consensus.cc:2410] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:05:13.016584 32539 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:12.968664 32706 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.065487 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:13.098070 29141 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.098208 29147 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:13.022219 32586 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:05:13.035243 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.014578 32662 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:05:13.013980 32661 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:05:13.069912 32705 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:13.075891 29070 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 6.
I20260709 13:05:13.017486 32605 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:13.120108 32605 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.120790 32605 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.042718 32702 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:05:13.130540 29139 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:13.099568 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2060) took 2166 ms. Trace:
I20260709 13:05:13.138640 29219 rpcz_store.cc:276] 0709 13:05:10.932972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.933148 (+   176us) service_pool.cc:224] Handling call
0709 13:05:10.955286 (+ 22138us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:10.972410 (+ 17124us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:11.099700 (+127290us) log.cc:844] Serialized 14 byte log entry
0709 13:05:11.537914 (+438214us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:11.552955 (+ 15041us) log.cc:844] Serialized 52 byte log entry
0709 13:05:11.553105 (+   150us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:11.553144 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.553209 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:12.772503 (+1219294us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:13.099198 (+326695us) raft_consensus.cc:1685] finished
0709 13:05:13.099291 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.099495 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30319,"prepare.queue_time_us":324964,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":9234,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":30350,"spinlock_wait_cycles":6822272,"thread_start_us":472169,"threads_started":2,"wal-append.queue_time_us":801967}
I20260709 13:05:13.141870 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.143142 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.021142 32178 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:05:13.144585 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.145545 29219 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:13.077600 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:13.146363 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 68909.4us
I20260709 13:05:13.146476 32702 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:13.146977 32702 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:13.147388 32702 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.111243 29213 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:13.149258 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:13.149989 29911 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
W20260709 13:05:13.115207 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.083007 32672 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.741s	user 0.008s	sys 0.001s
I20260709 13:05:13.150985 32672 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.152562 32618 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:13.153002 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 377) took 3314 ms. Trace:
I20260709 13:05:13.153265 29069 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 11. Ops: []
I20260709 13:05:13.153399 29068 rpcz_store.cc:276] 0709 13:05:09.838328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.211462 (+2373134us) service_pool.cc:224] Handling call
0709 13:05:12.211955 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.983087 (+771132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.152784 (+169697us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b54003edad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:13.152943 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1771834112}
I20260709 13:05:13.154697 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.042569 32352 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.157920 32352 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.158140 32623 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:13.095276 32700 consensus_queue.cc:1175] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a936878f59c840a6bd176a1d2d7347b1" responder_term: 10 status { last_received { term: 7 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:13.082129 32182 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:05:13.063774 32652 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:05:13.160287 32182 consensus_queue.cc:1243] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:13.160897 32182 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:13.161232 32182 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:13.161445 32536 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:13.161896 32536 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:13.162130 32182 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.132241 32660 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:05:13.132826 32723 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:05:13.094883 32080 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:13.095626 32683 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:05:13.134656 32706 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.119704 32586 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:13.136590 32670 consensus_meta.cc:287] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.000s
I20260709 13:05:13.058874 32572 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.149471 29214 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:05:13.091715 32351 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.152133 29148 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:13.059778 32580 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.155786 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.100350 32389 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.116894 32705 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:13.163666 29073 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 373) took 3326 ms (client timeout 2069 ms). Trace:
I20260709 13:05:13.114265 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.228863 29905 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
W20260709 13:05:13.227205 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 147934us
W20260709 13:05:13.229744 29073 rpcz_store.cc:269] 0709 13:05:09.837205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.153391 (+316186us) service_pool.cc:224] Handling call
0709 13:05:13.163479 (+3010088us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54002cbad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:13.163596 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3856362240}
W20260709 13:05:13.095347 32357 consensus_meta.cc:287] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.001s
W20260709 13:05:13.143718 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 338111us
I20260709 13:05:13.232328 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.167021 29141 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:13.167697 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.063607 32354 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:05:13.175547 29217 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:05:13.175820 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 238312us
I20260709 13:05:13.247468 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:13.248076 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:05:13.248553 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 293307us
I20260709 13:05:13.205348 32182 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:13.208243 32482 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.687s	user 0.000s	sys 0.010s
I20260709 13:05:13.251633 29064 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:13.209041 32614 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:13.169910 32641 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:13.252432 29072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2056) took 1913 ms (client timeout 2236 ms). Trace:
W20260709 13:05:13.252837 29072 rpcz_store.cc:269] 0709 13:05:11.339022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.492230 (+153208us) service_pool.cc:224] Handling call
0709 13:05:13.252205 (+1759975us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001d8b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:13.252370 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2358924672}
I20260709 13:05:13.254002 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.254601 29072 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
W20260709 13:05:13.216169 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.217471 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 213019us
W20260709 13:05:13.256760 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.255427 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.257722 29072 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:13.258867 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.223148 29186 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 2.008s (SENT)
W20260709 13:05:13.201545 32561 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.894s	user 0.006s	sys 0.005s
I20260709 13:05:13.151316 32253 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:13.169118 32660 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:13.211710 32539 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:13.147414 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:13.230918 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.142305 32554 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.233594 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:13.296325 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 62809us
I20260709 13:05:13.294943 32561 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.251613 32721 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:05:13.233655 29070 raft_consensus.cc:2376] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11. Re-sending same reply.
I20260709 13:05:13.316845 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.317606 29070 raft_consensus.cc:2376] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7. Re-sending same reply.
I20260709 13:05:13.318486 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.319104 29070 raft_consensus.cc:2376] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9. Re-sending same reply.
I20260709 13:05:13.238979 29068 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:13.223239 32652 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.325094 32652 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.325664 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.326534 29068 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:13.327335 32652 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.249222 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:13.328184 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 79063.7us
I20260709 13:05:13.329352 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:13.327579 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.259503 29213 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:13.330782 29068 raft_consensus.cc:2376] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7. Re-sending same reply.
I20260709 13:05:13.331796 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.331744 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.333600 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.231709 32705 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.268436 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:13.269701 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:13.235512 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 223522us
I20260709 13:05:13.272331 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.262395 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:13.336138 29214 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:13.271790 29911 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:13.275772 32351 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.279227 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:13.402061 29909 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.224406 32670 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.279501 29039 leader_election.cc:400] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10.
I20260709 13:05:13.410589 29039 leader_election.cc:403] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:05:13.411378 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 132030us
W20260709 13:05:13.284564 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 61443.4us
I20260709 13:05:13.257113 32572 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:13.292273 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:13.293169 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.268088 32628 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:13.420830 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:13.422395 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:13.423154 32602 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:05:13.423995 32572 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.425640 32572 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:13.428511 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.428303 32571 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:13.428907 32389 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:13.429497 32389 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:13.429608 32571 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:13.430166 32389 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:13.236258 32357 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.430344 32571 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:13.431274 32354 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:13.300318 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.222177 32586 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.244792 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.308900 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.208426 32645 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.432273 32357 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.431802 32354 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.431691 32571 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.461412 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.462188 32354 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.433104 32653 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.334946 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.341432 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:13.221010 29908 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:13.439734 29142 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:13.469156 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 127823us
I20260709 13:05:13.298094 32706 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:13.303815 32561 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.308727 32535 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:05:13.304855 32539 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:13.355093 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 92939.5us
I20260709 13:05:13.484112 32539 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.485549 32618 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:13.304023 32436 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:13.486651 32618 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:05:13.357039 32629 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.720s	user 0.011s	sys 0.000s
I20260709 13:05:13.487058 32618 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:13.487581 32623 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:13.487404 32629 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.364346 32683 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:13.365949 32352 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:13.359529 32542 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:13.293720 32614 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:05:13.372112 32632 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:13.375646 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2040) took 3530 ms. Trace:
I20260709 13:05:13.501677 29069 rpcz_store.cc:276] 0709 13:05:09.844838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.270418 (+2425580us) service_pool.cc:224] Handling call
0709 13:05:12.270777 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.153756 (+882979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.375351 (+221595us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b54002085d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:13.375567 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2029133056}
I20260709 13:05:13.380421 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2195) took 1077 ms. Trace:
I20260709 13:05:13.502844 29140 rpcz_store.cc:276] 0709 13:05:12.302888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.303093 (+   205us) service_pool.cc:224] Handling call
0709 13:05:12.747850 (+444757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.748094 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:12.748396 (+   302us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.994396 (+246000us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.995059 (+   663us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:12.995107 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:12.995150 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.995213 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.146618 (+151405us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b180022a200. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20522 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:13.216262 (+ 69644us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b180022a200. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20522 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:13.378484 (+162222us) spinlock_profiling.cc:243] Waited 7.87 ms on lock 0x7b180022a200. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:13.379636 (+  1152us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b180022a200. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4
0709 13:05:13.380233 (+   597us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b540029d450. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:13.380354 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21884,"prepare.queue_time_us":8901,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":91,"raft.queue_time_us":550,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":13643,"spinlock_wait_cycles":1100619776,"thread_start_us":244681,"threads_started":1,"wal-append.queue_time_us":487830}
I20260709 13:05:13.381628 32554 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.296104 32625 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:05:13.439394 32670 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.269279 32634 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.291778 32253 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:13.263571 32482 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:13.383430 32712 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.710s	user 0.001s	sys 0.002s
I20260709 13:05:13.384738 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:13.516563 29216 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:13.387262 29903 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:13.297354 32672 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.390283 32536 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.391690 32707 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:13.383477 32702 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.392459 32660 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.393618 32675 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:13.393833 32726 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:05:13.406472 29072 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:13.557112 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.558466 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.409816 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2038) took 3582 ms. Trace:
I20260709 13:05:13.559360 29066 rpcz_store.cc:276] 0709 13:05:09.827207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.849041 (+2021834us) service_pool.cc:224] Handling call
0709 13:05:11.849453 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.532877 (+683424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.409371 (+876494us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b5400321ed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:13.409705 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546106112}
I20260709 13:05:13.560817 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.561507 29066 raft_consensus.cc:2376] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7. Re-sending same reply.
I20260709 13:05:13.562541 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.430586 32389 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:13.441669 32641 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:13.433775 29143 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:05:13.462700 32586 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:13.576645 29067 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:13.492444 32653 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.450167 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.464497 29073 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:13.464972 29220 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:13.332079 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.300922 29906 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:13.462401 29065 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:13.467327 32426 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.001s
I20260709 13:05:13.581394 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.582223 29071 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:13.448410 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:13.456117 32287 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:13.471115 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 65123.2us
I20260709 13:05:13.472119 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:13.495472 32706 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.474381 32703 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:13.475132 32351 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:13.477365 32572 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:13.612921 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 334461us
I20260709 13:05:13.566563 32670 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:13.479125 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:13.480897 32645 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:13.482419 32561 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.486356 32539 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:13.486404 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.488750 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.493772 29909 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:13.500504 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:13.509682 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.486969 32721 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:13.382272 32605 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.517445 29902 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:05:13.534545 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:13.359684 32697 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.645112 29216 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.511384 32735 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:13.570938 29215 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:13.538730 32542 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:05:13.473423 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 181330us
I20260709 13:05:13.657135 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:13.657362 29216 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:13.588826 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.594038 32426 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.595191 32672 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:13.595626 32705 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.550834 32707 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:13.668119 29910 raft_consensus.cc:2410] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:13.603713 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.477795 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:13.615669 29871 leader_election.cc:336] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.750s (SENT)
I20260709 13:05:13.685659 29213 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.685803 29871 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:13.641601 29214 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:13.643229 32389 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.688257 32705 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:13.674925 32629 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:13.597904 32641 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.642207 32670 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.689545 32641 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.671418 32735 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.691254 32670 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.658021 29220 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:13.691624 32641 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:13.513248 32618 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.692101 32735 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:13.692466 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:13.692688 32618 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.693629 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:13.694646 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:13.695148 29220 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.615204 32623 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:05:13.695899 32705 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:13.696316 32705 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:13.658483 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:13.698887 29216 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.699347 32634 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:13.668953 32602 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10.
I20260709 13:05:13.701471 32536 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.642820 32645 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.674190 29221 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:13.678267 32482 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.668215 29218 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 9: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:13.660580 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:13.705667 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 45205.3us
I20260709 13:05:13.706880 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:13.688522 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:13.708498 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 92869us
I20260709 13:05:13.644604 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:13.689010 32707 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:13.674535 32426 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:13.710899 29216 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:13.710985 32426 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:13.711627 32426 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:13.711328 29217 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.713290 32426 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.669935 32702 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.671186 32706 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.677390 32653 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:13.673823 32572 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.697268 32705 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.670989 32660 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.688670 32389 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.667327 32605 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:13.679844 32697 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.660147 32721 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:13.662748 32580 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.721166 32705 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.722186 32580 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.704713 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:13.709468 32733 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:13.723565 29221 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 9: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:13.709657 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.661974 32542 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.712008 29214 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:13.650802 32253 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:13.712141 32641 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:13.698423 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.664466 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 506) took 1168 ms. Trace:
I20260709 13:05:13.731238 29215 rpcz_store.cc:276] 0709 13:05:12.495695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.627162 (+131467us) service_pool.cc:224] Handling call
0709 13:05:13.664225 (+1037063us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54003552d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:13.664372 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152539008}
I20260709 13:05:13.732282 32554 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:13.722751 29222 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:13.733788 29215 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: [4.3-8.4]
I20260709 13:05:13.734167 32253 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:13.736264 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:13.737859 32641 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:13.738652 32641 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:13.741731 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:13.608314 32287 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:13.638106 29219 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:13.750913 29871 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.891s (SENT)
I20260709 13:05:13.740867 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:13.753659 32733 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:13.754243 32287 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.746691 29213 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:13.753247 32352 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:13.712011 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:13.750836 29217 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:13.703719 32645 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:13.765648 29218 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:13.768638 29217 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:13.770344 29217 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:13.761108 29220 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
W20260709 13:05:13.774328 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 445013us
I20260709 13:05:13.775275 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:13.776743 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:13.777897 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:13.779083 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:13.780260 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:13.781348 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:13.787161 29219 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:12.062295 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:13.931284 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 526972us
W20260709 13:05:13.583881 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 111828us
W20260709 13:05:13.934481 28968 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 262) took 18353 ms (client timeout 15000 ms). Trace:
I20260709 13:05:13.432940 28969 catalog_manager.cc:5697] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.943794 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:14.037065 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2064) took 1208 ms. Trace:
I20260709 13:05:13.534439 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:13.946203 29066 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:13.589946 29070 raft_consensus.cc:2376] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6. Re-sending same reply.
I20260709 13:05:14.045845 28969 catalog_manager.cc:5697] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.047155 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1950) took 1210 ms. Trace:
I20260709 13:05:13.946426 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 1946) took 1229 ms. Trace:
I20260709 13:05:14.048148 29067 rpcz_store.cc:276] 0709 13:05:12.716840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.716966 (+   126us) service_pool.cc:224] Handling call
0709 13:05:13.945776 (+1228810us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b54000686d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:13.946176 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1945671040}
I20260709 13:05:14.048411 28969 catalog_manager.cc:5697] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 4 to 8, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 8 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.050817 29067 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:13.496312 29068 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:14.058004 29068 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:13.948935 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2068) took 1118 ms. Trace:
I20260709 13:05:13.949786 29072 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:13.591949 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:13.972406 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.973431 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.973716 29064 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:13.797264 29220 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:13.881493 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 130597us
W20260709 13:05:14.069171 29871 leader_election.cc:336] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.069s (SENT)
W20260709 13:05:14.069623 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.071668 29073 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 9: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:13.773748 32352 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:14.073098 32352 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:13.793915 32554 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:14.075577 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:13.795675 32572 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:14.076938 29910 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:13.750445 32605 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:05:13.739874 32641 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:14.078449 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:14.080181 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:13.795091 32675 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:13.893496 32757 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.082659 32675 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:14.083314 32675 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:14.082989 32757 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:14.035284 28968 rpcz_store.cc:269] 0709 13:04:55.609054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.609223 (+   169us) service_pool.cc:224] Handling call
0709 13:05:13.934183 (+18324960us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:03.157567 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:03.208830 (+ 51263us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:03.208884 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:05:03.208914 (+    30us) tablet.cc:662] Decoding operations
0709 13:05:03.263558 (+ 54644us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:03.263682 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:03.263729 (+    47us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:05:03.266315 (+  2586us) tablet.cc:701] Row locks acquired
0709 13:05:03.266354 (+    39us) write_op.cc:260] PREPARE: finished
0709 13:05:03.266610 (+   256us) write_op.cc:270] Start()
0709 13:05:03.266791 (+   181us) write_op.cc:276] Timestamp: P: 1783602303266527 usec, L: 0
0709 13:05:03.266825 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:05:03.445855 (+179030us) log.cc:844] Serialized 8283 byte log entry
0709 13:05:03.936648 (+490793us) op_driver.cc:464] REPLICATION: finished
0709 13:05:04.041779 (+105131us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f7833a030ac 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f783a543ef0
0709 13:05:04.451442 (+409663us) write_op.cc:301] APPLY: starting
0709 13:05:04.451562 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:04.473604 (+ 22042us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:04.473653 (+    49us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:04.525721 (+ 52068us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:04.526232 (+   511us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=21
0709 13:05:04.526281 (+    49us) write_op.cc:312] APPLY: finished
0709 13:05:04.527055 (+   774us) log.cc:844] Serialized 145 byte log entry
0709 13:05:04.532101 (+  5046us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:04.532544 (+   443us) write_op.cc:454] Released schema lock
0709 13:05:04.532665 (+   121us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:04.601592 (+ 68927us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:04.601633 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1000,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":511952,"apply.run_cpu_time_us":7702,"apply.run_wall_time_us":136134,"num_ops":21,"prepare.queue_time_us":534444,"prepare.run_cpu_time_us":35064,"prepare.run_wall_time_us":521736,"raft.queue_time_us":80796,"raft.run_cpu_time_us":66793,"raft.run_wall_time_us":495073,"replication_time_us":669618,"spinlock_wait_cycles":28813184,"thread_start_us":1003608,"threads_started":4,"wal-append.queue_time_us":239110,"wal-append.run_cpu_time_us":8556,"wal-append.run_wall_time_us":1847741}]]}
I20260709 13:05:14.088814 32571 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:14.089399 32571 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:14.089867 32571 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:13.876384 32707 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.092913 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2199) took 1769 ms. Trace:
I20260709 13:05:14.093608 32707 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:13.887615 32731 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:14.095088 32731 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.095628 32731 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:14.102273 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2075) took 1033 ms. Trace:
I20260709 13:05:13.747565 32735 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:14.102718 29064 rpcz_store.cc:276] 0709 13:05:13.068807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.448021 (+379214us) service_pool.cc:224] Handling call
0709 13:05:14.102060 (+654039us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54002abf54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:14.102194 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":813364992}
I20260709 13:05:13.892976 32742 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:14.006734 29186 leader_election.cc:336] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 1.908s (SENT)
W20260709 13:05:14.011459 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.104399 29902 raft_consensus.cc:2376] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11. Re-sending same reply.
I20260709 13:05:14.105234 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:13.744385 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:14.028712 32389 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:14.106351 32742 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.106748 32389 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.107074 32742 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.109304 32389 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:14.109519 32572 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:14.043154 29069 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:14.060101 29071 rpcz_store.cc:276] 0709 13:05:12.829936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.947390 (+1117454us) service_pool.cc:224] Handling call
0709 13:05:13.948007 (+   617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.948421 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.948478 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.948534 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:13.948577 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.948642 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.948749 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:14.110131 32572 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:14.060014 28969 catalog_manager.cc:5697] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 2 to 3, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:14.040166 32354 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.112378 32354 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:14.114018 29071 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:05:14.067109 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:13.794396 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:14.130113 29071 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:14.129592 32572 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:14.131217 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 336775us
I20260709 13:05:13.825203 32586 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.080405 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:14.133330 29910 raft_consensus.cc:2368] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12.
I20260709 13:05:14.132483 28969 catalog_manager.cc:5697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 7, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 7 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:14.134298 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1957) took 1036 ms. Trace:
I20260709 13:05:14.134424 32586 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:14.134735 29068 rpcz_store.cc:276] 0709 13:05:13.098156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.059277 (+961121us) service_pool.cc:224] Handling call
0709 13:05:14.059760 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.133925 (+ 74165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:14.133957 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:14.134013 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:14.134048 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.134112 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.134223 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:14.138633 32354 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:14.141772 32605 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:14.142342 32605 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:14.141032 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 134588us
W20260709 13:05:14.144559 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 400079us
I20260709 13:05:14.145561 32539 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:14.146117 32539 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:14.146334 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 79576us
I20260709 13:05:14.146651 32539 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:14.146910 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:13.796131 32734 raft_consensus.cc:2764] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:05:14.147861 29113 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:14.037501 29065 rpcz_store.cc:276] 0709 13:05:12.828660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.942818 (+1114158us) service_pool.cc:224] Handling call
0709 13:05:14.036974 (+ 94156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:14.051834 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1951) took 1112 ms. Trace:
I20260709 13:05:13.892238 32733 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:14.096700 32645 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.149290 32354 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:14.141431 29071 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:05:14.141785 28969 catalog_manager.cc:5697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 3 to 8, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 8 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.148284 32629 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:14.150617 29071 pending_rounds.cc:85] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 8
I20260709 13:05:14.150422 32645 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:14.150401 29112 leader_election.cc:336] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 1.928s (SENT)
I20260709 13:05:14.151957 29071 pending_rounds.cc:107] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:05:14.152943 29071 raft_consensus.cc:2889] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:14.150110 29067 rpcz_store.cc:276] 0709 13:05:12.938962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.049748 (+1110786us) service_pool.cc:224] Handling call
0709 13:05:14.050418 (+   670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.051426 (+  1008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.051730 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:14.047780 29070 rpcz_store.cc:276] 0709 13:05:12.836593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.045149 (+1208556us) service_pool.cc:224] Handling call
0709 13:05:14.045802 (+   653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.046493 (+   691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:14.046529 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:14.046608 (+    79us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:14.046698 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.046769 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.046936 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":230,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":227}
I20260709 13:05:14.156253 29910 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:05:14.156785 29910 pending_rounds.cc:85] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 8
I20260709 13:05:14.157230 29910 pending_rounds.cc:107] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:05:14.157498 29910 raft_consensus.cc:2889] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:14.157660 29907 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:05:14.158097 29907 pending_rounds.cc:85] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 7
I20260709 13:05:14.158389 29907 pending_rounds.cc:107] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:05:14.156116 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.158651 29907 raft_consensus.cc:2889] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:14.158071 32645 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:14.161974 32354 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:14.163528 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:14.075162 32352 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.171204 32757 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:14.175031 32645 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:14.175769 32645 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.176271 32645 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:14.093333 29143 rpcz_store.cc:276] 0709 13:05:12.323172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.324250 (+  1078us) service_pool.cc:224] Handling call
0709 13:05:12.324616 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.571658 (+1247042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.092217 (+520559us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400190d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:14.092648 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2550176640}
I20260709 13:05:14.174222 29113 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:14.135643 32586 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:14.173113 28969 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: leader changed from <none> to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 7 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:14.060662 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 420289us
I20260709 13:05:14.205304 32605 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.206293 32742 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:14.207283 29186 leader_election.cc:336] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 1.994s (SENT)
I20260709 13:05:14.208544 32351 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.209424 32351 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.212833 32683 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:14.213028 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.213454 32683 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.213994 29910 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:14.215562 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:14.215874 29909 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:14.213917 32683 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:14.223878 29908 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:14.225378 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.226343 29911 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
W20260709 13:05:14.230494 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 152465us
I20260709 13:05:14.232586 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:14.233749 32561 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.234411 29906 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:14.235935 32705 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:14.236181 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:14.238824 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.239722 29911 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:14.207720 32605 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:14.242779 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.236059 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.189244 32389 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.219408 32586 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:05:14.261013 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 45596.7us
W20260709 13:05:14.262616 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44673.1us
I20260709 13:05:14.271505 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:14.272049 29903 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:14.269974 32705 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:14.270453 32351 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:14.296185 32536 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:14.296985 32536 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:14.297454 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:14.298524 29143 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:14.298297 32536 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:14.300133 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:14.300529 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:14.301235 29142 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:14.288035 32352 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:14.234508 32561 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.288255 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:14.283792 32389 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:14.305599 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:14.305473 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:14.306596 32287 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:14.325563 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 45461.1us
I20260709 13:05:14.282456 28969 catalog_manager.cc:5697] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 7, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 7 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.316746 32357 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:14.317873 32561 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:14.319169 32586 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.281831 32705 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.320511 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.339195 29148 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:14.340760 29142 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:14.345600 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.347812 29910 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:14.327967 32357 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:14.340601 32705 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:14.360857 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:14.363132 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 127156us
I20260709 13:05:14.304261 32354 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:14.367120 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.373795 32561 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:14.357154 32357 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:14.377872 29906 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:05:14.379323 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:14.360680 29185 leader_election.cc:336] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.867s (SENT)
I20260709 13:05:14.393165 29148 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:14.394810 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:14.400740 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:14.401541 29142 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:14.400666 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.399549 29910 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:14.403156 29906 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:14.408818 29222 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:14.425007 29903 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:05:14.353461 32287 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:14.407168 29220 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:14.427537 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.427635 29037 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:14.395210 32561 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:14.438769 29186 leader_election.cc:336] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 1.867s (SENT)
W20260709 13:05:14.442426 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 124297us
I20260709 13:05:14.437615 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:14.439217 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:14.448623 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.450356 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:14.450901 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.451603 29904 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:14.452319 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.454313 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:14.455190 29146 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:14.459595 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:14.460354 29903 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:14.463456 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:14.416056 32675 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.466853 29907 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:14.467622 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:14.465229 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:14.470616 32561 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.471391 29911 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:14.473198 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:14.486091 32675 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:14.492659 32389 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:14.493592 32389 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:14.494779 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2179) took 1098 ms. Trace:
I20260709 13:05:14.495256 29214 rpcz_store.cc:276] 0709 13:05:13.282903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.729727 (+446824us) service_pool.cc:224] Handling call
0709 13:05:13.739504 (+  9777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.739713 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:13.740311 (+   598us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.194548 (+454237us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.220293 (+ 25745us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:14.230056 (+  9763us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:14.242342 (+ 12286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.265290 (+ 22948us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.357172 (+ 91882us) spinlock_profiling.cc:243] Waited 9.45 ms on lock 0x7b540023bc50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:14.381551 (+ 24379us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18326,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":96,"raft.queue_time_us":13343,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":265,"spinlock_wait_cycles":21731712,"thread_start_us":453231,"threads_started":2,"wal-append.queue_time_us":289670}
I20260709 13:05:14.497215 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:14.504534 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 71004.6us
I20260709 13:05:14.506891 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:14.507457 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:14.507668 29143 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:14.508407 29147 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:14.508641 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:14.509192 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:14.510007 29145 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:14.506027 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:14.512370 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:14.513290 29142 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:14.509382 29139 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:14.515558 29148 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:14.515789 28969 catalog_manager.cc:5697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 5, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 5 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.518172 28969 catalog_manager.cc:5697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 2 to 8, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 8 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.526093 32571 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:14.529145 32760 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 13:05:14.533294 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51518.8us
I20260709 13:05:14.534735 29069 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:14.554301 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:14.556572 29908 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:14.541179 28969 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:14.564934 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:14.601869 29218 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:14.613816 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2079) took 1422 ms. Trace:
I20260709 13:05:14.614313 29069 rpcz_store.cc:276] 0709 13:05:13.186069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.110550 (+924481us) service_pool.cc:224] Handling call
0709 13:05:14.168190 (+ 57640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.541096 (+372906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.602328 (+ 61232us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54002ad854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:14.608516 (+  6188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832290176}
W20260709 13:05:14.616961 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60881.2us
I20260709 13:05:14.563482 32539 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:14.628141 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2109) took 1016 ms. Trace:
I20260709 13:05:14.628640 29217 rpcz_store.cc:276] 0709 13:05:13.552528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.006118 (+453590us) service_pool.cc:224] Handling call
0709 13:05:14.007074 (+   956us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:14.007855 (+   781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:14.007900 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:14.032423 (+ 24523us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.126536 (+ 94113us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:14.126594 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.126686 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:14.565758 (+439072us) raft_consensus.cc:1685] finished
0709 13:05:14.565783 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.568270 (+  2487us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":350,"raft.run_cpu_time_us":367,"raft.run_wall_time_us":367,"thread_start_us":29697,"threads_started":1,"wal-append.queue_time_us":295728}
I20260709 13:05:14.630558 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:14.632135 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 193356us
I20260709 13:05:14.636507 28969 catalog_manager.cc:5697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:14.647122 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.620314 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2076) took 1430 ms. Trace:
I20260709 13:05:14.665400 32539 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:14.668503 29214 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:14.678849 32287 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:14.621402 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:14.616523   300 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:05:14.685282 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:14.687458 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2077) took 1502 ms. Trace:
I20260709 13:05:14.690371 29218 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:14.695211 29903 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:14.662958 29073 rpcz_store.cc:276] 0709 13:05:13.185161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.072673 (+887512us) service_pool.cc:224] Handling call
0709 13:05:14.073287 (+   614us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:14.073900 (+   613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:14.073944 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:14.074677 (+   733us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.204598 (+129921us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:14.439766 (+235168us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.609948 (+170182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.610153 (+   205us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:14.610205 (+    52us) raft_consensus.cc:1685] finished
0709 13:05:14.610220 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.615702 (+  5482us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":181211,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":14047,"raft.queue_time_us":67658,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":290,"thread_start_us":468379,"threads_started":3,"wal-append.queue_time_us":294552}
W20260709 13:05:14.704756 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 163784us
I20260709 13:05:14.707648 29220 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:14.710587 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:14.688895 29072 rpcz_store.cc:276] 0709 13:05:13.185486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.094679 (+909193us) service_pool.cc:224] Handling call
0709 13:05:14.095303 (+   624us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:14.100593 (+  5290us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:14.203942 (+103349us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.536241 (+332299us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.536506 (+   265us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:14.615153 (+ 78647us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.615311 (+   158us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:14.615631 (+   320us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.615830 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.615950 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:14.680431 (+ 64481us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b180044c140. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20522 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:14.680840 (+   409us) spinlock_profiling.cc:243] Waited 5.99 ms on lock 0x7b180044c140. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:14.687210 (+  6370us) raft_consensus.cc:1685] finished
0709 13:05:14.687234 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.687386 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":289069,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":112,"raft.queue_time_us":56312,"raft.run_cpu_time_us":505,"raft.run_wall_time_us":7103,"spinlock_wait_cycles":23486208,"thread_start_us":383413,"threads_started":2,"wal-append.queue_time_us":383119}
I20260709 13:05:14.718459 32641 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:14.718273   301 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 13:05:14.719590 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 311493us
I20260709 13:05:14.720048 32536 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:14.720520 29037 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:14.720645 32536 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:14.721727 32536 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.720048   301 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:14.730373 28969 catalog_manager.cc:5697] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 2 to 5, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 5 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:14.712184 32571 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:14.744210 29219 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:14.734951 32357 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:14.746919   301 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:14.731670 32354 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:14.756289 28969 catalog_manager.cc:5697] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:14.764348 29874 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:14.696861   300 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 13:05:14.781693 32767 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:05:14.782521 28969 catalog_manager.cc:5697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 3, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 3 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:05:14.749790 32731 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.006s
I20260709 13:05:14.784237 32731 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.784415 28969 catalog_manager.cc:5697] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 3 to 7, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 7 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:14.786681 32731 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:14.786234 28969 catalog_manager.cc:5697] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:14.764139 32357 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:14.788004 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:14.790788 29902 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:05:14.765305 32683 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.507s	user 0.000s	sys 0.013s
I20260709 13:05:14.765079 32561 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:14.794299 32352 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:14.799777 32586 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:14.797600 29037 leader_election.cc:400] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12.
I20260709 13:05:14.758347 32354 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:14.800696 29037 leader_election.cc:403] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:05:14.799232 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:14.800256 32352 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:14.800019 32571 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:14.800062 32561 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.808761 32352 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.816514   303 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:14.820138 32571 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:14.821251 32571 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:14.823218 29219 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:14.831094 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 167879us
I20260709 13:05:14.796293 32683 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:14.824663 32561 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:14.763149   301 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:14.846580 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.853762 29903 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:14.854257 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.855746 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.856859 29141 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:14.856963 29911 raft_consensus.cc:2368] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9.
I20260709 13:05:14.857297 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:14.857513 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.861970 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2078) took 1676 ms. Trace:
I20260709 13:05:14.862517 29064 rpcz_store.cc:276] 0709 13:05:13.185777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.103635 (+917858us) service_pool.cc:224] Handling call
0709 13:05:14.113530 (+  9895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.114077 (+   547us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:14.570599 (+456522us) log.cc:844] Serialized 14 byte log entry
0709 13:05:14.861508 (+290909us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:14.861589 (+    81us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:14.861657 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.861747 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:14.861898 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4547,"prepare.queue_time_us":243323,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":2116,"thread_start_us":636379,"threads_started":2}
W20260709 13:05:14.848851 32645 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.668s	user 0.011s	sys 0.003s
W20260709 13:05:14.866321 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.869261 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.871714 29910 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:14.872546 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:14.876725 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 260041us
I20260709 13:05:14.878329 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.879297 29903 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:14.880421 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:14.829833 32705 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:14.883298 29908 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
W20260709 13:05:14.883708 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.884240 29072 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:14.887806 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:14.873211 32351 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:14.926728 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.935410 32705 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:14.936764 29148 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:05:14.937036 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.937844 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:14.938030 29908 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:14.938612 29141 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
W20260709 13:05:14.960090 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51956.1us
I20260709 13:05:14.961011 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:14.961887 32705 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:05:14.963267 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 278054us
I20260709 13:05:14.965193 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:14.966245   309 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:14.967212   309 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:14.967484   307 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:05:14.966322 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:14.968367 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 58735.3us
I20260709 13:05:14.968760   301 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:14.971370   309 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:14.973416 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:14.974267 29907 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:05:14.887375   304 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:05:14.984648 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 52252.8us
I20260709 13:05:14.907392 32572 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.986460 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:14.872629 32645 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.015820 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.019685 29908 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:15.020805 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:15.021405 29907 raft_consensus.cc:2376] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10. Re-sending same reply.
I20260709 13:05:14.907341 32389 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.107s
I20260709 13:05:15.025490 29064 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:05:15.054560 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:14.925302 32641 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:05:15.004920 32351 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.029819 29141 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:15.060524 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:15.061961 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 91659.4us
I20260709 13:05:15.018941 32705 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:15.064662 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:15.058128 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 95399.2us
W20260709 13:05:15.066187 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 156820us
I20260709 13:05:15.066589 29148 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:15.050568 32357 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:15.056277 32539 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:05:15.082180 29112 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:15.083817 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:15.012727 32605 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:05:14.933023 32354 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:15.086374 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 219730us
I20260709 13:05:15.060746 32586 raft_consensus.cc:2764] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:05:15.094470 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:15.097744 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:15.098603 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:15.101732 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:15.102885 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:15.025789 32572 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.116770 29139 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:05:15.117004 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 97490.8us
I20260709 13:05:15.121474 29146 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:15.097024 32389 consensus_queue.cc:1243] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:15.130527 29141 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:15.135177 29907 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:15.109297 32683 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:15.142714 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 58990.6us
I20260709 13:05:15.139349 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:05:15.170099 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.170998 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:15.135938 32539 consensus_queue.cc:1059] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:05:15.159713 32354 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:15.174837 29911 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:15.177583   313 raft_consensus.cc:2764] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:05:15.177639 32629 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:15.182233 32629 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:15.182631 32629 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:15.183519 32629 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.184825 32629 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:15.179325 32539 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:15.188146 32539 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:15.188437   310 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:05:15.189320 32539 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:15.190189   308 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:05:15.180075 32705 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.181365 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:15.205751   320 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:15.205809 32539 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:15.208066 32572 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:15.209295 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.210222 29141 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:15.175380 32352 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:15.213672 32352 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.214116 32352 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:15.217784 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 251633us
I20260709 13:05:15.219206 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:15.220335 32641 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:15.220832 32641 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.221211 32641 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:15.209422 32645 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:15.224107 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.228823   300 raft_consensus.cc:1064] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:05:15.230150   304 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:15.230608   304 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:15.234167 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:15.239115 29218 raft_consensus.cc:2410] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:15.234764 29112 leader_election.cc:400] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9.
I20260709 13:05:15.239626 29112 leader_election.cc:403] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:15.175534 32357 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:15.242857 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 107353us
I20260709 13:05:15.176687   326 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.248425 29148 raft_consensus.cc:2410] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:15.236467 32351 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:15.224078 32645 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:15.235750   303 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12.
I20260709 13:05:15.247676   320 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:15.249871 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:15.250633 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 79722.5us
I20260709 13:05:15.253520 29071 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:15.257681 29037 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:15.258721 32571 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:15.260166 32571 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:15.260092 32389 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:15.260788 32571 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.261281 32571 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:15.262120 32389 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:15.262631 32389 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:15.263773 32389 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.265596 32389 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:15.292794 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 58146.5us
I20260709 13:05:15.296744 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:15.251520 32629 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:15.168308 32536 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 13:05:15.251391 32287 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.303751   329 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:15.306749 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 173323us
W20260709 13:05:15.309044 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59345.1us
I20260709 13:05:15.231542   300 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.231575   304 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:15.321545 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 78426.2us
I20260709 13:05:15.289892   313 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:05:15.306408 32645 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:15.317361 32287 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.306350 32629 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.329088   339 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:15.331200   320 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.308391   309 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:05:15.314512 32605 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:15.340787   313 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:15.372151   313 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:15.373234 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:15.373077   313 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.374167   309 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:15.374718   313 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:15.375114   309 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:15.375660   309 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:15.341805 32357 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.343008   337 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:15.376513   337 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.343994 29185 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:15.344168 32683 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:15.347122 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:15.348011   328 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:05:15.348424   321 raft_consensus.cc:2764] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:05:15.348598 29186 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:15.349529 32645 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.361950 32629 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:05:15.362897 32389 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.363875 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.338186   339 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.366617   304 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:15.369728 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:15.378674   337 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:15.372167 32605 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.381526   301 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:15.350769 32561 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.382571   309 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.299942 32571 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.379491 29146 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:15.387003 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:15.396978 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 48508.5us
W20260709 13:05:15.407059 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.313383   326 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.411574 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:15.423434 29218 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:15.412978 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:15.403290 32539 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:15.404910   337 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:15.428942 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:15.429395 29219 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:15.431634 29214 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:05:15.432222 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:15.432621   317 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.429917 32586 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
W20260709 13:05:15.433041 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 89202.4us
I20260709 13:05:15.433300   317 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.434409 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:15.434723 29217 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: [5.4-8.6]
I20260709 13:05:15.435950 29216 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
W20260709 13:05:15.436868   332 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:05:15.437085 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:15.437582   339 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:15.437718 29219 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:15.439395 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:15.439563 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:15.440047 29217 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:15.440409 29219 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:15.440634 32287 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:15.441277 32571 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:15.442028 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:15.443672   330 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:15.443843 32586 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:15.408797   301 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:15.397832 32352 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:15.449178 29871 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.603s (SENT)
W20260709 13:05:15.453699 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 106644us
I20260709 13:05:15.454160 29220 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:15.447958 29217 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:15.404595 32389 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.460927 32683 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:15.439724   337 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:15.424203 32351 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:15.469278 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:15.471582 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:15.447642   317 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:15.476665   346 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:15.478396 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.478725 32389 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:15.479516   319 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.480257 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 46307.5us
I20260709 13:05:15.483019 32539 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:15.483331   313 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:15.484275 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.465615 32683 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:15.500092 29064 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:15.468367   337 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:15.501605 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55629.2us
W20260709 13:05:15.502048 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 115143us
I20260709 13:05:15.449295 32641 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.496160 29073 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:05:15.515094   345 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:15.515455 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:15.515617   321 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.516129 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 53077us
I20260709 13:05:15.496948   346 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.501534 32683 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:15.529635 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:15.524658 29908 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:15.499665   319 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.534538 29186 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:15.535029 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:15.536154 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.537674 29067 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:15.546187 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:15.538146 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.537830 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.551779 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:15.552616 29907 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:15.553601 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 183961us
I20260709 13:05:15.554888 29112 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:15.556486 29069 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:15.558951 29068 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
W20260709 13:05:15.565971 29188 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.952s (SENT)
I20260709 13:05:15.566999 29066 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:15.568189 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:15.498725   301 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.505092 32351 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.526078   345 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:15.576705 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:15.534406 32641 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:15.645056 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:15.645165 29186 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:15.646447 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:15.532876   346 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:15.648813 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:15.655134 29220 raft_consensus.cc:2410] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:05:15.649276 29871 leader_election.cc:336] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.718s (SENT)
I20260709 13:05:15.651535 29215 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:05:15.564015 32389 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:15.657613 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:15.658263 29219 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:15.653627 29218 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:15.660018 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:15.657481 29216 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:15.653537 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.561815   317 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:15.662828 29070 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:15.664034 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 164041us
I20260709 13:05:15.664000 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:15.666352 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.534705   339 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
I20260709 13:05:15.575961 32571 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:15.652323   329 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:15.647045 32605 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:15.647246   345 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:15.657281 32572 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.657927 32389 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.680497 29908 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:15.535704   327 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.133s
I20260709 13:05:15.663777   303 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:15.671718 32641 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:15.530591   321 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.667389   318 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:15.669420 32629 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:15.692694 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:15.712353 29070 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:15.668766 32561 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:15.677793   329 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:15.693692 32389 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:15.717204 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.690289 29907 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
W20260709 13:05:15.720681 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.708253 29071 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
W20260709 13:05:15.726344 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 123994us
W20260709 13:05:15.727975 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 173156us
W20260709 13:05:15.701093   341 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 13:05:15.671412   317 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:15.730595 29188 leader_election.cc:336] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.086s (SENT)
W20260709 13:05:15.791776 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64576.1us
I20260709 13:05:15.681087 32571 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.795393 29213 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:15.703725 32352 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:15.793521 29112 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:15.792192 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:15.795306 29067 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 6.
I20260709 13:05:15.804823 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:15.805689 29067 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:15.664510   342 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.321s
I20260709 13:05:15.731309 32641 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:15.732585   326 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:15.834405 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:15.835059 29908 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:15.703473 32572 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.803170   301 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:15.833484 29037 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:15.730959   300 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 13:05:15.798878 32536 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.729660   303 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:15.853543   355 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:05:15.855261   303 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:15.856422   351 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:15.858672 29073 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:15.859544 29069 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:05:15.861033   359 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:05:15.862385 29908 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
W20260709 13:05:15.864127   349 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:05:15.865418   344 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 13:05:15.866052 32287 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:15.866466   344 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:15.866907   344 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:15.867224 32287 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:15.867584 32287 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:15.868052   352 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.868561   352 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.868485 32287 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:15.870968   352 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:15.859707 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:15.874712 29907 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:05:15.875720 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 85152.1us
I20260709 13:05:15.887547 29073 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:15.892021 29072 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
W20260709 13:05:15.892604 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 91583.6us
I20260709 13:05:15.900866 29064 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:15.907051 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.915688 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:15.872699 32287 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:15.919407   355 consensus_queue.cc:1243] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:05:15.920265 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:15.920819   351 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:15.941066   321 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:15.981657 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.949385 32572 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:15.952888   319 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:15.953683 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:15.985049 29068 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:15.960562 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.962208 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:15.906872   326 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:15.910017   339 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:15.965286 32641 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:15.967644 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.997551 29071 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:15.876531   303 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:15.972278 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59369.8us
I20260709 13:05:15.978971 29065 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6.
I20260709 13:05:15.979581 29067 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:15.980144 29070 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:15.941432   311 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:15.982733 32629 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:15.993059 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.013794 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.014617 29065 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:16.014467 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:16.005002   304 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.603s	user 0.011s	sys 0.000s
I20260709 13:05:15.967844 32571 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:16.024087 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:16.007179 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:16.011111 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.013007 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.026192 29070 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:16.026922 32351 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:16.028247 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.028800 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.029897 29147 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:05:16.031797 32354 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.794s	user 0.009s	sys 0.004s
I20260709 13:05:16.032425   350 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:16.033851 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:16.009490 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.034715 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.034965 29148 raft_consensus.cc:2410] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:05:16.023518 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:15.901010 32536 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.018581 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:15.993233   326 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.964866   329 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.036893 29143 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:16.035609 29146 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:15.867739   344 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.039294 29073 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:16.044363 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:16.045831 29064 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:16.037662 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:16.050319 29069 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:16.050653 29065 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 9: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:16.005838   311 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:16.011960   355 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.054308 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:15.993374 32683 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.054426 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.055414 29072 raft_consensus.cc:2410] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:16.064196 29071 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:16.057850 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:16.058456 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.015973   303 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:16.043809   326 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:16.039259 32586 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.069422 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:16.057591   328 consensus_meta.cc:287] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.000s
I20260709 13:05:16.042860   351 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:16.051098   339 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.081736 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:16.083083 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.088341 29142 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
W20260709 13:05:16.086983 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.090714 29072 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:05:16.092399 29037 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:16.095238 29073 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:16.095803 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.096865 29142 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:16.098271 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:16.098472 29065 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:16.113355 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.114223 29072 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
W20260709 13:05:16.098568 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 178368us
I20260709 13:05:16.087740   328 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9.
I20260709 13:05:16.085302   344 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:16.096858   320 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:16.068943   355 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.139997 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.140588 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.149916 29072 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
W20260709 13:05:16.142477 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.148918 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.152395 29219 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:16.153990 29073 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:16.156365 29073 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:16.084964 32586 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.162452 29068 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:16.164153 29144 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:16.165549 29140 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:16.166970 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.098410 32571 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:05:16.094537   339 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.135329   321 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:16.177850 29142 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:16.151630   320 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:16.182524 29073 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:16.172397   328 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:16.186671 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 481061us
I20260709 13:05:16.188987 29069 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
W20260709 13:05:16.189440   358 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:05:16.192270 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.193149 29073 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:16.093928   351 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.130039 32629 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.115175   311 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.219466   371 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:16.173696   303 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.222091   339 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:16.224972 32586 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:16.229246 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.231882 29902 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEARNER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:16.232354 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:16.233011 29908 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:16.233546   318 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.238328 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.184569   321 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.193444   328 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.220230   371 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:16.234055   318 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:16.466161 29871 leader_election.cc:336] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.705s (SENT)
I20260709 13:05:16.407003 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.485922 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:16.491394 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.418511 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.257340 29141 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:16.254743 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:16.487782 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.492296 29143 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:16.253038 29910 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:16.490408 29146 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:16.495157 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:16.411952   354 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.571s	user 0.000s	sys 0.005s
I20260709 13:05:16.496368 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.376271 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.498073 29146 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:16.498047 29145 raft_consensus.cc:2410] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:05:16.499516 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:16.500276 32731 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:16.504978 29147 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:05:16.500458   301 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.650s	user 0.011s	sys 0.000s
W20260709 13:05:16.241786 29185 leader_election.cc:336] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.825s (SENT)
I20260709 13:05:16.502038 29144 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:16.503151 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:16.504298 29219 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:16.508581 29214 raft_consensus.cc:2410] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:16.508158 29185 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:16.303578   346 consensus_meta.cc:287] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.629s	user 0.006s	sys 0.001s
W20260709 13:05:16.424963   374 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.215s	user 0.003s	sys 0.000s
W20260709 13:05:16.411984   372 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.158s	user 0.002s	sys 0.000s
W20260709 13:05:16.515328 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 49212.2us
W20260709 13:05:16.360942   345 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.650s	user 0.001s	sys 0.008s
I20260709 13:05:16.506577 29218 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
W20260709 13:05:16.302009   337 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.781s	user 0.010s	sys 0.000s
I20260709 13:05:16.510547 29146 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:05:16.510728 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:16.511085 29064 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:16.491091   311 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:16.305202   352 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:16.522816 29147 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:16.258834   351 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:16.519994 29214 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:16.259738   339 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.274s
I20260709 13:05:16.514250 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.512023 29139 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:16.241158 32536 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:16.376112 32586 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:16.515954 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:16.505908   347 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.008s	sys 0.009s
W20260709 13:05:16.521926 29871 leader_election.cc:336] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.111s (SENT)
I20260709 13:05:16.504982 32731 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:16.525261   301 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:05:16.540128 32731 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:16.541131 32683 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:16.525298 29219 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:16.242468 32389 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.513726 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.525991 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:16.519731 29146 pending_rounds.cc:85] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 4
I20260709 13:05:16.513919   348 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:16.497107   330 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:05:16.340838   320 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.456277   326 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.532626 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:16.486291   355 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:16.518237   362 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:16.522665 32572 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:05:16.523144   352 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:16.547536 29146 pending_rounds.cc:107] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:05:16.528681 29145 raft_consensus.cc:2410] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:05:16.517416   346 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:16.550616 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 308841us
I20260709 13:05:16.544185 29147 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:16.552314 29064 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:16.532728   347 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.008s	sys 0.010s
I20260709 13:05:16.550784 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:16.598238 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:16.596925 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 82733.2us
I20260709 13:05:16.515547   366 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:05:16.567237 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:16.568387 29220 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:16.569228 29217 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:16.587401 29073 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:05:16.542474 32629 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:16.523702   337 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.530560   351 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:05:16.544430   317 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:16.592006 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.522238   345 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:16.592448 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 76401.2us
I20260709 13:05:16.595144   301 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:16.571930   348 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:16.596963 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 107389us
I20260709 13:05:16.598920 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:16.591549   352 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.611975   383 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:05:16.612884 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.544364 32586 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.615139 29148 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:16.593219 29146 raft_consensus.cc:2889] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:16.616914 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.636298 29144 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:16.640707 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.641808 29143 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:16.641937 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.643112 29143 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:16.643325 29144 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:16.644910 29143 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:05:16.645306 29143 pending_rounds.cc:85] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 5
W20260709 13:05:16.645041 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46209.4us
I20260709 13:05:16.645723 29143 pending_rounds.cc:107] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:05:16.621516 29222 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:16.607543 32629 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:16.621001   301 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:16.632601 29068 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:16.635051 29145 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:05:16.620579 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.646312 29143 raft_consensus.cc:2889] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:16.626020 29216 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:16.659018 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:16.665190 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.670198 29064 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:16.649577   352 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:16.673750 32389 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:16.678296 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 60126us
W20260709 13:05:16.678699 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.657599   301 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:16.684751 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.708182 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.717460 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.709051 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:16.718708 29064 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 14 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:05:16.718634 29221 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:16.666985   321 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:16.664665   318 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:16.725564 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.720064   388 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:16.728238   388 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:16.729043   345 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:16.710049   337 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:16.729395   388 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.730288   326 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:16.709214   371 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:16.735349 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.728257   392 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:16.736310 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.737119 29146 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:16.737965 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.739064 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.739223 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.740197 29148 raft_consensus.cc:2410] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:05:16.741241 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.741434 32586 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:16.741818 29148 raft_consensus.cc:2376] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9. Re-sending same reply.
I20260709 13:05:16.720726   348 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.768092 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:16.769025 29070 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:16.770020 29070 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:16.744637   346 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:16.748623 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.749620 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.752388 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:16.777907 29065 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:16.740289 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.753698 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.691725   328 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:16.778862 29144 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:16.788432 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.788683 29140 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:16.791453 29148 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:16.793327 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.794327 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:16.795066 29064 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:05:16.770948   362 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:16.812651 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.816671 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:16.819084 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 167163us
I20260709 13:05:16.707733 32629 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.820801 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:16.813630 29144 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:16.799388 32561 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.774677 32586 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:05:16.795739   371 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.852949 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:16.843580 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.848721 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.859549 29148 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:16.815249 32389 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.805636   320 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.851032 29146 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:16.889568 29148 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:16.787847   318 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:05:16.788168   389 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:05:16.850179 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:16.791365 32572 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:05:16.820241   360 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.860332 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.852380 32561 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:16.866564   376 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.611s	user 0.000s	sys 0.003s
I20260709 13:05:16.869652   391 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:16.877704 29907 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:16.878744 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.881116 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2124) took 1307 ms. Trace:
I20260709 13:05:16.819950   362 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.891312   303 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.858620   390 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:16.892043   317 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:16.892762 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.893253 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:16.894666 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.899096 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:16.900245 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:16.972246 29139 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:16.972738 29213 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:16.900715 29148 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:16.906737 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:16.905689 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55576.7us
I20260709 13:05:16.908150 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:16.943222 29141 raft_consensus.cc:2410] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:16.944995 29144 raft_consensus.cc:2410] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:05:16.976826 29903 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:16.948966 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.977952 29146 raft_consensus.cc:2410] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:05:16.978642   398 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:05:16.978793 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.979537 29146 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:16.971382   303 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.981155 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.982127   303 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:16.982288 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.982686   303 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:16.983283 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.983660 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.909271   366 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:16.905184 32389 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.984741 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:16.945202 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:16.985176 32561 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:16.968219   399 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.985618 32389 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:16.985760 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.955523   391 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:16.986025   399 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.986778   390 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:16.988206   326 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.987865   390 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.988719   326 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.989306 29112 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:16.990002 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:16.990016   328 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:16.990763 29908 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:16.990665   348 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:16.991756   348 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.992014 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1966) took 1047 ms. Trace:
I20260709 13:05:16.992282   348 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:16.992473 29067 rpcz_store.cc:276] 0709 13:05:15.944347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.069605 (+125258us) service_pool.cc:224] Handling call
0709 13:05:16.092993 (+ 23388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.129720 (+ 36727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.129750 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:16.138761 (+  9011us) log.cc:844] Serialized 26 byte log entry
0709 13:05:16.298470 (+159709us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b180047cd40. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1
0709 13:05:16.298674 (+   204us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:16.298723 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.298809 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:16.865874 (+567065us) spinlock_profiling.cc:243] Waited 52.9 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:16.991783 (+125909us) raft_consensus.cc:1685] finished
0709 13:05:16.991806 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.991954 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24831,"raft.queue_time_us":39516,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":17944,"spinlock_wait_cycles":148257408,"thread_start_us":24638,"threads_started":1,"wal-append.queue_time_us":45097}
I20260709 13:05:16.942461   392 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:05:16.995992   317 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.998840 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:16.999614 29140 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:17.000586 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:17.001353 29140 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:17.002511 29140 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:05:16.948119   320 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:16.910590 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:16.972306 29216 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:16.978857 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:16.980257 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:16.956066 29222 rpcz_store.cc:276] 0709 13:05:15.573696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.574041 (+   345us) service_pool.cc:224] Handling call
0709 13:05:15.652745 (+ 78704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:16.649000 (+996255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.880807 (+231807us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b5400339ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:16.881025 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2224167680}
W20260709 13:05:16.970608 29188 leader_election.cc:336] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.556s (SENT)
I20260709 13:05:16.945518   360 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:16.969549   396 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:05:16.864264   371 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:16.986382 29148 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:17.017619 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 406) took 1295 ms. Trace:
W20260709 13:05:17.059684 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.923677   301 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:17.016829 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:17.061517 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:16.955399 32629 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:17.057927 29909 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:17.030298 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.984992 29144 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:17.008430 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:17.061813 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 71362.2us
I20260709 13:05:17.072816 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.073680 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.073899 29908 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:17.013245 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.074458 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.063586 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:17.060189 29066 rpcz_store.cc:276] 0709 13:05:15.676480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.676674 (+   194us) service_pool.cc:224] Handling call
0709 13:05:15.967100 (+290426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.045587 (+ 78487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.045632 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:16.048070 (+  2438us) log.cc:844] Serialized 26 byte log entry
0709 13:05:16.152385 (+104315us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:16.212133 (+ 59748us) log.cc:844] Serialized 14 byte log entry
0709 13:05:16.214373 (+  2240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.214494 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:16.674821 (+460327us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b480027c258. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783ba36aa1 00007f783ba2db37 00007f783ba2d9ca 00007f783bb2526f 00007f783bb24f37 00007f783bb3955c 00007f783bb39508 00007f783bb39488 00007f783bb39443 00007f783bb386ec 00007f783aa62a96
0709 13:05:16.733104 (+ 58283us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b1800421480. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1
0709 13:05:16.734815 (+  1711us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b1800421480. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:16.902223 (+167408us) raft_consensus.cc:1685] finished
0709 13:05:16.902250 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.971199 (+ 68949us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b5400187c50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:16.971309 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":188667,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":45416,"raft.run_cpu_time_us":5252,"raft.run_wall_time_us":683309,"spinlock_wait_cycles":775906688,"thread_start_us":185373,"threads_started":3,"wal-append.queue_time_us":148836}
I20260709 13:05:17.080116 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.078289 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:17.081100 29910 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:17.075444 29139 raft_consensus.cc:2376] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12. Re-sending same reply.
W20260709 13:05:17.095894 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 125317us
I20260709 13:05:17.002187 29903 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:17.096997 29907 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:05:17.101490 29145 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:17.061372   366 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:16.904747 32731 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.129997 29222 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:05:17.256316 29066 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:05:17.409610 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 502944us
I20260709 13:05:17.324982 29146 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:17.410102 29219 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:17.220132   350 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.725306 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.725565   350 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.727126 29216 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:17.452193   402 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:05:17.305202 29910 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:17.298671 29909 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
W20260709 13:05:17.462318 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:17.740186 29911 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.003s	sys 0.010s
W20260709 13:05:17.741112 29911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.003s	sys 0.010s
W20260709 13:05:17.212792 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.742326 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:17.742980 29144 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
W20260709 13:05:17.743149 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:17.743865 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.744120 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:17.745332 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.747013   360 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:17.723484 29875 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.040s (SENT) [suppressed 64 similar messages]
I20260709 13:05:17.749449 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:17.750994 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:17.751538 29144 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
W20260709 13:05:17.751762 29188 leader_election.cc:336] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.547s (SENT)
I20260709 13:05:17.265275 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:17.550644   301 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:17.755784 29905 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:17.749581 32287 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.754133   407 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:17.757642 32287 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.757895   407 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.266047   364 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:17.756026 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:17.761653 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:17.265287   326 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:17.762746 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:17.763012 29906 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:17.766861 29038 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:17.769824   348 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.777s	user 0.009s	sys 0.000s
I20260709 13:05:17.770298   348 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.764214 29064 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:17.771786 32287 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:17.776144   406 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.256119   399 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:17.515865   320 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:17.764209 29071 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:05:17.765383 29065 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:17.778496 29071 pending_rounds.cc:85] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 4
I20260709 13:05:17.778784 29071 pending_rounds.cc:107] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:05:17.303275   371 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:17.319343   317 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:17.763688   364 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.779001 29071 raft_consensus.cc:2889] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:17.303574   391 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.782182   364 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:17.778972   409 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:05:17.781476 29064 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:17.100940   321 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.665s
I20260709 13:05:17.796221 29038 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:17.796679 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44965.8us
I20260709 13:05:17.797855 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:17.799218 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:17.803131 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 1960) took 1073 ms. Trace:
I20260709 13:05:17.803568 29145 rpcz_store.cc:276] 0709 13:05:16.724286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.724463 (+   177us) service_pool.cc:224] Handling call
0709 13:05:17.797339 (+1072876us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54003b4054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:17.797475 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831372160}
I20260709 13:05:17.796118   320 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:17.750547   366 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.795604   321 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:17.802469   371 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:17.804240   317 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:17.804139   360 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:05:17.819487 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 96031.1us
I20260709 13:05:17.830917 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:17.831228 29214 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:17.832239   412 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:17.832021   409 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:17.832823   412 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:17.833292   405 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:17.833750   412 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.834048   405 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.808157   320 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.848466 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2143) took 1073 ms. Trace:
I20260709 13:05:17.849694 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:17.852507 29146 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:17.846086 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 1981) took 1604 ms. Trace:
I20260709 13:05:17.853367 29073 rpcz_store.cc:276] 0709 13:05:16.242131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.295991 (+ 53860us) service_pool.cc:224] Handling call
0709 13:05:16.348473 (+ 52482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.601919 (+253446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.845781 (+1243862us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54002aba54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:17.846015 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548618496}
I20260709 13:05:17.851902 29221 rpcz_store.cc:276] 0709 13:05:16.775214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.775375 (+   161us) service_pool.cc:224] Handling call
0709 13:05:16.792773 (+ 17398us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:05:16.799378 (+  6605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.799424 (+    46us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:05:16.843866 (+ 44442us) log.cc:844] Serialized 104 byte log entry
0709 13:05:17.143710 (+299844us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:17.250714 (+107004us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.251083 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.728986 (+477903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.729057 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:17.848155 (+119098us) raft_consensus.cc:1685] finished
0709 13:05:17.848175 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.848344 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2625,"prepare.queue_time_us":124585,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"raft.queue_time_us":486208,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":13375,"spinlock_wait_cycles":23310592,"thread_start_us":678017,"threads_started":2,"wal-append.queue_time_us":866164}
I20260709 13:05:17.854753 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:17.855420 29146 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:17.813072   321 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:17.788079   406 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:17.844787 32629 consensus_meta.cc:287] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.744s	user 0.011s	sys 0.000s
W20260709 13:05:17.859385 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 61219.5us
W20260709 13:05:17.859697 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 60555.4us
I20260709 13:05:17.862370 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:17.863397 29145 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:17.857805 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:17.817001   317 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.843705 32287 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:17.859243 32572 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.818771   360 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:17.867837 29188 leader_election.cc:336] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.641s (SENT)
I20260709 13:05:17.843822 32731 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:17.845503   405 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:17.846606   410 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.856613   320 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:17.887342 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:17.876490   350 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:17.877152 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:17.878623 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:17.869831 29213 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:17.885980 29145 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:17.889389 29146 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:17.869135   317 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:17.893368   366 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:17.880304   360 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:17.896236 32287 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:17.897637 32287 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:17.880196 32572 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.900271   410 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:17.902093 32572 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:17.902442   410 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:17.903460 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:17.904683   406 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:17.905194 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:17.905992 29064 raft_consensus.cc:2376] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12. Re-sending same reply.
I20260709 13:05:17.906301   321 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.907217 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:17.907811   326 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:17.908450   326 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:17.909472 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:17.909606   326 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:17.910328 29066 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:17.911379   409 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:05:17.912029 29185 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.895s (SENT)
I20260709 13:05:17.912101   409 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:17.912819   371 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:17.913266 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.913690   337 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:17.914078   337 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:17.914909   337 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:17.911820 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:17.916002   412 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:17.916575 29070 raft_consensus.cc:2376] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 6. Re-sending same reply.
I20260709 13:05:17.916839   412 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:17.917238   412 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:17.908367 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:17.918249 29071 raft_consensus.cc:2376] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11. Re-sending same reply.
W20260709 13:05:17.922075 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59787.2us
I20260709 13:05:17.923148 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:17.918089   412 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:17.913307   371 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:17.925804   360 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:17.926445   412 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:17.927161 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.927743 29219 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:17.928088 32731 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:17.928244   371 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:17.928550 32731 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.928719 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.928905 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.928972 32731 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:17.929369 29219 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:17.930079 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.930254 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.931298 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.932008 29222 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:17.932379 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.933188 29215 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:17.938016   362 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:17.938628   362 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:17.938980 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.939606 32629 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:17.940225 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:17.940696 32629 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:17.940925 29216 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:17.941658 32629 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:17.939764 29221 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:17.940090   371 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:17.944063   371 consensus_queue.cc:1243] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:17.943053 32629 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:17.944417   317 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:17.945868 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:17.946451 29908 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:17.946940   317 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:17.948341   371 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:17.949651 29069 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:17.949705   371 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:17.950234   371 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:17.950181 32641 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:17.951467   360 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:17.951303   371 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:17.952267 29188 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:17.953444 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:17.960986 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:17.962054 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:17.963229 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:17.970037   362 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:17.998379 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 77322.7us
I20260709 13:05:18.000548 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:18.008634 29066 raft_consensus.cc:2410] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:05:18.012930 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 155249us
I20260709 13:05:18.014309 29908 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:18.018308 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:18.024765 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 101744us
I20260709 13:05:18.025672 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.026731 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.028668 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:17.964413 32641 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:18.031883 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.001477   371 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:18.001995 32354 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:18.034628 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:18.044942 29905 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:18.040467 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.035318 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:18.048408 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.051666 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.012859   362 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:18.056656 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:18.031105   350 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:18.035141   406 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:18.031354   410 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.028635   415 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:18.061936   390 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:18.061839   426 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:18.063973   337 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:18.061856   407 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:18.031605   422 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:18.075270   406 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.041931   352 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:18.082393 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 64540.2us
W20260709 13:05:18.083087 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.080435   415 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:18.109736   317 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.110375   317 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:18.112812   317 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:18.115671 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:18.116371 29217 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:18.125082 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:18.125967 29221 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:18.126348 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:18.128067 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:18.128822   317 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:18.129190   317 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.129544   317 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:18.131245 32629 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:18.096578 32354 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:18.140633 32354 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.142930 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:18.099314   406 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:18.136503 29221 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:18.138592 29140 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:18.099201   350 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.155200 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:18.155974   421 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.086350   320 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.158476 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:18.081353   429 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:05:18.159267 29065 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:18.176177 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:18.096271   432 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:18.146461 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2231) took 1253 ms. Trace:
I20260709 13:05:18.217128 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:18.148445   317 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.151077 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:18.148520   348 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:18.152885 29066 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:18.153105   362 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.153898 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:18.236049 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 175733us
I20260709 13:05:18.236835 29064 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 6: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:18.099846   352 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:18.085913   431 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:18.214758 29140 rpcz_store.cc:276] 0709 13:05:16.893240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.004972 (+111732us) service_pool.cc:224] Handling call
0709 13:05:18.138102 (+1133130us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b5800003098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:05:18.138350 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.146213 (+  7863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.146402 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6706048}
I20260709 13:05:18.143904 29217 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:18.235929 29904 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:05:18.242313 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 190762us
I20260709 13:05:18.108660   415 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:18.135747   428 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.270418 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:18.280601 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:18.297703 29071 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:18.278455 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:18.302189 29904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 640) took 1225 ms (client timeout 1545 ms). Trace:
I20260709 13:05:18.302490 29068 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:18.297235 29186 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:18.304986 29069 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:18.159657   429 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:18.308849 32287 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:18.303216 29904 rpcz_store.cc:269] 0709 13:05:17.077282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.150689 (+1073407us) service_pool.cc:224] Handling call
0709 13:05:18.302131 (+151442us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:18.301465 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 158602us
I20260709 13:05:18.214779   350 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.312263 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.248430   421 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:18.314136 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.312430 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.297925 32572 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.300719   348 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:18.298453   352 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:18.280459   318 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:18.321674 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.316068   350 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:18.325278   421 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:18.365628 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:18.320683   430 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:18.215153 32629 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:05:18.301306   320 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:18.315352 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 76882.7us
I20260709 13:05:18.328626   330 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:18.336545 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 238189us
I20260709 13:05:18.235944   407 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:05:18.365198 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:18.396220 29904 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 6.
W20260709 13:05:18.396968 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 99837.4us
I20260709 13:05:18.397367 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:18.398391 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.330816 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:18.368505 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:18.400516 29148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2241) took 1382 ms (client timeout 1813 ms). Trace:
I20260709 13:05:18.369457 32645 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:18.369573   327 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:18.370703 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:18.455610 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 84985.8us
I20260709 13:05:18.456965 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:18.458029 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:18.371291   435 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:18.374531 29068 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:18.371737   410 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:05:18.099684   422 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:18.479225   452 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:18.214278   364 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:18.392702 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.318445   326 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:05:18.487354 29068 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:18.487888 29068 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:18.392946   366 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:18.310320   432 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:05:18.337406 32572 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:18.305008   428 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.354555   348 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:18.461417 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 130663us
W20260709 13:05:18.453953 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 137462us
I20260709 13:05:18.464087 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:18.516072 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:18.516430 29871 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.472318 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:18.476923   431 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:18.477262   447 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:18.478026 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:18.523869 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:18.478328 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.524497 29073 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:18.314339   429 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.373478   430 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:18.401444   413 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:18.400964 29148 rpcz_store.cc:269] 0709 13:05:17.018093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.063987 (+ 45894us) service_pool.cc:224] Handling call
0709 13:05:18.367719 (+1303732us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b5800003098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:05:18.400447 (+ 32728us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14992640}
I20260709 13:05:18.541970 29146 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:18.542593 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:18.504515   434 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:18.509683 29070 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:05:18.517220   432 consensus_queue.cc:1243] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:05:18.513477 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 201337us
I20260709 13:05:18.548561 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.541307 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:18.523591   431 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.469223   449 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:18.529331   366 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:18.511739 29068 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.522811 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.480660   452 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:18.530472 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:18.527561 29218 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:18.510077 32287 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:18.505610 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:18.454768 32645 consensus_queue.cc:1243] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:18.546893 29907 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:18.366508   318 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:18.550081 29871 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.516952   435 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:18.551718 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:18.552341 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:18.568385 29140 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:18.568668 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.517669 29065 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:05:18.560421 29185 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:18.552026 32731 consensus_meta.cc:287] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.622s	user 0.013s	sys 0.000s
I20260709 13:05:18.568408 29073 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:18.564568 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:18.540792   422 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:18.569556 29065 pending_rounds.cc:85] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 4
I20260709 13:05:18.538910   417 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:05:18.573657 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:18.575687 29065 pending_rounds.cc:107] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:05:18.554903   431 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:18.577390 32731 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:18.556126   434 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:18.581458   318 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:18.581336 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.583498 29065 raft_consensus.cc:2889] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:18.555907   449 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.586020 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:18.601655 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:18.584126   371 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.000s
I20260709 13:05:18.568228   413 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:18.587220   422 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.589319 32572 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:18.589082   435 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.590350 32731 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:18.594143 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52927.6us
I20260709 13:05:18.575815   317 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:18.602555 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:18.584635   407 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.585214   366 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.608248 29907 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:05:18.608637   360 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:18.608757 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:18.591806   320 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:18.603401   449 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:18.608062   337 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:18.624536   433 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.624910   454 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:18.640314   434 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.625056   369 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:18.642064   369 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:18.625484   443 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:05:18.644296 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:18.632495   453 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:18.647948 29907 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:18.649804 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:18.591425   348 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.626505   440 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:05:18.627144   450 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:18.627374   317 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:18.627635   350 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:18.628592   364 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:18.616142   371 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:18.623607 32641 consensus_meta.cc:287] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.583s	user 0.005s	sys 0.006s
W20260709 13:05:18.631371 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 669552us
I20260709 13:05:18.632635   428 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:18.633126   439 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:05:18.627177   446 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:18.640154   433 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.643589   321 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:18.721336 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 118847us
I20260709 13:05:18.643121   369 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:18.665741 29217 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:18.621407   360 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.622255   320 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.609521   422 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.710772 29188 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.662137   450 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:18.630576   412 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.704s	user 0.007s	sys 0.006s
I20260709 13:05:18.662652   317 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.722733 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:18.742439 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 138307us
I20260709 13:05:18.748189 29038 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:18.750881 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:18.749644 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:18.752481 29065 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:18.752624 29070 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:18.754117 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:18.750233 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:18.755842 29073 raft_consensus.cc:2376] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10. Re-sending same reply.
I20260709 13:05:18.757292 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:18.757822 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.757871 29216 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:18.758699 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.712587 32641 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.601236   430 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:18.768644 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:18.739853   360 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:18.769531 29215 raft_consensus.cc:2410] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:18.744870   320 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:18.803635 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 92932.2us
I20260709 13:05:18.802243 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:18.721593   446 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.634891 32287 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:18.749202   450 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:18.837514 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:18.822531 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 74531.9us
I20260709 13:05:18.870163 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:18.870689 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:18.821133 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 575) took 1825 ms. Trace:
W20260709 13:05:18.871417 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 69248.8us
I20260709 13:05:18.769814   407 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:18.848327 29148 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:05:18.873265 29148 pending_rounds.cc:85] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 8
I20260709 13:05:18.873528 29148 pending_rounds.cc:107] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:05:18.873751 29148 raft_consensus.cc:2889] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:18.822070 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:18.654102   452 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:18.769474   429 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:18.859577 29141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 1945) took 2682 ms (client timeout 2564 ms). Trace:
W20260709 13:05:18.877460 29141 rpcz_store.cc:269] 0709 13:05:16.164958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.501660 (+336702us) service_pool.cc:224] Handling call
0709 13:05:18.847376 (+2345716us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b54002e8954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:18.847572 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":915606400}
W20260709 13:05:18.878754 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56821.7us
I20260709 13:05:18.760682   317 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:18.803062   427 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.132s
I20260709 13:05:18.881268   364 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:18.882314   448 raft_consensus.cc:2764] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:05:18.876524 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:18.767482   428 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:18.755451   371 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 13:05:18.804152 32645 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.879566 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.888870 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:18.906275 29146 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:18.743804 29140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79cdfc6a725145348b70034422ef477b"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42040
I20260709 13:05:18.907755 29140 raft_consensus.cc:606] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Received request to transfer leadership
I20260709 13:05:18.834659   468 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.889770 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:18.907823 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:18.871256 29902 rpcz_store.cc:276] 0709 13:05:16.974979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.975100 (+   121us) service_pool.cc:224] Handling call
0709 13:05:16.978440 (+  3340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:17.082854 (+104414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:17.082885 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.158988 (+1076103us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b2c00012928. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:05:18.159433 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.524131 (+364698us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:18.524166 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.524224 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:18.773870 (+249646us) raft_consensus.cc:1685] finished
0709 13:05:18.773906 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:18.800562 (+ 26656us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34292,"raft.queue_time_us":36529,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":54,"spinlock_wait_cycles":25072896,"thread_start_us":432477,"threads_started":2,"wal-append.queue_time_us":424692}
I20260709 13:05:18.910676 29146 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:18.872349   446 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:18.914793 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:18.914160 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:18.915591 29146 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:18.841037   352 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.917606 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:18.912580 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 65610.6us
I20260709 13:05:18.916438 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:18.921940 29072 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 7: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:18.924836 29039 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:18.925622 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:18.894644   407 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:18.933673 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 54187.2us
I20260709 13:05:18.926295 29072 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
W20260709 13:05:18.942826 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 111579us
W20260709 13:05:18.946823   460 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 13:05:18.934697   412 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.950125   433 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:18.934417 32629 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:18.952494 32629 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:18.953569 32629 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:18.954886   415 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.650s	user 0.009s	sys 0.001s
I20260709 13:05:18.955358   415 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:18.957783 29217 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:18.958349 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:18.959127 29072 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:18.961201 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:18.961560 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:18.962056 29216 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
W20260709 13:05:18.935376 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 57872.8us
I20260709 13:05:18.970778 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:18.969547 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:18.977219   434 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:18.978083   434 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:18.980511   429 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:18.980989   429 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:18.981402   429 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:18.986393 29902 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:18.998335 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:18.999403 29142 raft_consensus.cc:2410] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:18.912106 32645 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.000751 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:19.002259 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:19.004264   453 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:19.005810   453 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:19.006206   453 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:18.974267   412 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:19.007855   464 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:05:19.013383 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 576) took 2038 ms. Trace:
I20260709 13:05:19.013877 29906 rpcz_store.cc:276] 0709 13:05:16.975251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.975367 (+   116us) service_pool.cc:224] Handling call
0709 13:05:16.989648 (+ 14281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:17.766360 (+776712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.013096 (+1246736us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b5400023d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:19.013314 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778107136}
I20260709 13:05:19.014701   452 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:19.017639 32354 raft_consensus.cc:3055] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:19.018137 32354 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:18.957587   415 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:19.020426   407 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.019153 32354 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.022598 32354 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:19.023962 29147 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
W20260709 13:05:19.024523 29213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2144) took 2076 ms (client timeout 1545 ms). Trace:
W20260709 13:05:19.024609 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 87900.8us
W20260709 13:05:19.025012 29213 rpcz_store.cc:269] 0709 13:05:16.947724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.013718 (+ 65994us) service_pool.cc:224] Handling call
0709 13:05:19.024272 (+2010554us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b5400132ed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:19.024419 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1852003200}
I20260709 13:05:19.025944   352 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:05:19.029903   474 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:19.030630   474 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:19.036888 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:19.009862   465 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:19.037802 29139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 510) took 1964 ms (client timeout 1843 ms). Trace:
W20260709 13:05:19.038230 29139 rpcz_store.cc:269] 0709 13:05:17.073743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.095777 (+ 22034us) service_pool.cc:224] Handling call
0709 13:05:19.036702 (+1940925us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b5800003098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:05:19.037726 (+  1024us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9474944}
I20260709 13:05:19.040832   440 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:19.041348   440 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:19.041702   440 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:19.042827 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:19.044551 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2207) took 2178 ms. Trace:
I20260709 13:05:19.045084 29218 rpcz_store.cc:276] 0709 13:05:16.865640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.179959 (+314319us) service_pool.cc:224] Handling call
0709 13:05:17.729850 (+549891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.562011 (+832161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.044230 (+482219us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b540033b3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:19.044484 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1834144000}
I20260709 13:05:19.047580 32645 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:19.048900 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:19.049546 29069 raft_consensus.cc:2376] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7. Re-sending same reply.
I20260709 13:05:19.043560 29071 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
W20260709 13:05:19.052700 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63294.9us
I20260709 13:05:19.053665 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:19.069540 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 67412.7us
I20260709 13:05:19.071441 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:19.072917 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:19.074513 29217 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:19.075138 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:19.075523 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:19.077491 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:19.079442 29146 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:19.083504 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:19.083598 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:19.091313 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:19.096429 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:19.097451 29069 raft_consensus.cc:2368] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
W20260709 13:05:19.086031   406 consensus_meta.cc:287] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.932s	user 0.013s	sys 0.000s
I20260709 13:05:19.073629   468 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:19.102397 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48995.4us
I20260709 13:05:19.086447   427 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:19.085234   352 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:19.105785 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:19.085083   371 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:19.093870   388 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:19.088126 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:19.109176 29185 leader_election.cc:400] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:19.125861 29185 leader_election.cc:403] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:05:19.094599   364 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:19.089988   389 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:19.127297 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:19.079025   448 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:05:19.138027 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:19.108824   413 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:19.031649   474 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.095610   449 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.117309   352 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:19.161159   486 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:19.116361   468 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.288658 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2103) took 2061 ms. Trace:
I20260709 13:05:19.258358   443 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:19.158747   466 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:19.157527   472 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:05:19.130587   388 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:19.132099   464 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:19.220779   446 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.222314 29221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:05:19.230444 29037 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:19.298226 29221 raft_consensus.cc:606] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:19.147235   434 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:19.114419   406 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:19.240882   478 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.003s
I20260709 13:05:19.244728   329 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:19.213584   371 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:19.298504 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 68184.4us
I20260709 13:05:19.256618 29072 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:19.300736   371 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.301699 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:19.193001   364 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
W20260709 13:05:19.266880 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 139677us
I20260709 13:05:19.214107   482 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:19.283241 29071 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:19.116425   450 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.289345 29067 rpcz_store.cc:276] 0709 13:05:17.226781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.226971 (+   190us) service_pool.cc:224] Handling call
0709 13:05:18.797297 (+1570326us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b5800005198. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8eee 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:05:18.797485 (+   188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.797728 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:18.798018 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 13:05:19.288219 (+490201us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:19.288286 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:19.288348 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.288448 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.288593 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18894,"prepare.queue_time_us":3289,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":184865,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":13307,"spinlock_wait_cycles":10431872,"thread_start_us":472218,"threads_started":2,"wal-append.queue_time_us":210552}
W20260709 13:05:19.329000 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.292469 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 184651us
W20260709 13:05:19.285135   431 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.690s	user 0.005s	sys 0.008s
I20260709 13:05:19.268275 32731 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:19.296700 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 191150us
I20260709 13:05:19.298612 29221 raft_consensus.cc:614] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:19.300740 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:19.084674   428 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:19.286684   352 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:19.304579 29037 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:19.320202 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:19.330374 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.361965 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 57505.8us
I20260709 13:05:19.335296   431 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.348742   428 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.350670   406 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.351667   430 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:19.366266 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46150us
I20260709 13:05:19.355036 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:19.367112 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:19.344063   479 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:19.359077 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.360074 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:19.296669   464 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.310578   486 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:19.296833   388 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:19.335345 29905 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:19.364676 29185 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:19.374714   448 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:19.376387   412 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:19.377241   465 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:19.377674   429 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.395500 29216 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:19.382925 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:19.383414 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:19.365928   421 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:19.392930 29219 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:19.396971 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:19.397291 29222 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:19.398097 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:19.379201   360 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.543s	user 0.004s	sys 0.007s
I20260709 13:05:19.402210 29222 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:19.402606 29218 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:19.407495 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:19.406562 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:19.407198 29216 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:19.410136 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:19.380102   392 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:19.392268 32731 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:19.400936   320 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.000s
I20260709 13:05:19.391570   464 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:19.391193   486 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.394811   390 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:19.416535 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:19.477458 29146 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:19.414556 29113 connection.cc:256] RPC call timeout handler was delayed by -1.2169960569999603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
I20260709 13:05:19.362247   317 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.418259 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:19.411268   389 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.421118   490 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:19.402213   465 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:19.421980 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:19.422758 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:19.452028   468 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:19.310802   443 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:19.473425 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.478785 29113 connection.cc:256] RPC call timeout handler was delayed by -1.1831709600000977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
I20260709 13:05:19.411720   369 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.495663 29146 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:19.508404 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 511) took 2434 ms. Trace:
I20260709 13:05:19.476924   486 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:19.524348 29144 rpcz_store.cc:276] 0709 13:05:17.073990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.752102 (+678112us) service_pool.cc:224] Handling call
0709 13:05:17.752442 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:17.779685 (+ 27243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:17.779732 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:17.784561 (+  4829us) log.cc:844] Serialized 26 byte log entry
0709 13:05:17.855460 (+ 70899us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:17.855513 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.479365 (+1623852us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b5400190354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:19.479454 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.479519 (+    65us) raft_consensus.cc:1685] finished
0709 13:05:19.479537 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.508320 (+ 28783us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13307,"raft.queue_time_us":14864,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":141,"spinlock_wait_cycles":8646912,"thread_start_us":70406,"threads_started":1,"wal-append.queue_time_us":204462,"wal-append.run_cpu_time_us":1864,"wal-append.run_wall_time_us":1005282}
I20260709 13:05:19.414086   346 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.413980   417 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.335155   482 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.416754   350 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.416158   392 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:19.416488   432 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.413789 32645 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 13:05:19.484088 32731 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:19.413966   360 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.555025   431 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.547572   346 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.559293 29213 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:19.567174 29070 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:19.573302 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 213279us
I20260709 13:05:19.480381   320 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.571419 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:19.580334 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:19.588999 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:19.566327   392 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:19.519404   440 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.572458   428 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:19.598721 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:19.594141   431 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:19.643853 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45218.6us
I20260709 13:05:19.644181   431 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.644668   431 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:19.644747 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:19.600625   495 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:19.646004   495 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:19.646415   495 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:19.601774   500 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:05:19.649125 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1980) took 1191 ms. Trace:
I20260709 13:05:19.608566 32731 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.649785 29148 rpcz_store.cc:276] 0709 13:05:18.457813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.542227 (+ 84414us) service_pool.cc:224] Handling call
0709 13:05:18.847821 (+305594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.894155 (+ 46334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.648830 (+754675us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54002e8950. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:19.648985 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674961664}
I20260709 13:05:19.651741   440 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.550057 32357 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:05:19.580626   432 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.609807   346 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:19.611361   443 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.613193   360 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.615038   320 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.568320   351 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:05:19.567866   417 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.565714   350 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.616074   429 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:19.617359   317 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:19.620695   453 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.003s
I20260709 13:05:19.685606   453 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.686573   486 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.688122   453 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.588912   449 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:19.689095   469 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.946s	user 0.000s	sys 0.004s
I20260709 13:05:19.622749 29143 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:19.598069   472 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.689944   472 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.690469 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 644) took 1080 ms. Trace:
I20260709 13:05:19.690850 29908 rpcz_store.cc:276] 0709 13:05:18.609954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.632192 (+ 22238us) service_pool.cc:224] Handling call
0709 13:05:18.632575 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.633064 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.633186 (+   122us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.633696 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.727677 (+ 93981us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:18.727736 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.727830 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.688353 (+960523us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:19.690177 (+  1824us) raft_consensus.cc:1685] finished
0709 13:05:19.690204 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.690409 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":788,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":48,"spinlock_wait_cycles":55186304,"thread_start_us":93553,"threads_started":1,"wal-append.queue_time_us":109198}
I20260709 13:05:19.692615   301 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.692713   472 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.693260   301 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.624050   327 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:19.695379 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:19.705089   447 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:19.705554   447 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:19.706020   431 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.706576   432 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:19.709812 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 641) took 2632 ms. Trace:
I20260709 13:05:19.710253 29903 rpcz_store.cc:276] 0709 13:05:17.077658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.097938 (+ 20280us) service_pool.cc:224] Handling call
0709 13:05:17.098603 (+   665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:17.316296 (+217693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:17.316327 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:19.184507 (+1868180us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b4c004d3800. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7834dd9a20 00007f7834deaa29 00007f7834db0644 00007f7834dd7f2d 00007f7834dd7b93 00007f7836b0818b 00007f7836b08cb5 00007f7836b0a004 00007f7836b09729 00007f7836b0ed96 00007f7836b03adb
0709 13:05:19.242758 (+ 58251us) log.cc:844] Serialized 564 byte log entry
0709 13:05:19.384973 (+142215us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:19.385004 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.385063 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.709453 (+324390us) raft_consensus.cc:1685] finished
0709 13:05:19.709471 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.709724 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33714432,"thread_start_us":118410,"threads_started":1,"wal-append.queue_time_us":451922}
I20260709 13:05:19.712581   492 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:19.713044   492 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:19.713980   492 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.715358   492 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:19.670755   504 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:19.626679   422 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.629405 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:19.631686   482 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:19.632477   490 raft_consensus.cc:2764] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:05:19.635511 29072 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:19.640427   427 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.517s	user 0.005s	sys 0.006s
W20260709 13:05:19.641455 29142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 506) took 3477 ms (client timeout 2040 ms). Trace:
W20260709 13:05:19.641882 32354 consensus_meta.cc:287] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.619s	user 0.014s	sys 0.005s
I20260709 13:05:19.510476   390 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.537801   369 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.595906   421 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:05:19.585263   485 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:19.643882 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 536813us
I20260709 13:05:19.517100   389 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.654443   498 raft_consensus.cc:2764] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:05:19.656044 29070 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:19.620850   464 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.680449   392 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.682152   417 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.683730   352 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.684604   350 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:19.720577 29069 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:19.724268 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:19.822443 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 98246.3us
I20260709 13:05:19.807269   406 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:19.733781 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.765555 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 347381us
W20260709 13:05:19.763957 29142 rpcz_store.cc:269] 0709 13:05:16.164555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.466799 (+302244us) service_pool.cc:224] Handling call
0709 13:05:18.644128 (+2177329us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b5800003098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:05:19.641179 (+997051us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b5400192154. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:19.641354 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":797256448}
W20260709 13:05:19.805745 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 74591.4us
I20260709 13:05:19.813431 32572 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:19.808722 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.813962 29072 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:19.720674   482 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:19.691671 29143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 1958) took 3027 ms (client timeout 1571 ms). Trace:
W20260709 13:05:19.841236 29143 rpcz_store.cc:269] 0709 13:05:16.664732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.667363 (+  2631us) service_pool.cc:224] Handling call
0709 13:05:19.691473 (+3024110us) spinlock_profiling.cc:243] Waited 2.91 s on lock 0x7b540029f254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:19.691585 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6686806784}
I20260709 13:05:19.806895   427 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:19.824331 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 179665us
W20260709 13:05:19.804525 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 402894us
I20260709 13:05:19.845463 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:19.815732   448 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.833132 29903 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:05:19.834707 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:19.821408   453 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:19.819221   435 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.939s	user 0.013s	sys 0.025s
I20260709 13:05:19.840179 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:19.844151 29064 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:05:19.847855 29148 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:05:19.836491 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:19.851884 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:19.706501   447 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.857966 29904 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:19.830653   490 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:19.854743 29908 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:19.813773   388 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.867864 29072 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:05:19.869359 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 276573us
I20260709 13:05:19.874210 29112 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:19.873018 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:19.876240 29072 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:19.879112 29072 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:05:19.880436 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 145110us
I20260709 13:05:19.888697 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:19.855906   435 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.013s	sys 0.025s
I20260709 13:05:19.902603 29148 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79cdfc6a725145348b70034422ef477b"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42040
I20260709 13:05:19.903287 29148 raft_consensus.cc:606] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Received request to transfer leadership
I20260709 13:05:19.905190 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:19.908185 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62217.7us
I20260709 13:05:19.909108 29069 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:19.910522 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:19.912887   510 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:19.844856   498 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:05:19.874673   427 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:19.917786 29909 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:19.917537 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:19.843727 32572 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.891484   447 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:19.848134   482 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.855764   448 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:19.924813 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:19.923496 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:19.885777   431 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.944005 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:19.859792   453 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:19.916019   346 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:19.946219 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:19.883930   317 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:05:19.920989   435 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:19.954893 29068 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:19.958158 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:19.956707 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:19.925665   490 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:19.923640   486 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:19.894327 32354 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:05:19.959764   428 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:19.953334   301 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:19.928901   350 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:19.951443   507 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:05:19.981419 29904 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:05:19.919062   351 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:05:19.985409 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:19.951597   467 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.921196   472 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:05:19.990034 29906 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:19.927340   300 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:05:19.946058   427 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:19.966223   498 consensus_queue.cc:1243] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:05:19.956044 32645 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:19.927977 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:20.045056 29069 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:20.168215 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:20.170066 29069 raft_consensus.cc:1217] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 5.5->[7.6-13.9]   Dedup: 9.7->[11.8-13.9]
I20260709 13:05:20.170758 29069 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: [7.6-13.9]
W20260709 13:05:20.045522 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 122082us
I20260709 13:05:19.947180   465 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.063145 29073 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:20.065819 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.001873   503 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 13:05:20.006775   430 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.626s	user 0.012s	sys 0.000s
W20260709 13:05:20.069185 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 125271us
I20260709 13:05:20.051245   346 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.971652   437 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.239s
W20260709 13:05:20.097836 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:20.104862 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.120867 29068 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:20.062487   504 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:19.991837   519 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:05:20.078042   428 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:20.061537 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:20.177454 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:20.206976 29140 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:20.115031   352 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:20.184095 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2005) took 1078 ms. Trace:
I20260709 13:05:20.200145 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2118) took 1714 ms. Trace:
I20260709 13:05:20.174629 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:20.174290 29069 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:05:20.215176 29070 rpcz_store.cc:276] 0709 13:05:19.106475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.300487 (+194012us) service_pool.cc:224] Handling call
0709 13:05:20.183723 (+883236us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54003bbfd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:20.184018 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533270912}
I20260709 13:05:20.215310 29068 rpcz_store.cc:276] 0709 13:05:18.485699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.486822 (+  1123us) service_pool.cc:224] Handling call
0709 13:05:18.487207 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.954538 (+1467331us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b540010caa0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a82a1d 00007f7836a691a4 00007f7836aeb447 00007f7836b03fe7 00007f7836b057de 00007f7836b0fb49 00007f7836b10416 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419
0709 13:05:20.199769 (+245231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.200069 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189270400}
I20260709 13:05:20.218014 29066 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:20.179493   517 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
W20260709 13:05:20.181869   511 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
W20260709 13:05:20.225284 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 52904.5us
I20260709 13:05:20.204941   346 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:20.225901 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 298011us
I20260709 13:05:20.235630   390 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:20.236025   328 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:20.068344   495 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:20.238200   495 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:20.239507 29066 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:20.205505   337 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:20.241109   514 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:05:20.241427   337 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:20.225647 29217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2223) took 1472 ms (client timeout 1500 ms). Trace:
I20260709 13:05:20.228251 29070 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:20.242865 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2004) took 1147 ms. Trace:
I20260709 13:05:20.180054   431 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:20.185019 32645 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:20.180634   448 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:20.168017   464 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:20.181130   453 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:20.153385   467 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:20.174871   421 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:05:20.243551   337 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:20.235777   329 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
W20260709 13:05:20.324175 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 68004.6us
I20260709 13:05:20.270849 29073 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:05:20.245877 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:20.255473 29070 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:20.243317 29064 rpcz_store.cc:276] 0709 13:05:19.095462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.634269 (+538807us) service_pool.cc:224] Handling call
0709 13:05:19.634684 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.856565 (+221881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.242633 (+386068us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b540031e7d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:20.242807 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480164096}
I20260709 13:05:20.327490 29071 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:20.242678 29217 rpcz_store.cc:269] 0709 13:05:18.753637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.753826 (+   189us) service_pool.cc:224] Handling call
0709 13:05:20.225498 (+1471672us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54001329d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:20.225590 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466193792}
I20260709 13:05:20.329857   328 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:20.294552   512 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.330987   512 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:20.331959   520 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:05:20.332358   346 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.333406   512 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:20.333859 32572 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:20.334471   346 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:20.176992   482 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:20.268076   435 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:05:20.335798 29910 raft_consensus.cc:2955] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 5 to 8, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:20.267676   432 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:05:20.323809   431 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.338537   432 consensus_queue.cc:1243] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:20.338640   431 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:20.340766 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:20.342142   531 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:05:20.327032 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 81266.9us
I20260709 13:05:20.343690 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:20.345234   346 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:20.345988   346 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.344337   531 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
W20260709 13:05:20.268210   513 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:05:20.346668   346 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:20.214828   519 consensus_queue.cc:1243] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:20.357216 29140 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:20.359186 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:20.359997 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:20.213927   369 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:20.366708 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:20.380383 29064 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:20.350294 32645 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:20.362625   472 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:20.372917   453 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:20.372112   432 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:20.375867   448 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:20.365935   329 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 13:05:20.388175   492 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.672s	user 0.009s	sys 0.008s
I20260709 13:05:20.388988   328 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:20.388516   447 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:20.273526   427 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:20.404269 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 150166us
I20260709 13:05:20.318648   490 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:20.390851   472 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:20.390255   300 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:05:20.391701   351 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:05:20.386785   512 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:05:20.401870   453 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:20.429432 29064 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:20.435461 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:20.402035   448 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:20.416692   467 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:20.413672   328 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:20.441375 29144 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:05:20.442710 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:20.456822 29146 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:20.443195 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:20.443378   351 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:20.444386 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:20.446394   453 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.446825   428 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.476624 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:20.477099 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:20.477512 29141 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:20.410895   369 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:05:20.462373 29146 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:20.481091 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:20.481196 29146 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:20.481314 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:20.412422   432 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:05:20.411127   447 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.472306 29147 raft_consensus.cc:2368] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10.
I20260709 13:05:20.483990 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:20.485496 29145 raft_consensus.cc:2393] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:20.486656 29064 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:20.490973 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:20.491698 29142 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:20.483274   467 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:20.494081   467 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:20.494681   328 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:20.496460   467 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:20.498997 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.500558 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:20.500735 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:20.490973 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:20.503604 29140 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:20.487346 29146 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.485059   512 consensus_queue.cc:1243] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:20.484297   437 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:20.518712   437 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:20.548090 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.613127 29146 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:20.641717 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:20.655876 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:20.656764 29064 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:20.642434 29073 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:20.661610 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:20.664362 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 161857us
I20260709 13:05:20.507845   452 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.679778 29142 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:20.520900   464 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:20.474872   472 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.598522   467 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:20.786550 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.816934 29146 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:05:20.721632 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.864924 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:20.868103 29141 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:20.868566 29140 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:20.869518 29141 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:20.870877 29141 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:05:20.782063   495 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.870536   467 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.872382   495 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:20.875452   472 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:20.865504 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:20.885571 29073 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:20.655407   337 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:20.868440 29066 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:20.879981   448 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:20.880409 32287 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.706382   346 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:20.881490   542 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:20.885600   413 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:05:20.870694 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:20.685917   437 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:20.889863 29064 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:20.699858   543 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:20.713306   452 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.909821 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:20.905889 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:20.902863 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:20.916846 29064 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:20.919356 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:20.887085   464 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.901404   428 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:20.931244 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:20.913024 32572 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:20.899701   337 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.940061 29073 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:20.910991   448 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:05:20.900781 32287 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:20.931612   366 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:20.908234   542 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.969756 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:20.915685   437 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
W20260709 13:05:20.964041   544 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:05:20.950583   495 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:20.972108   447 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:20.941231   399 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:05:20.992725   388 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:20.995222   391 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:21.001564   452 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:21.005084   532 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.685s	user 0.007s	sys 0.008s
I20260709 13:05:20.951272 32572 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:20.915527   543 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:21.038463 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:21.039134 29145 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:21.041841 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:21.042816 29140 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:20.950492   551 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:05:20.992777   366 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:05:21.048364 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:21.023774   543 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:20.985776   545 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:21.061337 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:21.014338   512 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 13:05:21.062860 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.064397 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2127) took 1197 ms. Trace:
I20260709 13:05:21.064846 29072 rpcz_store.cc:276] 0709 13:05:19.867306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.880297 (+ 12991us) service_pool.cc:224] Handling call
0709 13:05:19.880660 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.923516 (+ 42856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.923638 (+   122us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:19.924542 (+   904us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.066230 (+141688us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:20.066303 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.066392 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:21.004460 (+938068us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:21.056920 (+ 52460us) raft_consensus.cc:1685] finished
0709 13:05:21.056946 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.064329 (+  7383us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64001,"raft.queue_time_us":47265,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":5943,"spinlock_wait_cycles":14470912,"thread_start_us":90790,"threads_started":1,"wal-append.queue_time_us":371158}
I20260709 13:05:21.063483   427 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:21.064050   388 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:21.071712   530 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:21.072212   530 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:21.071871   388 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.074767 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 421) took 2272 ms. Trace:
I20260709 13:05:21.075137 29067 rpcz_store.cc:276] 0709 13:05:18.802453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.335046 (+532593us) service_pool.cc:224] Handling call
0709 13:05:19.368419 (+ 33373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.587245 (+218826us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.587279 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:19.587324 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:19.587357 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.074516 (+1487159us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54002cbfd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:21.074564 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.074710 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502749056}
W20260709 13:05:21.079430 32645 consensus_meta.cc:287] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.686s	user 0.011s	sys 0.000s
I20260709 13:05:21.080013   392 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:21.080811 32287 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.084184   464 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:21.054955   346 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.085533 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:21.086717   464 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:21.087265   464 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.087726   464 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:21.089416 32572 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.090997 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 584) took 3075 ms. Trace:
I20260709 13:05:21.091544 29905 rpcz_store.cc:276] 0709 13:05:18.015646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.079737 (+ 64091us) service_pool.cc:224] Handling call
0709 13:05:18.082680 (+  2943us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.381921 (+1299241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.090665 (+1708744us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54006ffc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:21.090920 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2880336640}
I20260709 13:05:21.098341 29145 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:21.099504 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:21.100057 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.136802 29145 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 9: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:21.137365 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 234479us
I20260709 13:05:21.139210 32645 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.140812 29217 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:05:21.141322 29217 pending_rounds.cc:85] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 8
I20260709 13:05:21.141603 29217 pending_rounds.cc:107] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:05:21.141892 29217 raft_consensus.cc:2889] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:21.143585 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:21.144372 29215 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.144039   492 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:21.145689 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:21.146458 29145 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:21.147152 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.148015 29216 raft_consensus.cc:2410] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:05:21.148257 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.148895 29222 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:21.103942   360 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:21.106849   440 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:21.107772 32354 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:21.113863   337 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.114684   555 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:21.115478 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 416) took 2347 ms. Trace:
I20260709 13:05:21.116101   491 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:21.126494   522 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.805s	user 0.004s	sys 0.011s
I20260709 13:05:21.127141   515 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:21.068116   545 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.180763 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 525) took 1772 ms. Trace:
I20260709 13:05:21.136242   427 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:21.145802 32645 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.130610 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:21.192456 29141 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.133275   450 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:21.133776   431 consensus_meta.cc:287] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.793s	user 0.009s	sys 0.003s
I20260709 13:05:21.135394   465 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.151204 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:21.152700 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.198994 29218 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:21.152943 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:21.200111 29219 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.153364 29214 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:21.153859 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:21.064848   490 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.017s	sys 0.033s
W20260709 13:05:21.155040 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 187790us
I20260709 13:05:21.154950 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:21.203003 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.155014 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.203846 29214 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:21.203831 29221 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:21.157610 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.161123 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:21.128698 29875 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:21.205453 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 76876.8us
I20260709 13:05:21.167241   464 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.130723   498 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:21.181231 29139 rpcz_store.cc:276] 0709 13:05:19.408385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.409041 (+   656us) service_pool.cc:224] Handling call
0709 13:05:19.422798 (+ 13757us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.686089 (+263291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.686131 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:19.686746 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 13:05:19.853960 (+167214us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:19.853997 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.989757 (+1135760us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54001bc6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:20.989825 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:21.125905 (+136080us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:21.180512 (+ 54607us) raft_consensus.cc:1685] finished
0709 13:05:21.180536 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.180695 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5441,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":661870592,"thread_start_us":166760,"threads_started":1,"wal-append.queue_time_us":633898}
I20260709 13:05:21.152271   409 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.156665   495 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:21.201388 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47620.3us
I20260709 13:05:21.205358 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:21.161194 29065 rpcz_store.cc:276] 0709 13:05:18.768217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.768364 (+   147us) service_pool.cc:224] Handling call
0709 13:05:18.768710 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.094715 (+326005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.094746 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:19.095212 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:05:19.290785 (+195573us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:19.321271 (+ 30486us) log.cc:844] Serialized 14 byte log entry
0709 13:05:19.639767 (+318496us) spinlock_profiling.cc:243] Waited 982 us on lock 0x7b1800191da0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:20.237856 (+598089us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b1800191da0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:21.114849 (+876993us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b1800191da0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287
0709 13:05:21.114999 (+   150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.115109 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:21.115167 (+    58us) raft_consensus.cc:1685] finished
0709 13:05:21.115186 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.115350 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30799,"prepare.run_cpu_time_us":1416,"prepare.run_wall_time_us":26402,"raft.queue_time_us":80249,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":16991,"spinlock_wait_cycles":9238400,"thread_start_us":493716,"threads_started":3,"wal-append.queue_time_us":227580}
I20260709 13:05:21.207058 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:21.207305 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:21.156734   360 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.202293   431 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:21.156755 32354 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:21.166447   542 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.170262   515 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:21.152128   492 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.161211   440 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.403750 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42040 (request call id 79) took 1518 ms. Trace:
I20260709 13:05:21.284627 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:21.409077 29902 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:21.337090 29215 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
W20260709 13:05:21.338125 29188 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.571s (SENT)
I20260709 13:05:21.203161 29222 raft_consensus.cc:2376] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11. Re-sending same reply.
I20260709 13:05:21.410986 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:21.352855 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.413447   498 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:21.366070   329 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.414433   431 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.414597   542 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.412811   465 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:21.349085   467 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.415297   465 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.414810   498 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.321671 29141 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:21.417248   498 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:21.418244 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:21.418370   465 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.213357 29213 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:21.419586 32354 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:21.421849   542 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:21.414350   329 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:21.423225 32354 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:21.420660 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.189908   555 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.426015 29213 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 9: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:21.426152   492 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.423408   464 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:21.412120 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 204923us
I20260709 13:05:21.427150 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.427933 29213 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:21.424278   542 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.318814 29219 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:21.424278   329 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.428110   555 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.217126 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:21.432121 29217 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:21.409879 29148 rpcz_store.cc:276] 0709 13:05:19.885334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.888195 (+  2861us) service_pool.cc:224] Handling call
0709 13:05:21.381872 (+1493677us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b5400296920. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a8a520 00007f7836b07652 00007f783bad560a 00007f783505b7d4 00007f783505b762 00007f783505b691 00007f783505b60c 00007f783505a8c2 00007f7834e4848c 00007f7834e478c6
0709 13:05:21.403636 (+ 21764us) inbound_call.cc:177] Queueing failure response
Metrics: {"spinlock_wait_cycles":5520768}
I20260709 13:05:21.434944 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:21.435883 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.436631 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:21.433071 29221 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:21.437639 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:21.439175 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:21.202615   490 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.017s	sys 0.033s
I20260709 13:05:21.436311 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:21.441171 29905 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:21.447366 29215 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:21.448362 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:21.449697 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 111646us
I20260709 13:05:21.453058 29185 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:21.215054   337 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:21.375303 32645 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.214334   427 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.410238   445 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.454409 29215 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:21.457147 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.476326 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:21.481885 29903 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:21.455595 29904 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:21.482439 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:21.482256 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:21.241775   557 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:21.484830 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:21.479859 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:21.440331   391 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:21.513432 29910 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:21.514108 29215 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:21.515028 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.502471 29217 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:05:21.514849 29139 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:21.517450 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:21.515244 29911 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:21.521241 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.522228 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.521080 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:21.525419 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 72444.8us
I20260709 13:05:21.535205 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:21.432529   407 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:21.538910 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:21.459867   448 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.525303 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:21.419173   409 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.592726   467 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:21.593400   467 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.593917   467 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.594444   409 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.600512 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:21.486078   447 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 13:05:21.460011   451 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:05:21.542284 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.435233   495 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:21.542111 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.480041   492 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:21.540094 29903 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:21.522523   545 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.592948 32357 raft_consensus.cc:993] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:05:21.604245 29217 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:21.488549   490 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.591468 29910 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:21.545683 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.603700 29222 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:21.524951   427 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:21.616564 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.620569 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:21.621655 29217 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:21.622597 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.544062 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:21.625214 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 81337.8us
I20260709 13:05:21.626024   409 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:21.626971 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 176454us
I20260709 13:05:21.627506 29185 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:21.510706   360 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.638013 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.543586   391 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:21.637738 29139 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:21.623267 29215 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:21.490322 32645 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:21.544569   415 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:21.652529 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:21.419174   515 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.539491   556 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.626574   409 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.654798 29215 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 13:05:21.692728   515 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:21.692658   556 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.693805 29215 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:05:21.525219   555 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 13:05:21.725622 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:21.726441 29217 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:21.510485   445 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.603426   407 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.730026 29219 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:21.607982   448 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:21.620703   465 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 13:05:21.699379   551 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.696838   492 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.702482   545 raft_consensus.cc:1081] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Signalling peer 7e78ffbcc98242068a874a5d9e341477 to start an election
I20260709 13:05:21.645119   450 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:21.718519   415 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.653425   451 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:21.768187   415 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:21.769104   451 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.721943   447 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:21.738562   527 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.520s	user 0.003s	sys 0.002s
I20260709 13:05:21.756742 29064 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:21.718149   409 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:21.642606   490 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:21.761307 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:21.734848   552 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.762977   556 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:21.779507   472 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.882s	user 0.013s	sys 0.001s
I20260709 13:05:21.763275   492 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:21.759889   551 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.774966   337 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:21.797699 29911 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:21.790256   440 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.794286   542 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:21.811338   542 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:21.812492 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:21.825098   575 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:05:21.779448   447 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:21.832542   447 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:21.833694   447 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:21.787194   552 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:21.784690   391 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.777742   450 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:21.834895 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.863982 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:21.880752 29069 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:21.897265 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:21.887764 29214 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:05:21.898237 29214 pending_rounds.cc:85] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 5
I20260709 13:05:21.898658 29214 pending_rounds.cc:107] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:05:21.899119 29214 raft_consensus.cc:2889] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:21.900483 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:21.901183 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 200323us
I20260709 13:05:21.863778   583 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:21.903311 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.903867 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:21.904215 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:21.886489   533 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.564s	user 0.017s	sys 0.000s
I20260709 13:05:21.903720   583 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.905287 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 593) took 2497 ms. Trace:
I20260709 13:05:21.905642 29909 rpcz_store.cc:276] 0709 13:05:19.407801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.413683 (+  5882us) service_pool.cc:224] Handling call
0709 13:05:19.455499 (+ 41816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.920129 (+464630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:21.905075 (+1984946us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54002a08d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:21.905224 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062576896}
I20260709 13:05:21.906965   583 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:21.867806   552 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.917196   389 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:21.889561 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:21.926795 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 299986us
W20260709 13:05:21.927075 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.927455 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:21.868528 29902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 591) took 2461 ms (client timeout 2142 ms). Trace:
W20260709 13:05:21.929014 29902 rpcz_store.cc:269] 0709 13:05:19.407213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.408672 (+  1459us) service_pool.cc:224] Handling call
0709 13:05:21.868245 (+2459573us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b5400682754. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:21.868399 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4356780288}
I20260709 13:05:21.946281 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:21.945564 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:21.947369 29145 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:21.946101 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:21.947685 29148 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
W20260709 13:05:21.948603 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:21.854162 32645 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.835444   495 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:21.848006   448 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:21.858130   492 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:21.856065   490 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.869686   450 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.859243   551 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.850055   446 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:21.876843   407 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:21.964870 29068 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:05:22.145350 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 75247.4us
I20260709 13:05:22.006248 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:22.046957 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 70723.8us
I20260709 13:05:21.997642 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:22.146963 29039 leader_election.cc:336] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.780s (SENT)
I20260709 13:05:22.005795 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.059870 29067 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:22.148053 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 58501.5us
I20260709 13:05:22.045085 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:22.149380 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.149461 29064 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:21.890630 29139 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 13:05:22.057770 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.150735 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.006808 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:21.962715 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:21.907860   525 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:21.870901 29218 raft_consensus.cc:1217] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 5.7->[8.8-10.9]   Dedup: 8.8->[10.9-10.9]
W20260709 13:05:22.152073 29039 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.936s (SENT)
I20260709 13:05:22.151767 29066 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:22.167313 29218 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:05:22.166672 29905 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
W20260709 13:05:22.168021 29908 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.005s	sys 0.015s
I20260709 13:05:22.168561 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:21.865855   542 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.834506   447 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:21.859575   582 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.848651   577 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:22.004900   432 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.005470   587 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:22.032317 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:21.893177   451 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:22.163818   557 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:21.888079   472 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.998067 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:21.983606 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.169054 29188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.006s	sys 0.014s
I20260709 13:05:22.176954 29141 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:22.168308 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2242) took 1019 ms. Trace:
I20260709 13:05:22.177464 29218 rpcz_store.cc:276] 0709 13:05:21.148512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.219106 (+ 70594us) service_pool.cc:224] Handling call
0709 13:05:21.870435 (+651329us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:05:21.870682 (+   247us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:22.167733 (+297051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.168252 (+   519us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48705920}
I20260709 13:05:22.178308 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:21.879544   337 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:22.173426 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.180073 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:22.177098 29188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.006s	sys 0.014s
I20260709 13:05:22.180820 29065 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:22.005316   399 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:22.168730 29908 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.005s	sys 0.015s
W20260709 13:05:22.173950 29221 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.012s	sys 0.019s
I20260709 13:05:22.179989 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:22.182775 29218 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:22.182865 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.187144 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:22.182220 29221 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.012s	sys 0.019s
I20260709 13:05:22.169986   525 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:22.170046   556 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:22.194933 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.197091 29065 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:22.197193 29188 leader_election.cc:400] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10.
I20260709 13:05:22.192214 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.198843 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:22.174268   587 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:22.175712   557 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:22.204871 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.211805 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:22.213028 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.214303 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.217073   448 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.217669   448 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:22.187959   542 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.207012 29071 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:22.183501   577 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:22.200778   525 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.240976 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 1993) took 1778 ms. Trace:
I20260709 13:05:22.241858   450 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:22.242293 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:22.244169   490 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:22.243836   450 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:22.244407 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.244624   490 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.245076   490 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:22.245119 29222 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:22.245747   450 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:22.246290 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.247534   498 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:22.176079   432 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:22.241493 29144 rpcz_store.cc:276] 0709 13:05:20.462270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.598484 (+136214us) service_pool.cc:224] Handling call
0709 13:05:20.655031 (+ 56547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.096126 (+441095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.096156 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:21.096191 (+    35us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:21.096217 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.240725 (+1144508us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54001bd0d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:22.240773 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.240910 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013941120}
I20260709 13:05:22.183105   472 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.250366 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.251329 29874 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:22.243592 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:22.183243   451 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:22.179101   582 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.209059   587 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.248543   492 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:22.190930   399 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:22.203950   556 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.267978 32572 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:22.268352   450 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:22.265439 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:22.266748 29222 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:22.231280   448 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:22.259174 29071 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:05:22.248216   498 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:22.287412 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:22.266748 29218 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:22.182575   589 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:05:22.192855   584 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:05:22.293795 29113 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:22.299280   407 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:22.299503   431 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:05:22.300709   432 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:22.293483   450 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:22.301223   415 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.001s
I20260709 13:05:22.303273   585 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:22.304427   360 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:22.292248 29038 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:22.305847   427 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.000s
I20260709 13:05:22.305944 29222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:05:22.309607 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:22.316236 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:22.291213   492 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:22.330617 29071 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:22.339584 29113 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:22.352934 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.353585 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.320780   450 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:22.319106 32357 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:22.284052   557 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:22.356923 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.324115 32572 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:22.332837   491 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:22.327157   360 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:22.326951   407 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:22.327795   467 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.729s	user 0.007s	sys 0.007s
I20260709 13:05:22.382786 29908 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:22.384932   595 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:05:22.387493   582 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:22.389065   594 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.508s	user 0.002s	sys 0.001s
I20260709 13:05:22.392129   491 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:22.393307   491 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.326857   413 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.058s
W20260709 13:05:22.373868 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.326918   415 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.405824 29071 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:22.409685 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.380544 32572 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.407507 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:22.359508   515 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.414243 29071 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:22.359992   592 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:22.362337 32645 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:22.359777   432 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:22.380712 32354 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:05:22.358990   498 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.407377   446 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:22.356604   552 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:05:22.438238 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.400969 32357 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:05:22.414018   557 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.416253   413 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:22.417850 32572 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.399443   467 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.470549 29071 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:22.446430 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.363787   577 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:22.450713 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 122899us
I20260709 13:05:22.474749 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:22.442903   596 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:05:22.419092   592 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:22.456751   498 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:22.456264   413 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:22.415795   530 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:22.464102 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.404825   360 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:22.472123 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:22.473197 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 250505us
I20260709 13:05:22.481489 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.455255   447 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:22.492250 28958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 227316us
I20260709 13:05:22.488867 29071 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:22.488967 29218 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:22.491320 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.429188   515 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:22.499348 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.500867 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:22.506309 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 125553us
I20260709 13:05:22.425719   492 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:22.508050 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.508584 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.508733 29218 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:22.509326 29071 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:22.509574 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.510172 29218 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:22.507112 29113 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:22.511421 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:22.512516 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:22.513085 29218 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:22.516919 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:22.518723 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 200400us
I20260709 13:05:22.519205   615 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:05:22.518954 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44315.1us
I20260709 13:05:22.489794   451 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:22.562898 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.592620 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 80471.8us
W20260709 13:05:22.594516 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.600142 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 80593.6us
I20260709 13:05:22.497560   472 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:22.450356   407 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.611902   615 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:22.605964 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:22.614709 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 94495.5us
I20260709 13:05:22.511238   360 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.516181   530 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:22.666109 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.670248 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:22.529203   409 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.745s	user 0.013s	sys 0.000s
I20260709 13:05:22.671232 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:22.520483   432 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:22.517091   399 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:22.682246 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.673869 29216 raft_consensus.cc:2410] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:05:22.674588 29218 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:22.528940   582 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:05:22.682996 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.516801   447 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.164s
I20260709 13:05:22.516333   448 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.161s
I20260709 13:05:22.520857   337 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.158s
I20260709 13:05:22.520700   431 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.158s
I20260709 13:05:22.705379 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.696709 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.516669   603 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.175s
I20260709 13:05:22.674477   552 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.678102   432 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.710911 29906 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:22.704403   619 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:22.677772   472 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:05:22.616398   413 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.680295   409 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:22.730398 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 136074us
I20260709 13:05:22.736310 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:22.715746   415 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:22.726576   432 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:22.739682 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:22.738406 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 46648us
I20260709 13:05:22.741886   413 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:22.743201   605 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
W20260709 13:05:22.748360   607 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:05:22.749317 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.750818 29071 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:05:22.752115 29071 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 13:05:22.753168   490 consensus_meta.cc:287] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.507s	user 0.006s	sys 0.004s
I20260709 13:05:22.754571   388 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:22.763034 29038 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.900s (SENT) [suppressed 38 similar messages]
I20260709 13:05:22.764812 29219 raft_consensus.cc:1217] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[3.3-12.7]   Dedup: 3.3->[6.4-12.7]
I20260709 13:05:22.770071 29219 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: [3.3-12.7]
W20260709 13:05:22.787104 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.795440 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.760689   602 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 13:05:22.836265 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:22.828857 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:22.838308 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:22.834254 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:22.844187 29066 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:22.741287   619 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:22.725183   552 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:22.730170   610 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.110s
W20260709 13:05:22.824878   613 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:05:22.739346   615 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.814947   608 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:22.875753 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.814229   490 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.915720   445 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:22.917697   490 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:22.919469 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.920658 29071 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:05:22.922199   619 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:22.925258   525 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.667s	user 0.011s	sys 0.000s
I20260709 13:05:22.926156   525 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:22.929301   587 consensus_meta.cc:287] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.001s
I20260709 13:05:22.929776   587 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.920428   552 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:22.925832   450 consensus_meta.cc:287] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.000s
I20260709 13:05:22.935621   515 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:22.936295   450 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:22.939364   450 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:22.907641   604 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:05:22.941218   608 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:22.873548   623 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:22.873805   617 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:22.944274   617 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.944805   617 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:22.945242 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.946336 29068 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:22.948930   348 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:22.949471 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.758309   618 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.171s
I20260709 13:05:22.950543 29064 raft_consensus.cc:2410] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:05:22.951620   620 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.949503   348 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:22.953142   348 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.954779   348 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:22.958135   625 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:22.958837   625 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.960491   587 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:22.952507   620 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:22.962306 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 660) took 1065 ms. Trace:
I20260709 13:05:22.962850 29905 rpcz_store.cc:276] 0709 13:05:21.896828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.896978 (+   150us) service_pool.cc:224] Handling call
0709 13:05:22.962034 (+1065056us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54007102d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:22.962179 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617396096}
I20260709 13:05:22.964040   608 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:22.965540 29214 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:05:22.966290 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2181) took 1015 ms. Trace:
I20260709 13:05:22.966602 29214 rpcz_store.cc:276] 0709 13:05:21.950683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.964885 (+1014202us) service_pool.cc:224] Handling call
0709 13:05:22.965280 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.965994 (+   714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.966228 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:22.968031 29069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 436) took 1552 ms (client timeout 1905 ms). Trace:
W20260709 13:05:22.968668 29069 rpcz_store.cc:269] 0709 13:05:21.415951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.434658 (+ 18707us) service_pool.cc:224] Handling call
0709 13:05:22.967768 (+1533110us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54002ad354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:22.967878 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1023645056}
I20260709 13:05:22.947674 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:22.970292 29069 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:22.971525 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.973425 29072 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:05:22.974951 29072 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:05:22.845429   431 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:22.975402   525 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:22.975653   431 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:22.977046   431 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.978291   431 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.981886 32354 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:22.982313 32354 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:22.983225 32354 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:22.984952 29222 raft_consensus.cc:606] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:22.985157 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 568) took 1804 ms. Trace:
I20260709 13:05:23.015812 29222 raft_consensus.cc:614] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:22.989336 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 536) took 1929 ms. Trace:
I20260709 13:05:23.013906 29068 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:05:22.868389   614 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.002s
W20260709 13:05:23.033676 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.034760 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.017315 29140 rpcz_store.cc:276] 0709 13:05:21.060503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.060710 (+   207us) service_pool.cc:224] Handling call
0709 13:05:21.132599 (+ 71889us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.194528 (+ 61929us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:21.433921 (+239393us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.542407 (+108486us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:21.542462 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:21.542521 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.989012 (+1446491us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b54003467d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:22.989085 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.989254 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66615,"prepare.queue_time_us":315037,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":2513,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":10470,"spinlock_wait_cycles":141678976,"thread_start_us":299011,"threads_started":2,"wal-append.queue_time_us":313209,"wal-append.run_cpu_time_us":1580,"wal-append.run_wall_time_us":1064368}
I20260709 13:05:23.015897 29221 rpcz_store.cc:276] 0709 13:05:21.180985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.437853 (+256868us) service_pool.cc:224] Handling call
0709 13:05:21.438174 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.438433 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:22.189999 (+751566us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.984651 (+794652us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:22.984730 (+    79us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:22.984805 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.984918 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.985068 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56881,"prepare.queue_time_us":73297,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":11099,"raft.queue_time_us":51432,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":3328,"spinlock_wait_cycles":17438592,"thread_start_us":1529874,"threads_started":3,"wal-append.queue_time_us":570541}
I20260709 13:05:22.833305   409 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:22.891538   542 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.663s	user 0.008s	sys 0.004s
I20260709 13:05:22.845705   467 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:22.873682   557 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:22.870147   621 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:23.100713 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.141461 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.054777 32354 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.148515   620 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:23.174043   625 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:23.054859   557 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:23.187959 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.185916 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 342466us
I20260709 13:05:23.194803 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:23.205456 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:23.206815 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:23.207561 29217 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:23.210649 29065 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:05:23.211277 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:23.140429   621 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.213092   492 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:23.213795   492 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:23.214179   492 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:23.215379   621 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:23.216266 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:23.240839 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:23.241572 29068 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:23.240839 29066 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:23.216689 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:23.242798 29069 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:05:23.213223 29216 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:23.244580 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:23.245934 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:23.211907 29217 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:23.254220 29146 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:23.255417 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2000) took 2776 ms. Trace:
W20260709 13:05:23.258538 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.260983 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:23.261961 29216 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:23.263216 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:23.257772 29146 rpcz_store.cc:276] 0709 13:05:20.478902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.253429 (+2774527us) service_pool.cc:224] Handling call
0709 13:05:23.253992 (+   563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.254976 (+   984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.255331 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:23.054740   432 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:23.267966 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:23.268520 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.264983 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:23.223848   492 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:23.233868   498 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.746s	user 0.010s	sys 0.006s
I20260709 13:05:23.270076 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:23.238289 32645 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.766s	user 0.008s	sys 0.012s
I20260709 13:05:23.202665   557 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.280201   603 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:23.280642   603 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:23.281025   603 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:23.283469 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:23.286049 29216 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:23.286217   432 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:23.274444   608 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:23.287345   608 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.287734   608 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:23.279310   609 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.289402   609 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:23.292042   585 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:23.292865 32641 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:23.293036   625 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:23.293347 32641 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:23.273774   495 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:23.294194   617 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:23.294422 32641 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:23.278000 32645 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:23.276991   623 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.272446   629 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:05:23.294386   495 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:23.286540   492 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:23.302018 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 159686us
W20260709 13:05:23.307616 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.312538 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.271950   611 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:23.315032 29871 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:23.311322 32641 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.310616   623 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:23.316712   611 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.324303   611 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:23.357180 29217 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:23.310202   495 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.360339 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:23.318286   452 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.321671   609 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:23.380523 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 266185us
I20260709 13:05:23.371316 32645 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:23.369172   492 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:23.338059   623 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:23.390081 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.391973 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:23.393517 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:23.394227 29065 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.400311 32645 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.403065 32645 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:23.407665 29039 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.500s (SENT)
I20260709 13:05:23.408954 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:23.410698   603 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.414209   611 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:23.416388   617 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:23.428716   611 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:23.440774 29065 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:23.446257   556 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:23.389900   615 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:23.447295 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:23.389652   492 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:23.448499   492 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:23.373508   432 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.449517   492 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:23.450343   615 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:23.451442   615 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:23.449517   432 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:23.381412   530 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.704s	user 0.015s	sys 0.011s
I20260709 13:05:23.458371 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:23.384600 32572 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.906s	user 0.022s	sys 0.003s
I20260709 13:05:23.470084 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:23.472224 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 156967us
I20260709 13:05:23.472843 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:23.382177   452 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.473932 29905 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
W20260709 13:05:23.474501 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 195518us
I20260709 13:05:23.476840 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:23.483081 29185 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:23.483672 29903 raft_consensus.cc:2410] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:05:23.487215 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.487700 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.479144 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:23.474987 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:23.488844 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.489408 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.489732 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:23.489786 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.491390 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:23.491750 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.492551 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.493875 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.494202 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:23.494803 29908 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:23.469045   432 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:23.495419 29039 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.543s (SENT)
I20260709 13:05:23.495886 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:23.472316   453 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:23.500643   618 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:23.500905   603 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:23.502038 29140 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:23.498111 32357 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:23.503975   453 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:23.504452   453 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:23.506146   610 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:23.506503   452 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:23.506683   610 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:23.503454 32357 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.509856 32357 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:23.495894   556 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:23.512189   431 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.532s	user 0.002s	sys 0.009s
I20260709 13:05:23.507066   452 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:23.512701   556 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:23.513052   452 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.513727   556 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:23.515175   610 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:23.515347   452 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:23.465873 32572 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.518066 32572 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:23.520118   452 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:23.521159   452 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:23.522323   452 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.524859 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:23.547892 29140 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:23.526507 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:23.527072   348 consensus_meta.cc:287] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.004s
I20260709 13:05:23.528326 29141 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:23.529223 29142 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:23.529328 29139 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:05:23.529780 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.530627 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:23.532351 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:23.552243 29143 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:23.534361   608 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.551689 29148 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
W20260709 13:05:23.556218 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 82247.7us
I20260709 13:05:23.531617   557 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.571978 29065 raft_consensus.cc:2368] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12.
I20260709 13:05:23.573321 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:23.575649 29908 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:23.578751 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.555173 29903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79cdfc6a725145348b70034422ef477b"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:54892
I20260709 13:05:23.579744 29903 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:23.580240 29903 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:23.585065 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.590837 29910 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:05:23.569716   391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.688s	user 0.014s	sys 0.009s
I20260709 13:05:23.591765 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:23.594620 29908 raft_consensus.cc:2410] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:05:23.601684 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:23.604149 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:23.605367 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:23.606132 29140 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:23.606796 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:23.607564 29909 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:23.607806 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:23.608610 29908 raft_consensus.cc:2368] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
I20260709 13:05:23.611990 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:23.612604   589 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:23.612778 29140 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:23.612999   589 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:23.613816   589 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.614923   589 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.615048 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:23.615901 29140 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:05:23.627295 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.042s	sys 0.112s
I20260709 13:05:23.631724 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:23.632503 29068 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:23.633695 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:23.627791 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.043s	sys 0.113s
I20260709 13:05:23.634788 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:23.635190 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 898869us
I20260709 13:05:23.635468 29068 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:23.636116 29113 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:23.637693   369 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:23.638120   369 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.640486 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:23.643198 29140 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
W20260709 13:05:23.649556 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.651057 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:23.648993 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:23.645053 29903 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:23.595970   391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.714s	user 0.015s	sys 0.009s
I20260709 13:05:23.676633 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2174) took 2027 ms. Trace:
I20260709 13:05:23.647428   369 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:23.657958   640 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:23.660642 32572 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:05:23.600392   525 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:05:23.718462 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.713071 29219 rpcz_store.cc:276] 0709 13:05:21.648881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.741021 (+ 92140us) service_pool.cc:224] Handling call
0709 13:05:21.741371 (+   350us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 13:05:22.770490 (+1029119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.676292 (+905802us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540009f1d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:23.676566 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353120640}
I20260709 13:05:23.726203 29903 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:23.727686 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:23.728439 29145 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:23.724362 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:23.729610 29219 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.728335 29067 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:23.730967 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:23.731724 29140 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:23.709896 32357 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:23.712862   640 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.740003 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:23.743707 29065 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:23.745821 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:23.745860 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:23.653550   405 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:05:23.740655 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2166) took 1397 ms. Trace:
W20260709 13:05:23.731379   616 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.841s	user 0.015s	sys 0.005s
I20260709 13:05:23.751112 29073 rpcz_store.cc:276] 0709 13:05:22.342931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.972022 (+629091us) service_pool.cc:224] Handling call
0709 13:05:23.021107 (+ 49085us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:23.047543 (+ 26436us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:23.140488 (+ 92945us) log.cc:844] Serialized 14 byte log entry
0709 13:05:23.465446 (+324958us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:23.468160 (+  2714us) log.cc:844] Serialized 52 byte log entry
0709 13:05:23.468339 (+   179us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:23.740109 (+271770us) log.cc:844] Serialized 14 byte log entry
0709 13:05:23.740342 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.740412 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:23.740450 (+    38us) raft_consensus.cc:1685] finished
0709 13:05:23.740471 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:23.740599 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":92946,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":759,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":19750,"thread_start_us":515723,"threads_started":2,"wal-append.queue_time_us":375787}
I20260709 13:05:23.753700 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:23.754432 29073 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:23.746584 29065 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:23.756450 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:23.732666   495 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:23.749667 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:23.759419 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:23.758759 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.761224 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.761770 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.762310 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.739979 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:23.766170 29140 raft_consensus.cc:2410] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:23.766930 29219 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
W20260709 13:05:23.768239 29188 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.586s (SENT)
I20260709 13:05:23.775315 29067 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:23.724404   639 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:05:23.778381 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:23.782568 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:23.783409 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:23.786666 29219 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:23.788899 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:23.790115 29068 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:23.790861 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:23.783035 29067 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
W20260709 13:05:23.785110 29039 leader_election.cc:336] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.587s (SENT)
I20260709 13:05:23.793399 29039 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:23.747251 32357 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:23.795086 29067 pending_rounds.cc:85] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 9
I20260709 13:05:23.803565 29067 pending_rounds.cc:107] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:05:23.804018 29067 raft_consensus.cc:2889] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:23.749081   640 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:23.795981 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 139151us
I20260709 13:05:23.808557 29875 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:23.796584 29073 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:23.800926 29065 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:23.810859 29073 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:23.811350 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:23.812553 29140 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:05:23.764683   530 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:23.814409 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.775436   617 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.818994 29067 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:23.820955 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:23.822248 29219 raft_consensus.cc:2410] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:05:23.822683 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 101374us
I20260709 13:05:23.802441 29145 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:23.747139   512 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:05:23.791061   525 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:23.824142 29067 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:23.803822   431 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:23.825640 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:23.828825 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:23.829056 29067 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:23.809893   615 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:23.828032 29145 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:23.789882   621 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:05:23.827281 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:23.828655 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:23.832818 29217 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
W20260709 13:05:23.861994 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.821595   530 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.824324   617 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.874812 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:23.874303 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:23.828614   366 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:23.884392   366 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:23.884671   643 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:05:23.831646   557 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:23.884946   650 log.cc:927] Time spent T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:05:23.828824   391 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:05:23.884909   366 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:23.836264 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:23.897840 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.900738   652 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:23.903460   651 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:05:23.911123 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:23.912664 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:23.913285   495 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:23.922609   592 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.872584   621 consensus_queue.cc:1243] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:23.927008   592 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.925349 29145 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:23.924893   617 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:23.929628 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:23.934255 29145 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:23.939961 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:23.942389   617 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.941533 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:23.829121   641 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:23.945642 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:23.947013 29906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 652) took 2502 ms (client timeout 1510 ms). Trace:
W20260709 13:05:23.947398 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.948271 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2155) took 1722 ms. Trace:
W20260709 13:05:23.947396 29906 rpcz_store.cc:269] 0709 13:05:21.444391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.444535 (+   144us) service_pool.cc:224] Handling call
0709 13:05:23.946833 (+2502298us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b5400767fd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:23.946936 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912458496}
I20260709 13:05:23.948822 29069 rpcz_store.cc:276] 0709 13:05:22.225851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.971105 (+745254us) service_pool.cc:224] Handling call
0709 13:05:23.242473 (+271368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:23.253585 (+ 11112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:23.948089 (+694504us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b5400185e50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:23.948214 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623308544}
W20260709 13:05:23.953708 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.934664 29068 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:23.937939   348 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:23.937127   366 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:23.913532 29221 raft_consensus.cc:2368] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
I20260709 13:05:23.960258 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:23.961295 29907 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:23.993888 29145 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:23.999176 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.001281   391 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:05:24.001176 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 165268us
I20260709 13:05:24.001638 29218 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:05:24.001909 32572 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:24.007858   525 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.002601   366 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:24.008908   391 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:24.016608 29145 raft_consensus.cc:2368] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9.
I20260709 13:05:24.017500 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.017117   391 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.004005   621 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:24.018134   621 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:24.018246   391 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:24.018586   621 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:24.020289 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:24.021134 29217 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:24.022118 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:24.022713 29217 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:24.024765 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.026041 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:24.028362 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.834234   452 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.196s
I20260709 13:05:24.023707 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:24.051482 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:24.052428 29217 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:05:24.041824   640 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.058796 29145 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:24.059842 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.060473 29145 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.061277 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:24.012032 29218 pending_rounds.cc:85] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 8
I20260709 13:05:24.072201 29218 pending_rounds.cc:107] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:05:24.072515 29218 raft_consensus.cc:2889] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:24.014022 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:24.074222 29906 raft_consensus.cc:2376] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12. Re-sending same reply.
W20260709 13:05:24.106424 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.148629 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:24.158676 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.167217 29217 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:24.167276 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:24.168350 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:24.169312 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:24.170871 29221 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:24.205232 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:24.208882   369 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.210325 29140 raft_consensus.cc:2368] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11.
I20260709 13:05:24.211441   369 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:24.178388 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.180665   530 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:24.009635   453 consensus_meta.cc:287] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.004s
W20260709 13:05:24.217047 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.166808   640 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:24.184680 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.188849 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:24.219180   640 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:24.190943   617 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:23.980994   348 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:24.219694 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.220594 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.221369 29113 leader_election.cc:400] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
I20260709 13:05:24.220865   348 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.221817 29113 leader_election.cc:403] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:24.222487   348 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:24.191759 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.192289 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:24.223855 29217 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:24.224011   530 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
W20260709 13:05:24.224843 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.012349 32572 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:24.193662 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:24.228885 29219 raft_consensus.cc:2393] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 9: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:24.229967 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:24.230762 29221 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:05:24.197010 29037 connection.cc:256] RPC call timeout handler was delayed by -1.1062226099993495s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
W20260709 13:05:24.237772 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.216955   453 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.239825 29904 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:23.836409 32357 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.219777   640 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:24.233844 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:24.196267 29145 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:24.150517 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.241412   453 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:24.247941 29147 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:24.249053   615 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.249269 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:24.249836   615 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.250928 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:24.252166   623 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:24.252914   623 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:24.245357   640 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.253423   623 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:24.253611   348 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:24.257692   615 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:24.244326 29219 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:24.258950 29219 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:24.247386 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.262670 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:24.263475 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.258491   615 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.263891 29145 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:24.266407 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.267671 29145 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:24.101504   495 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.264218 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.287839 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2152) took 2084 ms. Trace:
I20260709 13:05:24.289587 29071 rpcz_store.cc:276] 0709 13:05:22.203232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.752802 (+549570us) service_pool.cc:224] Handling call
0709 13:05:22.753101 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:22.921239 (+168138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.287439 (+1366200us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540012aa54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:24.287748 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384417536}
I20260709 13:05:24.291445 29147 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:24.264438 29219 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.292461 29219 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:24.271438 29221 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:05:24.265095 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:24.281021 28960 maintenance_manager.cc:419] P 9c100c1aaf1b493089df815bded98302: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033395
I20260709 13:05:24.297254 29221 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:24.274268   623 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.277573 29217 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:24.227109 32572 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.269912   348 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:24.293622 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.299605 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:24.295694   491 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:24.305124 29038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.010s	sys 0.005s
I20260709 13:05:24.306309 29221 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:24.306208 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:24.317101 29038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.010s	sys 0.005s
I20260709 13:05:24.307410 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:24.317513 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 707527us
I20260709 13:05:24.318293 29145 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:05:24.319309 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.320314 29145 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 13:05:24.314831 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.317507 29147 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:24.324138 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.324918 29147 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:24.326085 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.327265 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:24.328485 29875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.015s	sys 0.006s
W20260709 13:05:24.329530 29875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.015s	sys 0.006s
W20260709 13:05:24.329869 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 517528us
I20260709 13:05:24.334930 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:24.344691 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.346597 29145 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:24.348371 29145 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:24.349375 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:24.351591 29112 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:24.361784   589 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.746s	user 0.008s	sys 0.005s
I20260709 13:05:24.362485   589 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:24.363338 29214 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
W20260709 13:05:24.364240 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.365860 29112 leader_election.cc:400] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
W20260709 13:05:24.316397   577 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.569s	user 0.022s	sys 0.007s
W20260709 13:05:24.368065 29146 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.002s	sys 0.031s
I20260709 13:05:24.326946   453 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:05:24.369213 29146 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.002s	sys 0.031s
I20260709 13:05:24.369515 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:24.369213   577 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.622s	user 0.022s	sys 0.008s
I20260709 13:05:24.381516   446 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.381369   621 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.382098   446 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.383576   621 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:24.385627 29144 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:24.386238 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:24.330670   530 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:05:24.387779 29144 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: [6.7-9.8]
I20260709 13:05:24.390319 29145 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:24.388502 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:24.392398 29219 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:24.368350   417 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:24.297137   495 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.395794 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:24.397928 29907 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:24.353289   348 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:24.311355   369 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:05:24.329152   617 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:05:24.399842 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:24.400103   329 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:24.394526   417 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:24.401684   453 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:24.403184   446 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:24.418635   417 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:24.417464   617 consensus_queue.cc:1243] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:24.457906   623 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:24.470160   589 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:24.471295   589 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.472671   615 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:24.514991 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.394912 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.517088 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2046) took 1031 ms. Trace:
I20260709 13:05:24.519115 29144 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:24.399092 29147 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:24.517975 29065 rpcz_store.cc:276] 0709 13:05:23.485406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.810702 (+325296us) service_pool.cc:224] Handling call
0709 13:05:23.811026 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:23.811347 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:23.811372 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:23.811832 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:05:24.309756 (+497924us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:24.309808 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.309902 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:24.516373 (+206471us) raft_consensus.cc:1685] finished
0709 13:05:24.516417 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:24.516811 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25622,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3907,"thread_start_us":497571,"threads_started":1,"wal-append.queue_time_us":614953}
I20260709 13:05:24.530915 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:24.515048 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2049) took 1004 ms. Trace:
I20260709 13:05:24.532510 29073 rpcz_store.cc:276] 0709 13:05:23.510280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.818990 (+308710us) service_pool.cc:224] Handling call
0709 13:05:23.819384 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.819646 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:24.194192 (+374546us) log.cc:844] Serialized 14 byte log entry
0709 13:05:24.513805 (+319613us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:24.513992 (+   187us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:24.514215 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.514385 (+   170us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:24.514747 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":100657,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"thread_start_us":685130,"threads_started":2,"wal-append.queue_time_us":253782}
I20260709 13:05:24.548835 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.550076 29144 raft_consensus.cc:2376] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11. Re-sending same reply.
I20260709 13:05:24.551781 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.553041 29144 raft_consensus.cc:2410] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:24.554275 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.565301 32354 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:24.565418 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:24.566011 32354 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.566434 32354 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.581737 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:24.408887   329 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:24.585479   437 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:24.583941 32357 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:05:24.591737   453 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.597646   329 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.601231 29221 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:24.601451   329 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:24.603796 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 268800us
W20260709 13:05:24.601297 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 299646us
W20260709 13:05:24.610512 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.558353 29144 raft_consensus.cc:2376] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11. Re-sending same reply.
I20260709 13:05:24.622864 32354 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.556617 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:24.628379   617 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:24.634254 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 77931.6us
I20260709 13:05:24.637449 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:24.640101 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.642709 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:24.643267 29073 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:24.643618 29070 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.644069 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:24.645385 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.646456 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.647310 29188 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.718s (SENT)
I20260709 13:05:24.646330 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:24.648743 32357 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:24.650790 29907 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:24.651875 29906 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:24.653935 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:24.641763 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:24.655843 29144 raft_consensus.cc:2376] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12. Re-sending same reply.
I20260709 13:05:24.656713 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2271) took 1446 ms. Trace:
I20260709 13:05:24.650652 29213 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:24.659355   417 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.664563 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 29) took 2482 ms. Trace:
I20260709 13:05:24.667083 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 603) took 1193 ms. Trace:
I20260709 13:05:24.676493 29910 rpcz_store.cc:276] 0709 13:05:23.473410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.474693 (+  1283us) service_pool.cc:224] Handling call
0709 13:05:24.666620 (+1191927us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54005a2654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:24.666891 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234627840}
I20260709 13:05:24.677757 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:24.679178   453 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:24.679646   453 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.679847 32354 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:24.680120   453 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:24.659555 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 581) took 2337 ms. Trace:
W20260709 13:05:24.668066 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.668895 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 663) took 1102 ms. Trace:
W20260709 13:05:24.670773 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.675704 29070 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:24.672950 29216 rpcz_store.cc:276] 0709 13:05:23.209986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.262897 (+ 52911us) service_pool.cc:224] Handling call
0709 13:05:24.656212 (+1393315us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b54000cc6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:24.656473 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40030592}
I20260709 13:05:24.685163 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2180) took 2784 ms. Trace:
W20260709 13:05:24.695436 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.682425 29218 rpcz_store.cc:276] 0709 13:05:22.321820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.697500 (+375680us) service_pool.cc:224] Handling call
0709 13:05:22.697816 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.073041 (+1375225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.658908 (+585867us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54000bde54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:24.659307 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2997542784}
I20260709 13:05:24.696795 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:24.698160 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.699998 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.676412 29222 rpcz_store.cc:276] 0709 13:05:22.181472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.305787 (+124315us) service_pool.cc:224] Handling call
0709 13:05:24.664051 (+2358264us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54002f62d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b07853 00007f783bad560a 00007f783505b7d4 00007f783505b762 00007f783505b691 00007f783505b60c 00007f783505a8c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:24.664257 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1505825536}
I20260709 13:05:24.692350 29213 rpcz_store.cc:276] 0709 13:05:21.891894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.649621 (+2757727us) service_pool.cc:224] Handling call
0709 13:05:24.650167 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.673060 (+ 22893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.675953 (+  2893us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:24.702913 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.704329 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:24.705597 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.706297   676 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:24.706473 29144 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:24.707005   676 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:24.707216   512 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:24.708228 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.708518   676 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.709481 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.709532   542 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:24.682968 29904 rpcz_store.cc:276] 0709 13:05:23.566103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.566240 (+   137us) service_pool.cc:224] Handling call
0709 13:05:23.566641 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.240331 (+673690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.668316 (+427985us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b54004e61d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:24.668676 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1547629824}
I20260709 13:05:24.704351 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:24.710856 29144 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:24.711953 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.713130 29144 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:05:24.714089 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.715579 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:24.700562 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59097us
I20260709 13:05:24.716881 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.717334   434 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:24.717967 29065 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:24.717870   625 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:24.718230 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.696414   525 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.719481   525 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.721392   585 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:24.721925   585 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:24.722774   525 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:24.724730 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 71144.3us
I20260709 13:05:24.726081 29218 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:24.726264 32357 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.726465 32354 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:24.727092 32357 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.710160   542 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:24.727793   617 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:24.729668   617 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:24.730415   437 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:24.730978   437 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:24.733115 29222 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:24.730947   617 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.730270 32357 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:24.726111 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:24.732039 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:24.695892 29186 connection.cc:256] RPC call timeout handler was delayed by -1.4984566379998796s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19940000000000002s for a response
I20260709 13:05:24.741114 29903 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.724825   681 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:24.707764   512 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:24.729337   542 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:24.743726 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.738927   680 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.135s	user 0.004s	sys 0.000s
I20260709 13:05:24.750258 32357 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:24.747004 29186 connection.cc:256] RPC call timeout handler was delayed by -1.4453566379997937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260709 13:05:24.768440   608 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:24.770802 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 69053.5us
I20260709 13:05:24.767628   617 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:24.775887 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:24.777508 29186 connection.cc:256] RPC call timeout handler was delayed by -1.3049566380004762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 13:05:24.777808 29186 connection.cc:256] RPC call timeout handler was delayed by -1.2968566380004631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
I20260709 13:05:24.772413   542 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:24.762809   593 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.780186   625 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:24.782495   625 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.771876   584 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.777041 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:24.791289   512 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.792747   512 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:24.780026 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:24.796459   584 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.799587 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2190) took 1063 ms. Trace:
I20260709 13:05:24.800544 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.801018 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:24.801955 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.800064 29217 rpcz_store.cc:276] 0709 13:05:23.736434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.300430 (+563996us) service_pool.cc:224] Handling call
0709 13:05:24.307099 (+  6669us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:24.307379 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:24.307427 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:24.327358 (+ 19931us) log.cc:844] Serialized 26 byte log entry
0709 13:05:24.663199 (+335841us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:24.663314 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.663424 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:24.799189 (+135765us) raft_consensus.cc:1685] finished
0709 13:05:24.799214 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:24.799402 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":296168,"threads_started":1,"wal-append.queue_time_us":372359}
I20260709 13:05:24.804143 29214 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:24.805780   453 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.807488 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.808444 29148 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:24.809188 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.809908 29148 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:24.810674 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.811267 29148 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
W20260709 13:05:24.812475 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.799643 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.813445   437 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.815132 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:24.813463 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 166252us
I20260709 13:05:24.816663 29910 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
W20260709 13:05:24.817068 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.817129 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.818015 29112 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:24.818681   679 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:24.818979   681 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:24.819171   679 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:24.819793   681 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:24.820024   625 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:24.820690   625 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.820552   679 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.818048 29141 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:24.821308   625 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.822119   679 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:24.822383 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.824184   681 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:24.827020 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.827996 29147 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:05:24.829094 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.828923   585 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.830279   585 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:24.832368   684 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:05:24.828948 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.835017   348 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:24.836299 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 63595.1us
I20260709 13:05:24.851096 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:24.818965 29188 leader_election.cc:336] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.542s (SENT)
I20260709 13:05:24.854074   625 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:24.858876 29216 raft_consensus.cc:1217] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Deduplicated request from leader. Original: 2.2->[3.3-12.7]   Dedup: 3.3->[6.4-12.7]
I20260709 13:05:24.859256 29139 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:24.859570 29216 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: [3.3-12.7]
W20260709 13:05:24.860268 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.837289 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:24.852586 29148 raft_consensus.cc:2376] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10. Re-sending same reply.
I20260709 13:05:24.795953   417 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 7 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:24.862294 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:24.868642 29141 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:24.872314   452 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:24.872870 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 104930us
I20260709 13:05:24.873541 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.874253 29871 leader_election.cc:400] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12.
I20260709 13:05:24.874739 29871 leader_election.cc:403] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:24.878608   495 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:24.879550 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.879753   530 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:24.880355   530 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:24.880667 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 43525.5us
I20260709 13:05:24.881618   530 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.883059   530 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:24.884193 29186 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:24.885253   474 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:05:24.885983 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 99955.2us
I20260709 13:05:24.887004 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:24.887871 29219 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:24.888835   584 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:24.889273 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.889698 29219 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:05:24.890316 29072 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:24.888067 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.874599 29142 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:24.919230 29903 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:24.926306 29216 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:24.925108 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.930020 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.933912 29218 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:24.871156 29037 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:24.934746 29148 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:24.804143   593 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:24.941063 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:24.948990   525 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:05:24.953219   641 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:24.954129   641 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.956650 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:24.959123   593 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:24.937562   584 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:24.963279 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:24.967944 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.969518 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:24.973814 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:24.933712   348 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:24.986122 29139 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:24.988714   611 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:24.988997 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 209354us
I20260709 13:05:24.990561 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:24.990758 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:24.991766 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:24.992605 29071 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:24.993994 29220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2230) took 3840 ms (client timeout 1900 ms). Trace:
I20260709 13:05:24.994793   618 raft_consensus.cc:2764] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:05:24.995226   615 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:24.979221   417 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.999320 29073 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:24.968556   453 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:24.985246 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:24.979030 32357 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:05:24.994498 29220 rpcz_store.cc:269] 0709 13:05:21.138111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.143144 (+  5033us) service_pool.cc:224] Handling call
0709 13:05:24.929629 (+3786485us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54000799d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:24.978745 (+ 49116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2816728448}
I20260709 13:05:25.012899 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:25.016261   467 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:25.017431   405 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:25.017743 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 143851us
I20260709 13:05:25.018373 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2272) took 1186 ms. Trace:
I20260709 13:05:25.054392 29142 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:25.018226   417 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:25.033368   525 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12.
I20260709 13:05:25.073822   417 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:25.023684 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 204811us
I20260709 13:05:24.996289 32354 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.036250 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:25.023797 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:25.037954 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:25.024435   329 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:25.025161 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 138327us
I20260709 13:05:25.039536 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:25.028177 29144 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:25.028662   369 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:25.025410 29140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2256) took 1333 ms (client timeout 1672 ms). Trace:
I20260709 13:05:25.044412 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:25.001523   615 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:25.017127 29215 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:25.055967 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.089942 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.025322 29070 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
W20260709 13:05:25.090530 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 52719.4us
W20260709 13:05:25.035504 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 95582.4us
I20260709 13:05:25.068638 29139 raft_consensus.cc:2410] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:05:25.069205   452 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:25.082890 29221 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:25.083838 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:25.095746 29142 raft_consensus.cc:2410] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:25.054242   405 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.106139   405 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:25.111569   437 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:25.088057   329 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:25.123036   329 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:25.054927 29146 rpcz_store.cc:276] 0709 13:05:23.832085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.388870 (+556785us) service_pool.cc:224] Handling call
0709 13:05:24.389143 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:24.718190 (+329047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:24.718244 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:24.718879 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 13:05:24.774047 (+ 55168us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:24.774088 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.778807 (+  4719us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:25.017909 (+239102us) raft_consensus.cc:1685] finished
0709 13:05:25.017932 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:25.018164 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19747,"raft.queue_time_us":11046,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":2025,"thread_start_us":288082,"threads_started":2,"wal-append.queue_time_us":150105}
I20260709 13:05:25.125751 29146 raft_consensus.cc:1217] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 7.8->[8.9-11.10]   Dedup: 8.9->[11.10-11.10]
I20260709 13:05:25.126278 29146 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: [8.9-11.10]
I20260709 13:05:25.127070 29145 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:05:25.094462   369 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:25.088738 29148 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 13:05:25.085696 29140 rpcz_store.cc:269] 0709 13:05:23.704266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.820398 (+116132us) service_pool.cc:224] Handling call
0709 13:05:24.983623 (+1163225us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54001bd5d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:25.025208 (+ 41585us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875453824}
I20260709 13:05:25.098831 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:25.098904 29143 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 8: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:25.100101 29073 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:25.101150 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:25.103039 29139 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:25.099148 29142 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:25.105316 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:25.092136 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.085088 32354 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:25.104636 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:25.164208 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.130061 29141 raft_consensus.cc:2410] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:05:25.173425 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2278) took 1302 ms. Trace:
I20260709 13:05:25.175807 29071 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:25.176877 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2279) took 1297 ms. Trace:
I20260709 13:05:25.127837 29146 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:25.173862 29142 rpcz_store.cc:276] 0709 13:05:23.871506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.098325 (+1226819us) service_pool.cc:224] Handling call
0709 13:05:25.098825 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.173096 (+ 74271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.173361 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:25.177204 29139 rpcz_store.cc:276] 0709 13:05:23.879843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.099474 (+1219631us) service_pool.cc:224] Handling call
0709 13:05:25.099859 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.176714 (+ 76855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.176839 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:25.192128   582 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:25.192744   582 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:25.193782   582 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:25.195767   689 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.004s
I20260709 13:05:25.208920   611 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:25.210153   467 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:25.210335   609 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:25.210618   467 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:25.211730   609 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:25.212198   609 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:25.211834   467 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:25.213353   467 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:25.213253   609 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:25.214373 29141 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:25.214195   641 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:25.214687   609 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:25.191730 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2176) took 3437 ms. Trace:
I20260709 13:05:25.218588 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:25.229140 29141 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:05:25.219300 29215 rpcz_store.cc:276] 0709 13:05:21.754404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.882046 (+127642us) service_pool.cc:224] Handling call
0709 13:05:24.971457 (+3089411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.100002 (+128545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.191509 (+ 91507us) spinlock_profiling.cc:243] Waited 3.05 s on lock 0x7b54000799d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:25.191642 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7019800832}
I20260709 13:05:25.172737 32357 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:25.236325 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:25.215801   437 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:25.240613 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 141767us
W20260709 13:05:25.244011 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.244194 29141 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:25.248239 29143 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:25.254281 29139 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:25.240469   611 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:25.257577 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 182745us
I20260709 13:05:25.262993   700 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:25.264709   405 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:25.268262   318 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:05:25.275666   405 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:25.276046 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.245913   584 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:25.280112   453 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:25.271395 28967 catalog_manager.cc:5697] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 4 to 7, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 7 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:25.282205 28967 catalog_manager.cc:5697] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 4 to 6, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 6 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:05:25.284479   690 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.268s	user 0.002s	sys 0.000s
I20260709 13:05:25.285167 28967 catalog_manager.cc:5697] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:25.289330 29142 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:25.128808   615 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:25.293802 29141 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:05:25.295586 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.296124 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.297175   584 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:25.301669   584 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:25.304314   388 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:25.305276   585 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:25.305640 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.306229 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.306038   585 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:25.306658 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.302109   694 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:05:25.240196   525 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 13:05:25.308887 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 216782us
W20260709 13:05:25.288763   617 consensus_meta.cc:287] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.004s
I20260709 13:05:25.259953   611 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:25.311967 29188 leader_election.cc:336] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.747s (SENT)
I20260709 13:05:25.288586   452 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:25.316802 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 212375us
I20260709 13:05:25.320847 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:25.287588 28967 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 9 to 10, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 10 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:25.305657   388 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:05:25.338524 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.364764 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.319866   611 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:25.370805 29143 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:25.313050   405 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 13:05:25.382961 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.337551   388 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:25.373425   585 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:25.385100 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.385463 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 288902us
W20260709 13:05:25.386215 29039 leader_election.cc:336] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.609s (SENT)
I20260709 13:05:25.322224   618 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:25.386658 29064 raft_consensus.cc:1217] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Deduplicated request from leader. Original: 5.5->[7.6-13.9]   Dedup: 9.7->[11.8-13.9]
W20260709 13:05:25.387200 29039 connection.cc:256] RPC call timeout handler was delayed by -1.1378366710005139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.296s for a response
I20260709 13:05:25.387351 29064 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: [7.6-13.9]
I20260709 13:05:25.387045   618 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:25.387696 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.392892 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70412.5us
W20260709 13:05:25.390785 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 93714.5us
I20260709 13:05:25.389247 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:25.391841 29186 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:25.390918 32586 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:25.313335   318 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 13:05:25.389740 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.388161 32357 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:25.321060   452 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:25.405876 29039 connection.cc:256] RPC call timeout handler was delayed by -1.0840192110008502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4127s for a response
I20260709 13:05:25.419085   699 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:05:25.423386   699 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:25.423789   699 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:25.424441 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 167444us
I20260709 13:05:25.424813   699 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:25.413093   437 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:25.427660   437 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.394091 32354 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:25.432322 29222 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:25.430150 29147 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:25.403867   618 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:25.413146   452 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:25.428118   437 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:25.376186   608 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:05:25.428225   641 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:25.460671 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.461627 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.463761 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:25.465065   701 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.182s	user 0.000s	sys 0.002s
I20260709 13:05:25.465597 29215 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:25.466697 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:25.467562 29139 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:25.470317 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 81165.3us
I20260709 13:05:25.471340 29903 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:25.471653 32354 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:05:25.472111 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.472287 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:25.472520 32354 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:25.472929 32354 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:25.441924 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:25.437516   617 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:25.448050   673 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:25.337944   369 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:25.473024 29143 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:25.428205   699 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:25.476054 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.444331 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:25.481222 29073 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:25.384954   388 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:25.508164 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63208us
I20260709 13:05:25.499645   676 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.501394   618 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:25.529940 29039 leader_election.cc:336] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.510s (SENT)
I20260709 13:05:25.537230 29139 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:25.538353 32354 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:25.540027 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:25.540866 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 102469us
I20260709 13:05:25.541169   672 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:25.541738   672 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:25.510633   617 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:25.516330   585 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:25.554571 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 119886us
I20260709 13:05:25.531855   673 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:25.559335 32354 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:25.513584   641 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:05:25.543524   530 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:25.519534   618 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:25.581739 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56131.4us
I20260709 13:05:25.589507 32586 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:25.549271   676 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:25.571564   405 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:25.605213 29039 leader_election.cc:336] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.593s (SENT)
W20260709 13:05:25.623934 29188 leader_election.cc:336] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.061s (SENT)
I20260709 13:05:25.591975   617 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:25.624169 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:25.581111   615 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:25.578747 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 143162us
W20260709 13:05:25.620368 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.624606 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:25.607933 29213 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:05:25.606060   672 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:25.659729 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2153) took 3451 ms. Trace:
I20260709 13:05:25.662124   630 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:05:25.665187 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.668769 32572 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:25.677130 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 600) took 1661 ms. Trace:
I20260709 13:05:25.700781 29222 rpcz_store.cc:276] 0709 13:05:24.015886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.734022 (+718136us) service_pool.cc:224] Handling call
0709 13:05:24.734492 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.436058 (+701566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.676701 (+240643us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b54003a89d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:25.676998 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1604287744}
W20260709 13:05:25.680331 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.705287 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 564) took 1173 ms. Trace:
I20260709 13:05:25.705884 29147 rpcz_store.cc:276] 0709 13:05:24.521760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.828580 (+306820us) service_pool.cc:224] Handling call
0709 13:05:25.695203 (+866623us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b540029de54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:25.695369 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1110921600}
I20260709 13:05:25.691315 29064 rpcz_store.cc:276] 0709 13:05:22.208214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.951227 (+743013us) service_pool.cc:224] Handling call
0709 13:05:22.951616 (+   389us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:05:25.387907 (+2436291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.659204 (+271297us) spinlock_profiling.cc:243] Waited 2.29 s on lock 0x7b54002d3f54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:25.659612 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5268607232}
I20260709 13:05:25.698503 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:25.707497 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.719763 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 135099us
I20260709 13:05:25.717233 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:25.719202 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.585052   318 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 13:05:25.684743   388 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:25.692387   630 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:25.724246 29186 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:25.716526   708 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:05:25.700459 32572 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:05:25.729413 29188 leader_election.cc:336] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.142s (SENT)
I20260709 13:05:25.709158   585 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:25.734440 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.761142 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 155954us
W20260709 13:05:25.761873 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.758520   495 raft_consensus.cc:2764] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:05:25.765988 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:25.768185   714 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 13:05:25.771757   710 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:05:25.773811   453 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.702618   405 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:25.722518   512 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:25.768400   712 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 13:05:25.781045 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.774566   453 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:25.782200 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52827us
I20260709 13:05:25.787225 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 565) took 1062 ms. Trace:
W20260709 13:05:25.800792 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 359870us
I20260709 13:05:25.796922 29186 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:25.725091   700 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
W20260709 13:05:25.809435 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 201311us
W20260709 13:05:25.811622 29039 leader_election.cc:336] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.640s (SENT)
I20260709 13:05:25.792055 29140 rpcz_store.cc:276] 0709 13:05:24.724743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.218273 (+493530us) service_pool.cc:224] Handling call
0709 13:05:25.240288 (+ 22015us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:25.345127 (+104839us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:25.345180 (+    53us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:25.385839 (+ 40659us) log.cc:844] Serialized 78 byte log entry
0709 13:05:25.436126 (+ 50287us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:25.475456 (+ 39330us) log.cc:844] Serialized 14 byte log entry
0709 13:05:25.500722 (+ 25266us) log.cc:844] Serialized 14 byte log entry
0709 13:05:25.500963 (+   241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.501063 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:25.663264 (+162201us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b18004b9280. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20522 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:25.786962 (+123698us) raft_consensus.cc:1685] finished
0709 13:05:25.786996 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:25.787162 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28450,"prepare.queue_time_us":139271,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":4049,"raft.queue_time_us":20913,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":52339,"spinlock_wait_cycles":16350080,"thread_start_us":49014,"threads_started":1,"wal-append.queue_time_us":143586}
W20260709 13:05:25.812883 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.815969 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.816181 29143 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
W20260709 13:05:25.810655 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 272153us
I20260709 13:05:25.732297   615 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:25.834786 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.852020 29188 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.214s (SENT)
W20260709 13:05:25.852805 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.860805 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.865980 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.863611 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:25.815169   512 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.871515 28967 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 4 to 6, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 6 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:25.877204 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 101076us
I20260709 13:05:25.857713   593 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:25.883399 29222 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
W20260709 13:05:25.883293 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.850788   673 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:25.885768 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 558) took 1513 ms. Trace:
W20260709 13:05:25.885938 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.884313 29222 pending_rounds.cc:85] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 8
I20260709 13:05:25.886201 29146 rpcz_store.cc:276] 0709 13:05:24.372132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.189625 (+817493us) service_pool.cc:224] Handling call
0709 13:05:25.250560 (+ 60935us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:25.274233 (+ 23673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:25.274276 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:25.274829 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:05:25.414452 (+139623us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:25.414495 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.414572 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:25.885546 (+470974us) raft_consensus.cc:1685] finished
0709 13:05:25.885569 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:25.885708 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1212,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"thread_start_us":139192,"threads_started":1,"wal-append.queue_time_us":237763}
I20260709 13:05:25.863242   619 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:25.895184 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 90428.3us
I20260709 13:05:25.869563   618 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:25.899653 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.901161 29213 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:25.903257 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:25.903697   716 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
W20260709 13:05:25.903843 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 77664.2us
I20260709 13:05:25.904246 29219 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:25.905192 29221 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:05:25.905438 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.906319 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.910683   582 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.911279   582 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:25.912312   623 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.912631 29219 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:25.912938   623 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:25.914453   609 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.683s	user 0.006s	sys 0.006s
I20260709 13:05:25.915310 29216 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:25.916224 32586 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:05:25.917215   467 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.703s	user 0.014s	sys 0.000s
I20260709 13:05:25.887990 29222 pending_rounds.cc:107] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:05:25.920153 29222 raft_consensus.cc:2889] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:25.883793   348 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:25.942510 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:25.954639 29039 leader_election.cc:336] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.672s (SENT)
I20260709 13:05:25.892083   512 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:25.966339 29213 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:25.969233 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:25.974334 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 110834us
I20260709 13:05:25.894567 32572 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:25.950909   711 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:05:25.975170   495 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:25.987778 29188 leader_election.cc:336] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.773s (SENT)
I20260709 13:05:25.990190 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:25.992389 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 115564us
I20260709 13:05:25.988674 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:25.986925 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
W20260709 13:05:26.001747 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 98507.1us
W20260709 13:05:25.994036   611 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.591s	user 0.011s	sys 0.002s
I20260709 13:05:25.980636   619 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.009467 29222 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:05:26.009773 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:26.008265   611 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.012650 29220 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:26.013757 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:26.014267   728 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:05:26.014489   584 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:26.015780   348 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:26.016043 28969 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 272) took 28646 ms (client timeout 15000 ms). Trace:
I20260709 13:05:26.016461   348 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.016769 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:26.019361   348 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.021880 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:26.022781 29221 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
W20260709 13:05:26.023474 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 74264us
I20260709 13:05:25.988489   405 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:26.008512   369 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:26.024974 29213 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:05:26.030280 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 51987.2us
I20260709 13:05:26.033958   348 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:26.048629 28967 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:26.051051   673 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:26.031424 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:26.057741 28967 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 8, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 8 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:26.065531   731 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:05:26.068688   726 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:05:26.071094   512 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.035364   619 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.075301   619 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.035475   618 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:26.075733 29039 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.687s (SENT)
I20260709 13:05:26.076267 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:26.065896   721 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:26.077427   721 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:05:26.077747 29039 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.733s (SENT)
I20260709 13:05:26.078176 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:26.078389   721 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:26.082129   615 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.076507 32572 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:26.084328 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.067708 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:26.084883 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:26.092694 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:26.093575 29113 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:26.093964 29217 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:26.094826 29113 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:26.095115   348 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:26.095731   348 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.096231   348 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:26.095752   723 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:26.100675   723 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.101130   723 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:26.104429   676 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:26.076295   618 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:26.016525 28969 rpcz_store.cc:269] 0709 13:04:57.333416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.388113 (+ 54697us) service_pool.cc:224] Handling call
0709 13:05:25.975940 (+28587827us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:15.081251 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:15.081989 (+   738us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:15.082050 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:05:15.082075 (+    25us) tablet.cc:662] Decoding operations
0709 13:05:15.109532 (+ 27457us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:15.135152 (+ 25620us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:15.137620 (+  2468us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:05:15.221491 (+ 83871us) tablet.cc:701] Row locks acquired
0709 13:05:15.221548 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:05:15.221777 (+   229us) write_op.cc:270] Start()
0709 13:05:15.221949 (+   172us) write_op.cc:276] Timestamp: P: 1783602315221686 usec, L: 0
0709 13:05:15.221969 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:05:15.222793 (+   824us) log.cc:844] Serialized 7554 byte log entry
0709 13:05:15.383512 (+160719us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b18002d8320. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f7836a8a190 00007f7836a83ae7 00007f7836a82dd9 00007f7836a8d930
0709 13:05:15.668362 (+284850us) op_driver.cc:464] REPLICATION: finished
0709 13:05:16.342261 (+673899us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f7833a030ac 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f783a543ef0
0709 13:05:16.565072 (+222811us) write_op.cc:301] APPLY: starting
0709 13:05:16.588936 (+ 23864us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:16.611709 (+ 22773us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:16.611750 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:16.746824 (+135074us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:16.747249 (+   425us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0709 13:05:16.747302 (+    53us) write_op.cc:312] APPLY: finished
0709 13:05:16.747795 (+   493us) log.cc:844] Serialized 132 byte log entry
0709 13:05:17.253921 (+506126us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:17.357819 (+103898us) write_op.cc:454] Released schema lock
0709 13:05:17.357929 (+   110us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:17.731271 (+373342us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:17.731303 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5805862,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":849734,"apply.run_cpu_time_us":63863,"apply.run_wall_time_us":1189289,"num_ops":19,"prepare.queue_time_us":265856,"prepare.run_cpu_time_us":39900,"prepare.run_wall_time_us":252057,"raft.queue_time_us":169599,"raft.run_cpu_time_us":22825,"raft.run_wall_time_us":1051698,"replication_time_us":353603,"spinlock_wait_cycles":78113536,"thread_start_us":1546023,"threads_started":5,"wal-append.queue_time_us":599905,"wal-append.run_cpu_time_us":45474,"wal-append.run_wall_time_us":2650851}]]}
I20260709 13:05:26.119391 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:26.120281 29214 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
W20260709 13:05:26.122704   452 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.644s	user 0.008s	sys 0.004s
I20260709 13:05:26.123154   452 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:26.123495   618 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:26.131111 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:26.132026 29221 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:26.133493 29216 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:05:26.135291 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.137006 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:26.086370   615 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:26.141477 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63788.1us
I20260709 13:05:26.139670 29217 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:26.144512 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:26.146030 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:26.149132   452 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:26.064181   369 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:26.154100 29222 raft_consensus.cc:2393] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:26.125980   619 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:26.163259 29188 leader_election.cc:336] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.091s (SENT)
W20260709 13:05:26.168145 29039 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.798s (SENT)
I20260709 13:05:26.169188 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:26.156136 32357 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:26.155772   611 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:26.178117 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 119679us
I20260709 13:05:26.182314 29221 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:05:26.181439 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 130844us
I20260709 13:05:26.174424 32357 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.197023   737 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:26.197914   453 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.198356 29871 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:26.201129 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:26.202119 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:26.204023 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:26.197925   737 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:26.205024   495 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:26.205730   737 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:26.206581   697 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:05:26.207286   697 consensus_queue.cc:1243] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:26.213450   727 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 13:05:26.217262 28967 catalog_manager.cc:5697] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.218998 28967 catalog_manager.cc:5697] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 2 to 7, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 7 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:05:26.220750   738 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:05:26.220932 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:26.222252 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:26.222826 29906 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:26.224336 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:26.224960 29908 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:26.210801   453 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:26.228618 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:26.230767   720 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:26.234946 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:26.196612 32357 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:26.242640 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 157672us
W20260709 13:05:26.239269   437 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.003s
W20260709 13:05:26.251256 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 88297.1us
I20260709 13:05:26.252358 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:26.241453   727 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:26.258925 29903 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:26.264279   717 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.556s	user 0.006s	sys 0.000s
I20260709 13:05:26.241679   512 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:05:26.267338   512 consensus_queue.cc:1243] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:05:26.272503 29039 leader_election.cc:336] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.821s (SENT)
I20260709 13:05:26.273041 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:26.274201 29039 leader_election.cc:336] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.831s (SENT)
W20260709 13:05:26.275123 29039 leader_election.cc:336] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.843s (SENT)
I20260709 13:05:26.276382   618 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:26.249362   348 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.263870 29910 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:26.277863 29907 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:26.281466   720 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:26.282133   720 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:26.258810   672 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:05:26.283058   720 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:26.254191   437 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:26.284936 29039 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.900s (SENT)
I20260709 13:05:26.285475 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:26.287647 29906 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:26.259426   727 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.295424 29217 pending_rounds.cc:85] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 6
I20260709 13:05:26.295895 29217 pending_rounds.cc:107] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:05:26.296236 29217 raft_consensus.cc:2889] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:26.293532   437 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:26.304798 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.305769 29145 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:26.294350   727 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:26.298257   618 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:26.314688 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:26.315024   630 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.315800 29144 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:26.316273   630 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:26.320951 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.321674 29140 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:26.322137   582 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:26.324981 29906 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:26.334004 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 49107.9us
I20260709 13:05:26.335577 29906 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:26.335460 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:26.344760 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:26.346858 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 91533.8us
I20260709 13:05:26.348978 29908 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:26.352207 29909 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:26.352679 29909 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:26.353499 29909 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:26.354708 29909 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:26.367578 29909 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:26.373235 29908 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
W20260709 13:05:26.380695   699 consensus_meta.cc:287] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.867s	user 0.013s	sys 0.000s
I20260709 13:05:26.384316 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.384490   348 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.396667 29072 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:26.399817 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:26.400617 29064 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:26.389567   754 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:05:26.407451 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:26.413785 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 500626us
W20260709 13:05:26.412439   740 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
W20260709 13:05:26.419154 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 72690.1us
W20260709 13:05:26.420941 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.421612 29908 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: []
W20260709 13:05:26.425848 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.430994   723 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:26.432474 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.433626 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:26.435242 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:26.435876 29908 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:26.436157   623 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.440305 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.406281   727 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.441789 29072 raft_consensus.cc:2410] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:05:26.446080   369 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:26.446868 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.453274   723 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:26.454056 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 290830us
I20260709 13:05:26.454330 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:26.455072   740 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:26.455574   740 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:26.456403   467 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:26.468252 29039 leader_election.cc:336] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.938s (SENT)
W20260709 13:05:26.472028 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.477435 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:26.478157 29215 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:26.478408 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:26.479030 29220 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:26.479969 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:26.480300 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:26.480538 29220 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:26.482678 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:26.484368   584 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.484961   584 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.485910 29221 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:05:26.487231   584 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:26.483843 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:26.489378 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:26.487989 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:26.490751 29219 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:26.491735 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:26.494800 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.495229 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.495658 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.497211 29072 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:26.497946   750 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:26.498700 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.499346 29066 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
W20260709 13:05:26.500023   755 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:05:26.504135   585 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:26.508710 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:26.509682   593 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:05:26.510972   727 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.511620   585 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:26.512126   585 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:26.488381   676 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:26.514103 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.508296 29220 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:26.519629 29221 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: [4.3-10.5]
W20260709 13:05:26.522017 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 58361.8us
I20260709 13:05:26.487993   525 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:05:26.525090 29064 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:26.494925   369 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:26.523695   676 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.535104 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:26.456545   740 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:26.536829 29039 leader_election.cc:336] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.946s (SENT)
I20260709 13:05:26.531950   750 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:05:26.547797 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:26.513008   585 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:26.531230   617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.004s	sys 0.007s
I20260709 13:05:26.610662   623 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:26.603155 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.663005 29064 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:26.633482   700 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.604025 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:26.561558 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.505863   711 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 13:05:26.573684 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.636029 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:26.634277 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 88695.9us
W20260709 13:05:26.638510 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 90759.8us
I20260709 13:05:26.548877   369 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:26.680644 29215 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:26.682587 29214 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: [10.8-14.9]
I20260709 13:05:26.507191   725 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.136s
I20260709 13:05:26.633052 29072 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
W20260709 13:05:26.663895 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 109437us
I20260709 13:05:26.639564   672 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:26.662346   617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.006s	sys 0.025s
I20260709 13:05:26.556550   630 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 13:05:26.670483   623 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:26.673012 29113 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:26.675155 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:26.676257 29220 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:26.680780   369 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:26.680912 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:26.690454 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 86535.2us
I20260709 13:05:26.690814 29219 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:26.605993   727 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.688392 32357 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.681170   700 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.701186 29871 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:26.705650 32357 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.711684 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:26.694245   585 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:26.725616 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:26.697741   618 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:26.697525   348 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.698038   723 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.678313   388 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:05:26.731423 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:26.715669   542 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:26.705570   512 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.716657   745 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:26.739049 29903 raft_consensus.cc:2376] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8. Re-sending same reply.
I20260709 13:05:26.741585 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:26.743819 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 211624us
I20260709 13:05:26.748209 29213 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:26.717449   720 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:26.744803 29910 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:26.749426 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:26.697350   727 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.745630 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:26.754233 29906 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:26.705652   756 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:05:26.757198   765 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:05:26.768541 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:26.757597   623 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:26.758701   771 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:05:26.750200   720 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.774312   720 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:26.775173 29039 leader_election.cc:336] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.987s (SENT)
I20260709 13:05:26.740432   348 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:26.793359 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 47775.2us
I20260709 13:05:26.751240   615 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:05:26.799693 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.810353 29072 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:26.802765 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.768043   618 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:26.808208 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.811570 29068 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:26.813174 29066 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:05:26.750908   512 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.813513 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:26.815209 29072 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 7: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:26.815294 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.816924 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.822305 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.823503   512 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.823774   700 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.812686   623 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:05:26.827999   615 consensus_queue.cc:1243] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:26.828652   615 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:26.828976   615 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:26.829906   615 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:26.831194   615 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:26.834512   542 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:26.835562   542 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.796042   676 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:26.840153   699 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.840394   700 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.840812   699 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:26.840946   700 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.842973   700 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:26.843519 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.800037   697 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:26.844033   699 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:26.844158   697 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.844597   697 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:26.845839   348 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.823506 29071 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:26.847568   618 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:26.850140 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 94440.2us
I20260709 13:05:26.799983   672 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:26.767714   723 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.822398 29065 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:05:26.861698 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.863134 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:26.863520 29064 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: [3.3-12.7]
I20260709 13:05:26.885848 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:26.886559 29066 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:26.887411   723 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.895395 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:26.896184 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.896804   723 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.897141 29069 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:26.897401   723 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.898506 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.899138 29067 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:26.899418 32354 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:26.899809   348 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.900447 29143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79cdfc6a725145348b70034422ef477b"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42040
I20260709 13:05:26.908329 29143 raft_consensus.cc:606] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Received request to transfer leadership
I20260709 13:05:26.907955 32354 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.908720 29143 raft_consensus.cc:614] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:26.901626 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.901801 29065 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:26.901978 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.910692 29140 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:26.902900 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.896226 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.903590 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.903756 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.913152 29147 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:26.904383 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.904481 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:26.905227 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 165320us
I20260709 13:05:26.904474 29213 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:26.912182 29145 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:26.909610 29144 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:26.861554   672 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.912323 29141 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:26.918500 32354 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.919644 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:26.925129 29039 leader_election.cc:336] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.069s (SENT)
I20260709 13:05:26.925791 29071 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:26.925801 29146 raft_consensus.cc:2368] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:26.927071 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:26.927860 29071 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:26.899720   723 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.931603 29142 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:05:26.936805 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.937522 29141 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:26.923027   672 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:26.948220 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.948961 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:26.950831 29139 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:26.951275 29908 raft_consensus.cc:2410] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:26.953187 29037 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:26.956259 29067 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:05:26.958460 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.960978 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:26.962075 29071 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:26.963614   777 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:05:26.964305 29145 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:26.966626   727 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:26.968253 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48222.4us
W20260709 13:05:26.957901 29188 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.225s (SENT)
I20260709 13:05:26.971422 29188 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:26.974997 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:26.919534   348 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:26.949373   625 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:05:26.979310   348 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:26.980597 29903 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:26.982928   756 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:26.983503   756 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:26.987139 29070 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:26.987424 29067 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:26.988765 29871 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:26.990058   388 raft_consensus.cc:2764] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:05:26.990447 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:26.991658   721 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:26.992184   721 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:26.992779   608 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:26.993213   608 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:26.993079   721 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:26.994076   608 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:26.995326   608 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:26.997129   630 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:05:27.000634 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.004315 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:26.996989   756 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:27.011149 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.016256 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56121.3us
I20260709 13:05:27.017129 29143 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:27.022149 29066 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:27.024819 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:27.025758 29071 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:27.031437 29142 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:05:27.036265 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:26.962893   723 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.048030 29070 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 13:05:27.049247   785 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:05:27.050305 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 104453us
I20260709 13:05:27.003063   720 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.041191   495 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:27.053342   781 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:05:27.055272 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:27.056170 29139 raft_consensus.cc:2410] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:05:27.036873   782 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.092s	user 0.004s	sys 0.000s
W20260709 13:05:27.065994 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.078547 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 87105.8us
I20260709 13:05:27.079517 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:27.080520 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.041383   745 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:27.089129 29139 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:05:27.061231   625 consensus_queue.cc:1243] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:05:27.101405 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.117334 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:27.026173   740 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:05:27.126812 29139 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:27.062821   723 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.094767   745 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:27.151428 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 111003us
I20260709 13:05:27.157713 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.157956 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:27.158705 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:27.159161 29139 raft_consensus.cc:2410] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:05:27.131721   495 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:27.165072 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.166185 29221 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:27.167676 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.151369   609 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:27.131247   388 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:05:27.173970 29113 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:27.121500   697 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.170711 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:27.181233 29146 raft_consensus.cc:2368] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:27.165010   495 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:27.186666 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:27.188318 32354 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:27.188973 32354 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:27.170691   609 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.201920 29139 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:27.206773 32354 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.216549 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:27.202654   787 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.226s	user 0.003s	sys 0.000s
I20260709 13:05:27.199975   495 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:27.237733 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.203835   609 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:27.202541   348 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:27.263129 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.217542   582 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:27.276152 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.277321 29139 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:27.210772   623 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:05:27.196031   745 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:27.310428 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.302742 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 156707us
W20260709 13:05:27.312319 29039 connection.cc:256] RPC call timeout handler was delayed by -1.6094751910004561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2605s for a response
W20260709 13:05:27.316656 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.317549 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 158913us
I20260709 13:05:27.302122   618 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:27.295586   348 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:27.322566 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:27.301676   720 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:27.355549 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:27.356153 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.356382 29142 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:27.315677 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:27.322769 29139 raft_consensus.cc:1217] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 7.8->[9.9-9.9]   Dedup: 9.9->[]
W20260709 13:05:27.325582 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.298194   750 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.341869 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:27.364771 29146 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:27.295086   337 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.186s
I20260709 13:05:27.301115   743 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.188s
I20260709 13:05:27.301545   582 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.349843   745 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.318233   512 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:27.359699   786 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:05:27.339044   617 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.347434   369 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.646s	user 0.007s	sys 0.005s
I20260709 13:05:27.392845 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.326207   723 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:27.369458 29139 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:05:27.353682 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.393734 29143 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:27.353611 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:27.372668 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 57093us
I20260709 13:05:27.342970 29070 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:27.344465   737 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:05:27.375010 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:27.376317   792 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:05:27.377290   799 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:27.377748   512 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.380472 29907 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:27.377311   789 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:05:27.373461   750 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.384110   791 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.158s
W20260709 13:05:27.388235   795 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:05:27.388940   697 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:27.386718 29039 connection.cc:256] RPC call timeout handler was delayed by -1.0373136980006166s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31970000000000004s for a response
I20260709 13:05:27.342419   794 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:27.392614   623 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.422492 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:27.432341 29142 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:27.435786 29221 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:27.438529 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.434376   794 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:27.439595   791 consensus_queue.cc:1243] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:27.440181   791 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:27.440512   791 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:27.440397   623 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.402652   369 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.444090 29141 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:27.444444   337 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:27.425124   593 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:05:27.413398   801 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:05:27.445050   337 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:27.446204   793 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.098s	user 0.001s	sys 0.001s
I20260709 13:05:27.446053   337 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.409762   617 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.447722   369 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:27.449309   617 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:27.451260 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:27.453047   790 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 13:05:27.427639   615 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.596s	user 0.013s	sys 0.000s
I20260709 13:05:27.421368 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:27.420234 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 66688.5us
I20260709 13:05:27.423364 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.483984 29143 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:27.421525 29140 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:05:27.485004 29067 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:05:27.485389 29067 pending_rounds.cc:85] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 10
I20260709 13:05:27.422847   672 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.485627 29067 pending_rounds.cc:107] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:05:27.485883 29067 raft_consensus.cc:2889] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:27.486920   609 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.409829   348 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.457280   495 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.458865   512 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:27.434991   585 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.703s	user 0.005s	sys 0.008s
I20260709 13:05:27.460510 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:27.462001 29146 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:27.466305   582 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:27.528090 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.528843 29146 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:27.530211 29146 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:05:27.532344 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.467823   750 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:27.537818 29216 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:27.538268 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:27.401928 29144 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:27.475279   697 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.476145   743 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:27.477231 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:27.478677   713 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:27.480151   794 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.483397   623 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:27.485136 29140 pending_rounds.cc:85] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 5
I20260709 13:05:27.553054 29140 pending_rounds.cc:107] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:05:27.545488 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.553336 29140 raft_consensus.cc:2889] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:27.487287   672 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:27.552507   608 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.008s
I20260709 13:05:27.554987   608 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:27.555097 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 78003.2us
I20260709 13:05:27.544289   723 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:27.555673   495 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:27.487860 29065 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:27.403242   625 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:05:27.557657   743 consensus_queue.cc:1243] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:27.489084 29147 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:27.489662 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.491120 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.492736 29070 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:27.512187 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:27.512914 29188 leader_election.cc:336] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.605s (SENT)
I20260709 13:05:27.513250   737 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:27.515480 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:27.517035 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.518971 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.471304   791 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.519707 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.535494 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.578855   791 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:27.546615 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.581072 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.582839 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:27.551252 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:27.584211   609 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:27.584393 29068 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:27.454522 29220 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:27.585666 29220 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:27.567005 29215 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:27.568421 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:27.588727   737 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:27.589166   737 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.589303 29215 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:05:27.589591   737 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.586881 29220 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.596310 29220 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.600375 29871 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:27.601266   720 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:27.601863   720 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:27.602916   720 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.594447 29215 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:27.612959 29070 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:27.614645   388 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:27.615861   388 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:27.616408   388 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:27.618009   388 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.620378   388 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:27.625289 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:27.625743   805 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:05:27.625963 29219 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:27.559944   794 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:27.644238 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:27.645648 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:27.647029 29113 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:27.652505   617 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:27.653798   369 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:27.654093 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 141201us
I20260709 13:05:27.654439   369 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.655045   369 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:27.657235   318 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:27.567585   713 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.665053 29220 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:05:27.651266 32586 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:27.666668 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.669206 29219 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:27.546456   697 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.647359   737 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.669904   699 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:05:27.676891   615 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.677268 29070 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:27.677702   623 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:27.678506 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:27.572189   672 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 13:05:27.684000 29139 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:05:27.694120 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.698033 29145 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:05:27.688390 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 265968us
I20260709 13:05:27.696496 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.698900 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:27.699369 29139 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:27.709022 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.709892 29139 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:05:27.692009   584 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:05:27.687954   699 consensus_queue.cc:1243] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:27.689491   796 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:27.673156   713 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:27.714182 29071 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.021s	sys 0.006s
W20260709 13:05:27.735877 29071 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.021s	sys 0.006s
I20260709 13:05:27.731264 29139 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:27.732923 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.694857   776 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:05:27.711663 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.761250 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.761823 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.765543 29066 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.016s	sys 0.033s
W20260709 13:05:27.766172 29066 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.016s	sys 0.033s
I20260709 13:05:27.718981   743 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:27.689606   615 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:27.777974 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:27.777223   787 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:05:27.713490   750 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:05:27.760067   776 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:27.800053 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:27.800055 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:27.777483   625 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.801930 29069 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:27.779892   777 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.010s	sys 0.022s
W20260709 13:05:27.817338 29188 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.604s (SENT)
I20260709 13:05:27.780561   495 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.801852   799 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:27.780224   743 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:27.829934 29140 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:05:27.831836 29140 pending_rounds.cc:85] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 6
I20260709 13:05:27.832240 29140 pending_rounds.cc:107] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:05:27.832536 29140 raft_consensus.cc:2889] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:27.842558 32354 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:27.843187 32354 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:27.845566 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:27.831156   777 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.011s	sys 0.022s
I20260709 13:05:27.803308   512 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:27.848937 29188 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.900s (SENT)
I20260709 13:05:27.852062 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:27.852943 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.855710 29148 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:27.839144   743 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:27.866341   779 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.020s	sys 0.011s
W20260709 13:05:27.866914   779 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.020s	sys 0.011s
W20260709 13:05:27.862799 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 80608.2us
I20260709 13:05:27.871191   713 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:27.837234   495 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.874449   495 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:27.874642   608 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:05:27.877051 29188 leader_election.cc:336] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.940s (SENT)
I20260709 13:05:27.867539 32354 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:27.877648 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:27.880074 29147 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:27.875528   608 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:27.883355 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:27.834259   625 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:27.889550 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:27.902773 29069 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:27.893921 29903 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:27.905553 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:27.905373 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 256223us
W20260709 13:05:27.886193 29039 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.759s (SENT) [suppressed 46 similar messages]
W20260709 13:05:27.923391 29188 leader_election.cc:336] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.999s (SENT)
I20260709 13:05:27.924170 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:27.875300   799 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:27.923903 29039 leader_election.cc:336] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.772s (SENT)
I20260709 13:05:27.937985 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:27.939457 29069 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:05:27.883250   585 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.953537 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:27.955713 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:27.883296   813 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:27.876087   630 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:05:27.971740 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.882828   495 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:27.888195   625 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:27.992100 29140 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
W20260709 13:05:27.997038 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 90040.7us
W20260709 13:05:27.997939 29188 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.018s (SENT)
W20260709 13:05:27.999032 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.949043   806 raft_consensus.cc:3055] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:28.003274 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:28.007849 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 83983.5us
I20260709 13:05:28.009330 29069 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:28.037673 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:27.993010   813 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:28.041378 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.993808   756 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:28.046588 29140 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:28.005790   337 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:28.100741   807 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
I20260709 13:05:28.021519   818 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.991544   585 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:28.070513   815 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:05:28.075906   776 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:28.005326   806 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:28.080628   814 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
W20260709 13:05:28.086542 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 88624.6us
W20260709 13:05:28.096583 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.096859   615 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:28.011310   625 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:28.100175   618 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:28.104315 29148 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
W20260709 13:05:28.056272 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 66732.1us
I20260709 13:05:28.109284 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:28.116699   821 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:05:28.069293   797 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.718s	user 0.003s	sys 0.000s
I20260709 13:05:28.123737 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:28.115760 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:28.260299 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.333138 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.335744 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.334087 29140 raft_consensus.cc:2376] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12. Re-sending same reply.
I20260709 13:05:28.345129 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:28.345564 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:28.346081 29144 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:28.335237   625 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:28.123265   811 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 13:05:28.119451   808 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:05:28.117462   723 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:28.366299 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:28.369886 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:28.370314 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.171685   756 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.401068 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.402176 29148 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
W20260709 13:05:28.403194 29139 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.018s	sys 0.039s
W20260709 13:05:28.403692 29139 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.018s	sys 0.039s
I20260709 13:05:28.405265 29070 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:28.151087   818 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:28.418125 29145 consensus_meta.cc:287] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.617s	user 0.008s	sys 0.004s
I20260709 13:05:28.369632   776 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:28.373136   723 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:28.335098   618 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:28.370620   615 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.416519 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:28.429576 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:28.421651 29145 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:28.429545 29144 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:28.434145 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.434760 29146 raft_consensus.cc:2376] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10. Re-sending same reply.
I20260709 13:05:28.434959 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.435086 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.435611 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:28.436347 29146 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:28.365844   829 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.374769   676 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.444893 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 473) took 1397 ms. Trace:
W20260709 13:05:28.429690   794 consensus_meta.cc:287] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.763s	user 0.009s	sys 0.003s
I20260709 13:05:28.446780 29073 rpcz_store.cc:276] 0709 13:05:27.039499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.039620 (+   121us) service_pool.cc:224] Handling call
0709 13:05:27.039901 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:27.196518 (+156617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:27.196548 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:27.250094 (+ 53546us) log.cc:844] Serialized 26 byte log entry
0709 13:05:27.733609 (+483515us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:27.733668 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.733741 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:28.436990 (+703249us) raft_consensus.cc:1685] finished
0709 13:05:28.437016 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:28.437158 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":75467,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":19774,"thread_start_us":573126,"threads_started":2,"wal-append.queue_time_us":588196}
I20260709 13:05:28.213434   813 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:28.436621   837 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.476682 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.397006   756 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:28.500332 29146 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:28.503022 29144 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:28.438810   615 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:28.515509 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.516400 29148 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:28.448989   829 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:28.502282   820 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:05:28.520964 29146 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:28.520525 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.523880 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.433880   806 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:28.456984   676 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:28.532972 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:28.535504 29141 raft_consensus.cc:2376] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12. Re-sending same reply.
I20260709 13:05:28.523680 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.537914 29148 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:28.538467 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:28.539831 29141 raft_consensus.cc:2376] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10. Re-sending same reply.
I20260709 13:05:28.541528 29141 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:28.539098 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.542527 29146 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
W20260709 13:05:28.542991 29188 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.140s (SENT)
I20260709 13:05:28.542405 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:28.543594 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:28.544127 29144 raft_consensus.cc:2376] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8. Re-sending same reply.
W20260709 13:05:28.536958   798 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.160s	user 0.000s	sys 0.006s
I20260709 13:05:28.545957 29141 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:28.545995 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.546706 29144 raft_consensus.cc:2376] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11. Re-sending same reply.
I20260709 13:05:28.489569   837 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:28.551487 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:28.557358   337 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:28.558408 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.559604 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:28.560830 29146 raft_consensus.cc:2368] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10.
I20260709 13:05:28.560709 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.563690 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.564177 29141 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:28.565586 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.566291 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:28.567689 29146 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:28.565984 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:28.568866   833 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:05:28.578547 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:28.572702   818 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:28.574179 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:28.574865 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:28.561408   832 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:05:28.576972 29144 raft_consensus.cc:2410] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:28.569295   835 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.550570   806 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:28.560313   625 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:28.955649 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:28.691898   841 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:05:28.720307   756 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:28.695698   369 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.040s	user 0.010s	sys 0.004s
I20260709 13:05:28.722031   676 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:28.857712   843 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:05:28.654405   618 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:28.650568   585 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:28.599241   791 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.017s	user 0.009s	sys 0.003s
W20260709 13:05:28.579079   388 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.958s	user 0.014s	sys 0.000s
I20260709 13:05:28.653483   723 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.062461   723 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:29.064033   835 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.024058   829 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:29.066349   740 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:29.068339   743 consensus_meta.cc:287] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.187s	user 0.012s	sys 0.002s
I20260709 13:05:28.704149   452 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:29.049819   369 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.087119   835 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.088975   737 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:29.089505   740 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.068831   837 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:29.093475   369 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:28.653054   615 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:29.084808   743 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.091455   452 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.112324   743 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:29.113953   737 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:29.116519   737 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.120019   452 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:29.120023   737 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:29.122798   776 consensus_meta.cc:287] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.690s	user 0.010s	sys 0.000s
I20260709 13:05:29.123420 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2047) took 1600 ms. Trace:
I20260709 13:05:29.125880 29147 rpcz_store.cc:276] 0709 13:05:27.522928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.581186 (+ 58258us) service_pool.cc:224] Handling call
0709 13:05:27.581523 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:27.882582 (+301059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.123124 (+1240542us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54002e8bd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:29.123320 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645536512}
I20260709 13:05:29.127365 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.128602 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.129220 29147 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:29.131906 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.133077 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.133585 29147 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:29.134240 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.134634   806 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:29.134785 29147 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:29.135248   806 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.135810 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.136688 29147 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:29.137816   806 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.138114 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.138420   697 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.138947 29147 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:05:29.136080 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.139822 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:29.140522 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.140960 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.141628 29147 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:29.142108   776 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:29.142339   615 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.142449 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.143286 29147 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:29.142803   776 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.144526   740 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:29.144526 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:29.145357   776 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.145541 29147 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:29.146095   776 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.146661   776 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.142994   615 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.149664   615 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:29.151585   776 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:29.156278 29147 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:29.157351 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.158437 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.159106 29147 raft_consensus.cc:2368] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11.
I20260709 13:05:29.160197 29147 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:29.161322 29147 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:29.162329 29147 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:29.163445 29147 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:05:29.163743 29147 pending_rounds.cc:85] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 4
I20260709 13:05:29.163978 29147 pending_rounds.cc:107] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:05:29.164227 29147 raft_consensus.cc:2889] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:29.167155 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.168445 29908 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:05:29.168700 29909 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:29.169945 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:29.170982 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:29.171728 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:29.172127 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:29.172825 29903 raft_consensus.cc:2410] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:29.171661 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:29.173442 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.173938 29902 raft_consensus.cc:2410] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:05:29.171622 29906 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:29.174710 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:29.176575 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:29.181062   776 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:29.181690   776 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.182188   776 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:29.182256   835 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:29.184437   623 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.185199   623 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.189286   623 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.174124   806 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 13:05:28.595517 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52553.4us
I20260709 13:05:29.210083 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:29.211848 29188 leader_election.cc:336] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.148s (SENT)
I20260709 13:05:29.212319 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.213131   584 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:29.214202 29073 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:29.215459   825 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.216310   825 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:28.648985 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 481) took 1045 ms. Trace:
I20260709 13:05:29.218061 29068 rpcz_store.cc:276] 0709 13:05:27.603474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.603633 (+   159us) service_pool.cc:224] Handling call
0709 13:05:27.604070 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:27.604551 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:27.604944 (+   393us) log.cc:844] Serialized 14 byte log entry
0709 13:05:28.101979 (+497035us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:28.574599 (+472620us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b2c000014c8. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:05:28.574975 (+   376us) log.cc:844] Serialized 26 byte log entry
0709 13:05:28.575131 (+   156us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:28.575167 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.575233 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:28.615589 (+ 40356us) raft_consensus.cc:1685] finished
0709 13:05:28.648710 (+ 33121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:28.648902 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13258,"prepare.queue_time_us":208,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":35162,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":18386,"spinlock_wait_cycles":5601536,"thread_start_us":483139,"threads_started":2,"wal-append.queue_time_us":451554}
I20260709 13:05:29.222412 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2088) took 1506 ms. Trace:
I20260709 13:05:29.222888 29072 rpcz_store.cc:276] 0709 13:05:27.715859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.716077 (+   218us) service_pool.cc:224] Handling call
0709 13:05:27.716532 (+   455us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:27.816216 (+ 99684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:27.816245 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:27.816857 (+   612us) log.cc:844] Serialized 52 byte log entry
0709 13:05:28.012403 (+195546us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:28.079580 (+ 67177us) log.cc:844] Serialized 14 byte log entry
0709 13:05:28.079808 (+   228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.082090 (+  2282us) spinlock_profiling.cc:243] Waited 99.3 ms on lock 0x7b5400068bd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:28.082152 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.222081 (+1139929us) raft_consensus.cc:1685] finished
0709 13:05:29.222186 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.222346 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":71340,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":5664,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":11375,"spinlock_wait_cycles":228299648,"thread_start_us":156556,"threads_started":1,"wal-append.queue_time_us":609826}
W20260709 13:05:29.213477 29188 leader_election.cc:336] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.242s (SENT)
I20260709 13:05:29.231281   825 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:29.238346   776 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.239319 29140 raft_consensus.cc:2368] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:29.241302 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2067) took 1142 ms. Trace:
I20260709 13:05:29.244367 29140 rpcz_store.cc:276] 0709 13:05:28.099026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.344499 (+245473us) service_pool.cc:224] Handling call
0709 13:05:29.241034 (+896535us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b54002fa3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:29.241228 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2053204352}
W20260709 13:05:29.252652 29188 leader_election.cc:336] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.496s (SENT)
W20260709 13:05:29.253891 29188 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.623s (SENT)
I20260709 13:05:29.254945   584 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:29.254849   776 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.255918   584 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.259650 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:29.262976   584 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:29.263578 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:29.264264 29904 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:29.264560   584 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:29.265203   584 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.265846 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:29.266508 29147 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:29.268076   584 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.269546 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:29.270354 29140 raft_consensus.cc:2368] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11.
I20260709 13:05:29.272181 29904 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:29.272367 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.273660 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.274232 29140 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:29.274282 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:29.275017 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.275393   630 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:29.276350   630 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:29.278205 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.279335 29140 raft_consensus.cc:2410] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:05:29.280342 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:29.280979 29140 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:29.281770   630 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.283512 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2046) took 1844 ms. Trace:
I20260709 13:05:29.283929 29142 rpcz_store.cc:276] 0709 13:05:27.454310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.456715 (+  2405us) service_pool.cc:224] Handling call
0709 13:05:28.121716 (+665001us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b5800003098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:05:28.121912 (+   196us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:28.332488 (+210576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:28.332520 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:28.343165 (+ 10645us) log.cc:844] Serialized 26 byte log entry
0709 13:05:28.538675 (+195510us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:28.538725 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.538789 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:29.283178 (+744389us) raft_consensus.cc:1685] finished
0709 13:05:29.283199 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:29.283451 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":210113,"raft.queue_time_us":210213,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":9427,"spinlock_wait_cycles":77689088,"thread_start_us":135180,"threads_started":1,"wal-append.queue_time_us":207461}
I20260709 13:05:29.285908 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.286893 29142 raft_consensus.cc:2410] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:05:29.290663 29039 leader_election.cc:336] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.547s (SENT)
I20260709 13:05:29.318225 29142 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:29.323051 29147 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
W20260709 13:05:29.344532 29039 leader_election.cc:336] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.901s (SENT)
I20260709 13:05:29.345074 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:29.345780   608 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:29.350016 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.357489 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:29.360813 29072 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:05:29.358343 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:29.362841 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:29.363287 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.363984 29068 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 13:05:29.368167 29188 leader_election.cc:336] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.564s (SENT)
I20260709 13:05:29.368945 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:29.371035 29039 leader_election.cc:336] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.114s (SENT)
I20260709 13:05:29.372258 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:29.390944 29188 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.791s (SENT)
I20260709 13:05:29.405645   630 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:29.406929 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:29.407481 29188 leader_election.cc:336] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.830s (SENT)
I20260709 13:05:29.407852 29072 raft_consensus.cc:2410] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:05:29.408124 32354 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:05:29.408668 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.408883 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:29.413367 29214 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:05:29.414724 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.415565 29068 raft_consensus.cc:2410] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:29.343485   512 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:29.417878 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.419569 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.421407 29073 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:29.421509 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.423390 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.424281 29147 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:29.425063 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.412564   584 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:29.416016   369 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:29.412437   711 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:29.416249   625 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.427464 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:29.438326   830 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.911s	user 0.004s	sys 0.008s
I20260709 13:05:29.427502 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.439409 29142 raft_consensus.cc:2368] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11.
I20260709 13:05:29.440021 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:29.440246   854 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:05:29.432751   584 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:29.440641 29064 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:29.440996   584 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.443166   584 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.422865   585 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:29.444001 29065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2080) took 2514 ms (client timeout 1842 ms). Trace:
I20260709 13:05:29.444152   585 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:29.444417 29065 rpcz_store.cc:269] 0709 13:05:26.939774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.940063 (+   289us) service_pool.cc:224] Handling call
0709 13:05:29.443747 (+2503684us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b5400186354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:29.443880 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1209827200}
I20260709 13:05:29.446202 29147 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:29.446434   585 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.433736   711 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.448745   711 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:29.422631   512 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.434623   625 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.435322   743 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:29.420599   740 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:05:29.416364   676 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:29.431169   369 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:29.459424   676 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.460095   369 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.461796   369 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:29.451551   512 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.458478 29140 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:29.474309 29217 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:05:29.474759 29217 pending_rounds.cc:85] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 8
I20260709 13:05:29.474987 29217 pending_rounds.cc:107] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:05:29.475227 29217 raft_consensus.cc:2889] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:29.476572 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.477586 29214 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:29.477128 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:29.478142 29215 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:29.478179 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.478888 29219 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:29.477734 29217 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
W20260709 13:05:28.641543 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.487794   608 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:29.488377   608 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.490693   608 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.497200 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:29.502324 29215 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
W20260709 13:05:29.504601 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 97235.8us
W20260709 13:05:29.505772 29188 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.941s (SENT)
I20260709 13:05:29.506263 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.507097   743 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:29.494427   630 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:29.508972   608 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:29.509071   512 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:29.509778   630 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.509282   617 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:29.510970 29111 connection.cc:256] RPC call timeout handler was delayed by -1.7019846290004352s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 13:05:29.513275 29039 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 3.629s (SENT)
I20260709 13:05:29.513818   617 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.514498 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:29.514966   630 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:29.515672 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:29.516041   617 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:29.513157 29111 connection.cc:256] RPC call timeout handler was delayed by -1.5948846290011716s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
I20260709 13:05:29.510164   512 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:29.521880 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:29.522315 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:29.522869 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.523406   512 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.515551   630 raft_consensus.cc:2764] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:05:29.524125   864 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.524724   864 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.526906   864 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.532591 29039 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:29.516780 29111 connection.cc:256] RPC call timeout handler was delayed by -1.5867846290011585s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
I20260709 13:05:29.517464   623 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:29.533421   608 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.539444   630 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:29.534102 29111 connection.cc:256] RPC call timeout handler was delayed by -1.5849846290011556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16410000000000002s for a response
I20260709 13:05:29.534309 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:29.548309 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:29.549824 29039 leader_election.cc:400] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11.
I20260709 13:05:29.549486 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.540601 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:29.551283 29111 connection.cc:256] RPC call timeout handler was delayed by -1.5750846290011395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1652s for a response
I20260709 13:05:29.551654 29070 raft_consensus.cc:2410] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:05:29.551808 29217 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
W20260709 13:05:29.551859 29111 connection.cc:256] RPC call timeout handler was delayed by -1.516584629001045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
W20260709 13:05:29.552791 29111 connection.cc:256] RPC call timeout handler was delayed by -1.5093846290010333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
W20260709 13:05:29.553426 29111 connection.cc:256] RPC call timeout handler was delayed by -1.255584629000623s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20070000000000002s for a response
W20260709 13:05:29.553807 29111 connection.cc:256] RPC call timeout handler was delayed by -1.1061846290003814s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
I20260709 13:05:29.552464 29039 leader_election.cc:403] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:05:29.553920 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.555197 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.555873 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.556869 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.554937 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.557608 29219 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:29.558494 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:29.552834   864 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:29.559130 29064 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:29.554167 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:29.554209 29111 connection.cc:256] RPC call timeout handler was delayed by -1.0908846290012661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
W20260709 13:05:29.563299 29111 connection.cc:256] RPC call timeout handler was delayed by -1.043184629001189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2243s for a response
I20260709 13:05:29.556726 29217 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:29.560382 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.560708 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:29.560925 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.552711   585 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:29.572304 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:29.572886 29073 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:29.540568   806 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:05:29.573868 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.575193 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.573791 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.553313   623 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:05:29.540930   608 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.586866 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.582648 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:29.590431 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:29.592934 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.593538 29219 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:05:29.593824 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.574467   615 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:29.594419 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.598137 29219 raft_consensus.cc:2368] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11.
I20260709 13:05:29.598878 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.594696 29215 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:29.588814 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.581786   776 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11.
I20260709 13:05:29.606294 29064 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:29.585903   806 consensus_queue.cc:1243] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:29.588992   512 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:29.615347 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:29.615767 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.616437 29215 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:29.616501 29073 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:29.617424 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.618808 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.619628 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.622601 29219 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:29.624357 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.600521   825 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:29.594116   617 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:05:29.655858 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.622288   783 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:29.659749 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.634330   825 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.661541   877 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:29.633893   776 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:29.661271 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 69252.9us
I20260709 13:05:29.669545 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:29.675648 29215 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:29.655757   585 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:29.668834 29073 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:29.677263 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.679477 29064 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:29.677778 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:05:29.682456 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 81174.4us
W20260709 13:05:29.682839 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 121202us
I20260709 13:05:29.684305   870 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:29.684674   864 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:29.685658   864 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:29.686036 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51261.4us
I20260709 13:05:29.687659 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:29.689380 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:29.690223   740 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:29.691303 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 98213.8us
I20260709 13:05:29.692552   740 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:29.693240   740 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:29.693800   872 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:29.695654 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:29.700488   775 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:29.700927   775 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:29.701297   775 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:29.701476 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.702921   871 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:29.700240   864 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:29.703876   871 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:29.704712 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.705621 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:29.673804   825 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:29.712395 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:29.704720   871 consensus_queue.cc:1243] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:29.716154 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:29.714188   872 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:05:29.719111 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 109302us
I20260709 13:05:29.717113   585 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.722540 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:29.723986   872 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:29.724869   874 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:29.714959 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.713140   623 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:29.725894   623 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:29.725442   872 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.727720   874 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:29.714829 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:29.708274   870 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:29.663959   863 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:05:29.726315   623 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:29.724154   876 raft_consensus.cc:2764] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:05:29.728616 29073 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 7.
I20260709 13:05:29.740247 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.741030 29215 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:29.745038 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:29.751329 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.752898 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:29.669860   783 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.729919   776 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:29.700269   740 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.730095   874 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:29.746482   866 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:05:29.740747   835 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:29.758912 29217 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:29.759451   608 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:29.752496   867 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
W20260709 13:05:29.767266 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47245.4us
I20260709 13:05:29.748852   615 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:29.755079   806 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:29.757218   617 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:05:29.776185 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 52629.8us
I20260709 13:05:29.777818 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:29.765937   874 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:29.779431   868 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 13:05:29.785333 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.787667 29215 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:29.793401 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.794706   877 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:05:29.796319   869 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:05:29.797384   806 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:29.797835   740 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:29.798481   806 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.803223 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:29.787456 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 71545.6us
I20260709 13:05:29.760465   875 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:29.822909 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:29.825737 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47998.4us
I20260709 13:05:29.823727   877 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:29.817304   875 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:29.836529 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:29.837751 29215 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:29.802798   776 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:05:29.843631 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.834223   743 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:29.851411 29215 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:29.838217   617 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 13:05:29.857067 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50758.2us
I20260709 13:05:29.857862 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.860286 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.840790   875 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:29.841271   369 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:29.870271   895 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:29.871438   895 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:29.863547   871 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:29.848712   887 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:29.892241 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 126603us
I20260709 13:05:29.900947 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:29.902308 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:29.903584 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 140529us
I20260709 13:05:29.907646   879 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:29.909258 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:29.910149 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:29.910954 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 53203.4us
I20260709 13:05:29.911469 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:29.866298   886 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:29.896113   743 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:29.845285   615 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.912932   740 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:29.930260 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:29.863848   877 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:05:29.915632   879 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:29.914758   871 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:29.950253   871 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.880455   369 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:29.937791 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 74991.3us
I20260709 13:05:29.919413   318 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:29.940407   743 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:28.723367 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 627) took 1714 ms. Trace:
I20260709 13:05:29.941781 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:29.941792   864 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:29.943739   623 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:29.936841   870 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:05:29.918123   886 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:29.921031   783 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:29.958305 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:29.958534 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.972625 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.973223 29140 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:29.975149 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.975497 29217 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:05:29.976361 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.976842 29147 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 7.
I20260709 13:05:29.962424 29213 rpcz_store.cc:276] 0709 13:05:27.009215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.009759 (+   544us) service_pool.cc:224] Handling call
0709 13:05:27.035414 (+ 25655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:27.078495 (+ 43081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:27.078544 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:27.085424 (+  6880us) log.cc:844] Serialized 26 byte log entry
0709 13:05:27.203844 (+118420us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:27.214630 (+ 10786us) log.cc:844] Serialized 14 byte log entry
0709 13:05:27.215419 (+   789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.215550 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:28.497853 (+1282303us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:28.722897 (+225044us) raft_consensus.cc:1685] finished
0709 13:05:28.723055 (+   158us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:28.723278 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46495,"prepare.queue_time_us":10620,"prepare.run_cpu_time_us":186,"prepare.run_wall_time_us":187,"raft.queue_time_us":53698,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":17479,"spinlock_wait_cycles":60857088,"thread_start_us":51344,"threads_started":1,"wal-append.queue_time_us":222832,"wal-append.run_cpu_time_us":8869,"wal-append.run_wall_time_us":2379897}
I20260709 13:05:29.948134   879 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.978825   879 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:29.948859   740 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:29.984830   740 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.985829   369 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:29.969189 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:29.988577 29142 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:29.958168   318 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:29.989526   318 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:29.971133   886 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:29.938779   887 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:05:29.968748   863 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:30.000092 29065 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:29.966379   884 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:30.000072 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:30.001678 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:30.002660 29140 raft_consensus.cc:2376] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12. Re-sending same reply.
I20260709 13:05:29.955399   775 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:30.004459 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:30.004788 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:30.005153 29147 raft_consensus.cc:2376] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13. Re-sending same reply.
I20260709 13:05:30.005735 29140 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
W20260709 13:05:30.008999 29188 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.100s (SENT)
I20260709 13:05:30.009218 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2089) took 2293 ms. Trace:
I20260709 13:05:30.009497 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:30.006083 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:30.010850 29147 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 7.
I20260709 13:05:30.012890 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:30.042450 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:30.013506 29142 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:30.043715 29147 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:30.009577 29071 rpcz_store.cc:276] 0709 13:05:27.716217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.995339 (+279122us) service_pool.cc:224] Handling call
0709 13:05:27.995777 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:28.013061 (+ 17284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:28.037856 (+ 24795us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:28.039593 (+  1737us) log.cc:844] Serialized 26 byte log entry
0709 13:05:28.420347 (+380754us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:28.460830 (+ 40483us) log.cc:844] Serialized 14 byte log entry
0709 13:05:28.468711 (+  7881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.468817 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:30.008975 (+1540158us) raft_consensus.cc:1685] finished
0709 13:05:30.009022 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.009161 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7983,"prepare.queue_time_us":41256,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":7758,"raft.queue_time_us":8812,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":27047,"spinlock_wait_cycles":614656,"thread_start_us":232847,"threads_started":1,"wal-append.queue_time_us":380007}
I20260709 13:05:30.002466   884 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:30.012545   318 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:30.062752   623 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:30.063984 29213 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:30.064404   318 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:30.064579   775 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:30.065150   585 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:30.066318   884 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:30.067858 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:30.068701 29142 raft_consensus.cc:2410] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:05:29.990988   825 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:29.974469   901 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:05:30.082211   875 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:30.089308   875 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:30.073037   874 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:30.091898   874 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:30.097949   525 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:30.098423   525 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:30.116159   825 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:28.579242 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:30.120929 29141 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:30.121762 29141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2316) took 1564 ms (client timeout 1772 ms). Trace:
W20260709 13:05:30.122169 29141 rpcz_store.cc:269] 0709 13:05:28.556886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.578980 (+ 22094us) service_pool.cc:224] Handling call
0709 13:05:30.121696 (+1542716us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:30.123448 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 114480us
I20260709 13:05:30.123584 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:30.124233 29142 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:30.126058 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:30.099418   525 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:30.132295   525 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:28.620563 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:30.135680 29146 raft_consensus.cc:2410] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:05:30.136122 29146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2320) took 1578 ms (client timeout 1960 ms). Trace:
W20260709 13:05:30.136488 29146 rpcz_store.cc:269] 0709 13:05:28.557908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.576448 (+ 18540us) service_pool.cc:224] Handling call
0709 13:05:30.136063 (+1559615us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:30.125304   874 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.143601   874 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:30.282244   877 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:30.283154   877 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:30.345994 29213 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:05:30.375468 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:30.438462 29218 pending_rounds.cc:85] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 2
I20260709 13:05:30.439086 29218 pending_rounds.cc:107] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:05:30.439450 29218 raft_consensus.cc:2889] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:30.450480   916 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:30.457866 29145 raft_consensus.cc:2368] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12.
W20260709 13:05:30.462229 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 336279us
W20260709 13:05:30.464915 29145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2073) took 2043 ms (client timeout 2100 ms). Trace:
I20260709 13:05:30.465024   630 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:30.465652   630 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:30.466598   630 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:30.468077 29145 rpcz_store.cc:269] 0709 13:05:28.426017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.434701 (+  8684us) service_pool.cc:224] Handling call
0709 13:05:30.459471 (+2024770us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b540029e354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:30.464840 (+  5369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4645441664}
I20260709 13:05:30.469771 29145 raft_consensus.cc:1217] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[3.3-13.7]   Dedup: 3.3->[5.4-13.7]
I20260709 13:05:30.467820   630 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:30.474429   676 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:30.480068 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:30.480610 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.480892   892 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:30.480074 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 570893us
I20260709 13:05:30.483738 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:30.484761 29038 leader_election.cc:400] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11.
I20260709 13:05:30.484848   872 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:30.485081 29038 leader_election.cc:403] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:30.485392   872 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:30.486469 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:30.487339   615 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:30.487787   615 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.489682 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2247) took 1016 ms. Trace:
I20260709 13:05:30.489741   615 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:30.488354   880 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:30.490939   880 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.491433   880 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:30.488116   900 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:05:30.500623   615 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:30.501243 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:30.503595 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:30.504596   615 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:30.506536 29214 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:05:30.505509 29218 rpcz_store.cc:276] 0709 13:05:29.473167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.437410 (+964243us) service_pool.cc:224] Handling call
0709 13:05:30.437978 (+   568us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:30.440352 (+  2374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:30.440396 (+    44us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:30.477080 (+ 36684us) log.cc:844] Serialized 78 byte log entry
0709 13:05:30.485588 (+  8508us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:30.485633 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.485701 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:30.489373 (+  3672us) raft_consensus.cc:1685] finished
0709 13:05:30.489400 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:30.489609 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9890,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":131,"thread_start_us":8120,"threads_started":1,"wal-append.queue_time_us":7907}
I20260709 13:05:30.487696   872 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:30.509972 29214 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:30.510967 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.512363 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:30.517446   615 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:30.518244   615 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:30.519903   900 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11.
I20260709 13:05:30.521016   615 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:30.522200 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:30.524608   880 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:30.526568 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:30.551515 29073 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:30.527698 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:30.531136 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.531826 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:30.553602 29217 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:30.536521 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:30.554008 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.554231 29215 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:30.540778 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:30.557715 29218 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:30.542150 29145 pending_rounds.cc:85] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 3
I20260709 13:05:30.542575 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:30.559235 29145 pending_rounds.cc:107] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:05:30.559727 29145 raft_consensus.cc:2889] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:30.560231 29145 pending_rounds.cc:107] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:05:30.560612 29145 raft_consensus.cc:2889] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:30.561048 29145 pending_rounds.cc:107] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:05:30.561399 29145 raft_consensus.cc:2889] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:30.464941 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:30.543862   886 consensus_meta.cc:287] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.552s	user 0.014s	sys 0.003s
W20260709 13:05:30.562523 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 97658us
I20260709 13:05:30.563512 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:30.569273 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:30.572806 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:30.574884 29215 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
W20260709 13:05:30.544147   740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.007s	sys 0.013s
I20260709 13:05:30.611747 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:31.969544 29064 raft_consensus.cc:2410] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:05:31.970093 29064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2118) took 1368 ms (client timeout 1776 ms). Trace:
W20260709 13:05:31.970419 29064 rpcz_store.cc:269] 0709 13:05:30.602026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.611432 (+  9406us) service_pool.cc:224] Handling call
0709 13:05:31.970022 (+1358590us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:31.971349 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:30.592684 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:30.636889 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:31.972800 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2115) took 1443 ms. Trace:
I20260709 13:05:31.973445 29072 raft_consensus.cc:2410] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:05:31.973981 29072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2195) took 2568 ms (client timeout 2110 ms). Trace:
W20260709 13:05:31.974352 29072 rpcz_store.cc:269] 0709 13:05:29.405988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.636603 (+1230615us) service_pool.cc:224] Handling call
0709 13:05:31.973906 (+1337303us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:31.975065   740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.978s	user 0.010s	sys 0.029s
I20260709 13:05:31.975364 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:31.976063 29072 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:30.632900 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:30.613718 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:32.275409 29072 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:32.276010 29140 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:32.276122 29072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2205) took 1601 ms (client timeout 1860 ms). Trace:
W20260709 13:05:32.276690 29072 rpcz_store.cc:269] 0709 13:05:30.674357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.975108 (+1300751us) service_pool.cc:224] Handling call
0709 13:05:32.276051 (+300943us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:31.714341   941 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:31.717077   942 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.278452   941 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.278575   942 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.281028   942 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:32.281517   942 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:32.281667   941 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:32.281960   942 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.282433   941 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.283053   951 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.282974   941 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:31.727566 29144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2084) took 3174 ms (client timeout 1830 ms). Trace:
I20260709 13:05:32.283599   951 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:32.284093 29144 rpcz_store.cc:269] 0709 13:05:28.553337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.561466 (+  8129us) service_pool.cc:224] Handling call
0709 13:05:31.727435 (+3165969us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:31.733645 29069 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:32.285728   951 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.285987   676 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.286345 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2187) took 2931 ms. Trace:
I20260709 13:05:32.286523   676 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.286716 29069 rpcz_store.cc:276] 0709 13:05:29.354752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.733088 (+2378336us) service_pool.cc:224] Handling call
0709 13:05:31.733452 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.286102 (+552650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.286288 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.288017 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:32.288633 29069 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:32.288692   676 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:32.289160 29069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2206) took 1610 ms (client timeout 1882 ms). Trace:
W20260709 13:05:32.289529 29069 rpcz_store.cc:269] 0709 13:05:30.679536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.287725 (+1608189us) service_pool.cc:224] Handling call
0709 13:05:32.289103 (+  1378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.290479 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:32.291249 29069 raft_consensus.cc:2410] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:05:31.756454 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:31.754143 29145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.073s	user 0.014s	sys 0.013s
I20260709 13:05:31.634550 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2326) took 2278 ms. Trace:
W20260709 13:05:30.771049 29188 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.999s (SENT)
W20260709 13:05:32.337669 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 1.56657e+06us
I20260709 13:05:30.675582 29213 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:32.341285 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2354) took 1679 ms. Trace:
I20260709 13:05:32.341626 29213 rpcz_store.cc:276] 0709 13:05:30.662108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.664321 (+  2213us) service_pool.cc:224] Handling call
0709 13:05:30.675129 (+ 10808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.341028 (+1665899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.341232 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:30.639370 29217 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
W20260709 13:05:32.344131 29217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 660) took 1813 ms (client timeout 1841 ms). Trace:
W20260709 13:05:32.344497 29217 rpcz_store.cc:269] 0709 13:05:30.531165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.531547 (+   382us) service_pool.cc:224] Handling call
0709 13:05:32.344071 (+1812524us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.344746 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2113) took 1770 ms. Trace:
I20260709 13:05:30.638782 29218 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:05:32.345131 29147 rpcz_store.cc:276] 0709 13:05:30.573933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.301605 (+1727672us) service_pool.cc:224] Handling call
0709 13:05:32.301920 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.302147 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:05:32.302491 (+   344us) log.cc:844] Serialized 14 byte log entry
0709 13:05:32.344432 (+ 41941us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:32.344483 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:32.344526 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.344591 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.344697 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2105,"prepare.queue_time_us":205,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"spinlock_wait_cycles":24832,"thread_start_us":28158,"threads_started":1,"wal-append.queue_time_us":26045}
W20260709 13:05:32.346158 29218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2352) took 1806 ms (client timeout 2175 ms). Trace:
W20260709 13:05:32.346493 29218 rpcz_store.cc:269] 0709 13:05:30.540130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.540486 (+   356us) service_pool.cc:224] Handling call
0709 13:05:32.346101 (+1805615us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:30.681237 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:32.348107 29215 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:31.800299 29243 debug-util.cc:398] Leaking SignalData structure 0x7b080020d1c0 after lost signal to thread 27937
W20260709 13:05:32.355048 29243 debug-util.cc:398] Leaking SignalData structure 0x7b0800315940 after lost signal to thread 29005
W20260709 13:05:30.985373 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.358856 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.359208 29874 connection.cc:256] RPC call timeout handler was delayed by -1.1893361610000284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
I20260709 13:05:31.803287 29902 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:32.361334   883 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:31.695633 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.363619   630 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:31.764489 29904 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:31.727460 32354 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:32.367080   883 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:05:31.705564   525 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 1.572s	user 0.007s	sys 0.026s
W20260709 13:05:31.586062   585 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.520s	user 0.014s	sys 0.005s
I20260709 13:05:32.388870   585 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:31.600255   873 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.392073 29222 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:30.638128   874 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:30.777364   876 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.398661   876 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:30.583528   870 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:30.776943   880 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:30.579061   877 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:30.580653   878 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:32.404968   878 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.405548   878 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:30.600718   879 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.621s	user 0.014s	sys 0.005s
I20260709 13:05:32.409233   879 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.411568   879 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:30.578419   886 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:30.580849   909 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:32.420130 29221 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:32.422283 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2328) took 3065 ms. Trace:
I20260709 13:05:32.422749 29221 rpcz_store.cc:276] 0709 13:05:29.357160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:29.967208 (+610048us) service_pool.cc:224] Handling call
0709 13:05:29.977908 (+ 10700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.420624 (+2442716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.421957 (+  1333us) spinlock_profiling.cc:243] Waited 2.44 s on lock 0x7b540023c154. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:32.422198 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5616132992}
I20260709 13:05:31.013386   916 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.424911   916 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:31.715029 29186 connection.cc:256] RPC call timeout handler was delayed by -1.8541739560005226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21989999999999998s for a response
I20260709 13:05:32.430604 29186 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:31.837981 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2110) took 1265 ms. Trace:
I20260709 13:05:32.431313 29141 rpcz_store.cc:276] 0709 13:05:30.572257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.780247 (+207990us) service_pool.cc:224] Handling call
0709 13:05:30.780598 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:31.800028 (+1019430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:31.800069 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:31.800601 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:05:31.817479 (+ 16878us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:31.836602 (+ 19123us) log.cc:844] Serialized 14 byte log entry
0709 13:05:31.837598 (+   996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.837687 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:31.837732 (+    45us) raft_consensus.cc:1685] finished
0709 13:05:31.837747 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:31.837912 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1018979,"prepare.queue_time_us":19085,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":1181507,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":207,"thread_start_us":35320,"threads_started":2,"wal-append.queue_time_us":14859}
I20260709 13:05:28.642449 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:32.433710 29148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2324) took 3874 ms (client timeout 1681 ms). Trace:
W20260709 13:05:32.434088 29148 rpcz_store.cc:269] 0709 13:05:28.559079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.563386 (+  4307us) service_pool.cc:224] Handling call
0709 13:05:32.433626 (+3870240us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:31.858692 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2188) took 2503 ms (client timeout 2214 ms). Trace:
W20260709 13:05:32.434856 29067 rpcz_store.cc:269] 0709 13:05:29.355027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.858487 (+2503460us) service_pool.cc:211] Skipping call since client already timed out
0709 13:05:31.858610 (+   123us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:05:32.436029 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:32.436852 29067 raft_consensus.cc:2376] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12. Re-sending same reply.
W20260709 13:05:32.437476 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2197) took 1806 ms (client timeout 2208 ms). Trace:
W20260709 13:05:32.437896 29067 rpcz_store.cc:269] 0709 13:05:30.631455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.435668 (+1804213us) service_pool.cc:224] Handling call
0709 13:05:32.437405 (+  1737us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:31.860919 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2106) took 1839 ms. Trace:
I20260709 13:05:32.439137 29146 rpcz_store.cc:276] 0709 13:05:30.021533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.137209 (+115676us) service_pool.cc:224] Handling call
0709 13:05:30.137696 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:30.551940 (+414244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:30.551970 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:30.583363 (+ 31393us) log.cc:844] Serialized 26 byte log entry
0709 13:05:30.671842 (+ 88479us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:30.777397 (+105555us) log.cc:844] Serialized 14 byte log entry
0709 13:05:30.777635 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.777728 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:31.860640 (+1082912us) raft_consensus.cc:1685] finished
0709 13:05:31.860662 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:31.860847 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":107606,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":50,"raft.queue_time_us":14443,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":6481,"spinlock_wait_cycles":708992,"thread_start_us":57625,"threads_started":1,"wal-append.queue_time_us":182734,"wal-append.run_cpu_time_us":2386,"wal-append.run_wall_time_us":1002405}
W20260709 13:05:31.893903 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.896052 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:32.443063   623 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.443512 29143 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:32.443818   623 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.446568   623 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.447149   623 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:31.899351 29142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.121s	user 0.031s	sys 0.040s
W20260709 13:05:32.448261 29142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.671s	user 0.032s	sys 0.040s
I20260709 13:05:32.447634   623 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.449116 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2108) took 1878 ms. Trace:
I20260709 13:05:32.449501 29142 rpcz_store.cc:276] 0709 13:05:30.571063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.608597 (+ 37534us) service_pool.cc:224] Handling call
0709 13:05:30.609161 (+   564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.609432 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:30.776602 (+167170us) log.cc:844] Serialized 14 byte log entry
0709 13:05:32.448723 (+1672121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:32.448787 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:32.448843 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.448918 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.449058 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34437,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"thread_start_us":1808359,"threads_started":2,"wal-append.queue_time_us":13029,"wal-append.run_cpu_time_us":1044,"wal-append.run_wall_time_us":1001068}
I20260709 13:05:31.903447   900 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.450847   623 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.451363   900 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.454073   900 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:31.905946 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2110) took 2143 ms. Trace:
I20260709 13:05:32.454792 29070 rpcz_store.cc:276] 0709 13:05:29.762863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.478723 (+715860us) service_pool.cc:224] Handling call
0709 13:05:30.479038 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.479253 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:30.604236 (+124983us) log.cc:844] Serialized 14 byte log entry
0709 13:05:30.672433 (+ 68197us) log.cc:844] Serialized 14 byte log entry
0709 13:05:31.842739 (+1170306us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b18002dc9a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:05:31.843114 (+   375us) log.cc:844] Serialized 14 byte log entry
0709 13:05:31.905075 (+ 61961us) log.cc:844] Serialized 14 byte log entry
0709 13:05:31.905365 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 13:05:31.905472 (+   107us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:31.905512 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:31.905652 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.905730 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:31.905871 (+   141us) inbound_call.cc:177] Queueing success response
0709 13:05:31.962595 (+ 56724us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b18002dc9a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
Metrics: {"mutex_wait_us":1288635,"prepare.queue_time_us":407258,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":132,"raft.queue_time_us":1563108,"raft.run_cpu_time_us":466,"raft.run_wall_time_us":493,"spinlock_wait_cycles":2692118656,"thread_start_us":188473,"threads_started":3,"wal-append.queue_time_us":182139,"wal-append.run_cpu_time_us":1588,"wal-append.run_wall_time_us":1003591}
I20260709 13:05:32.458130 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:32.459120 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2201) took 1823 ms. Trace:
I20260709 13:05:31.932482   776 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.460273   776 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.462533   776 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:31.937067 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 469) took 4909 ms. Trace:
I20260709 13:05:32.463127 29066 rpcz_store.cc:276] 0709 13:05:27.027732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.037623 (+  9891us) service_pool.cc:224] Handling call
0709 13:05:27.038034 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:27.775647 (+737613us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:27.784499 (+  8852us) log.cc:844] Serialized 14 byte log entry
0709 13:05:28.082473 (+297974us) log.cc:844] Serialized 14 byte log entry
0709 13:05:28.082665 (+   192us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:28.083221 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:05:28.083357 (+   136us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:28.083555 (+   198us) log.cc:844] Serialized 14 byte log entry
0709 13:05:28.083689 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:28.083767 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:31.936690 (+3852923us) raft_consensus.cc:1685] finished
0709 13:05:31.936717 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:31.936943 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53374,"prepare.queue_time_us":58647,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":90098,"raft.run_cpu_time_us":429,"raft.run_wall_time_us":29228,"thread_start_us":887148,"threads_started":2,"wal-append.queue_time_us":382230,"wal-append.run_cpu_time_us":18201,"wal-append.run_wall_time_us":1854123}
W20260709 13:05:31.943416 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.467429 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.941650 28959 maintenance_manager.cc:643] P 9c100c1aaf1b493089df815bded98302: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 7.438s	user 0.146s	sys 0.131s Metrics: {"bytes_written":30045,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":404785,"dirs.run_cpu_time_us":371,"dirs.run_wall_time_us":2578,"drs_written":1,"lbm_read_time_us":390,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1046074,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":6,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":416788,"threads_started":2,"update_count":9,"wal-append.queue_time_us":29946,"wal-append.run_cpu_time_us":575,"wal-append.run_wall_time_us":1000575}
I20260709 13:05:30.604986 29073 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
W20260709 13:05:32.470211 29073 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 503) took 1944 ms (client timeout 1994 ms). Trace:
I20260709 13:05:32.470933 28960 maintenance_manager.cc:419] P 9c100c1aaf1b493089df815bded98302: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8449 bytes on disk
I20260709 13:05:31.973181 29068 rpcz_store.cc:276] 0709 13:05:30.529185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.584579 (+ 55394us) service_pool.cc:224] Handling call
0709 13:05:31.972733 (+1388154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.471225   608 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:32.472121   608 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.472738 29068 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:32.473444 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2111) took 1945 ms. Trace:
I20260709 13:05:32.473785 29068 rpcz_store.cc:276] 0709 13:05:30.528017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.472028 (+1944011us) service_pool.cc:224] Handling call
0709 13:05:32.472354 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.473185 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.473388 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.475098   608 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:31.998946 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 1.366e+06us
I20260709 13:05:32.477092 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:31.999135   947 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:32.004431   948 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.479075   877 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:32.479315   948 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.136945   740 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.995469   806 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:32.480705   806 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.482088 28959 maintenance_manager.cc:643] P 9c100c1aaf1b493089df815bded98302: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":412,"lbm_reads_lt_1ms":4}
I20260709 13:05:32.482920   806 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:32.219703 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.483379 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.484195 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:32.485769   951 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:32.485396 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:32.486858 29039 connection.cc:256] RPC call timeout handler was delayed by -2.3029151909995562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2561s for a response
I20260709 13:05:32.487020   951 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:32.470702 29073 rpcz_store.cc:269] 0709 13:05:30.526174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.526304 (+   130us) service_pool.cc:224] Handling call
0709 13:05:32.470128 (+1943824us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:32.487483 29039 connection.cc:256] RPC call timeout handler was delayed by -1.8690743719998864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
I20260709 13:05:32.237151   673 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.239061   895 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:32.488212   895 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.253224   512 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:32.263164 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.489306   512 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.291038 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:32.292429 29145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.620s	user 0.015s	sys 0.014s
I20260709 13:05:32.490530   895 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.490856 29072 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
W20260709 13:05:32.491330 29072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2209) took 1810 ms (client timeout 1953 ms). Trace:
W20260709 13:05:32.294899 29065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2196) took 1663 ms (client timeout 1514 ms). Trace:
I20260709 13:05:32.248724   615 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:32.296604   942 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:32.297709   813 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:32.491746 29072 rpcz_store.cc:269] 0709 13:05:30.680515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.290822 (+1610307us) service_pool.cc:224] Handling call
0709 13:05:32.491277 (+200455us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.488323 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.492244   615 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.493108   947 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.493716 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:32.291347   941 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:32.310798   925 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.667s	user 0.007s	sys 0.013s
I20260709 13:05:32.510013 29067 raft_consensus.cc:2410] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:05:32.319805 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2112) took 1746 ms. Trace:
I20260709 13:05:32.514209 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2109) took 2751 ms. Trace:
I20260709 13:05:32.514547 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2199) took 1882 ms. Trace:
I20260709 13:05:32.515202   916 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.322073   625 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.326289 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.331225 29216 rpcz_store.cc:276] 0709 13:05:29.356519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.633504 (+2276985us) service_pool.cc:224] Handling call
0709 13:05:31.633911 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.634206 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:31.634222 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:31.634276 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:31.634307 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.634357 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:31.634483 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79881,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":104}
I20260709 13:05:32.514442 29144 rpcz_store.cc:276] 0709 13:05:30.573350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.284888 (+1711538us) service_pool.cc:224] Handling call
0709 13:05:32.285361 (+   473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:32.306601 (+ 21240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.306636 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:32.307253 (+   617us) log.cc:844] Serialized 52 byte log entry
0709 13:05:32.319428 (+ 12175us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:32.319479 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.319551 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.319596 (+    45us) raft_consensus.cc:1685] finished
0709 13:05:32.319610 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.319742 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":575,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":32532,"threads_started":2,"wal-append.queue_time_us":10211}
I20260709 13:05:32.331769   928 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:32.516853 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.360260   369 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:32.364460   630 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:32.518991   630 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:32.519817   630 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.514717 29071 rpcz_store.cc:276] 0709 13:05:29.762634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.044855 (+282221us) service_pool.cc:224] Handling call
0709 13:05:30.045422 (+   567us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:30.482642 (+437220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:30.482674 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:30.483249 (+   575us) log.cc:844] Serialized 52 byte log entry
0709 13:05:30.606736 (+123487us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:30.606774 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:30.606837 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.310130 (+1703293us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:32.513889 (+203759us) raft_consensus.cc:1685] finished
0709 13:05:32.513913 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.514147 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":436807,"raft.queue_time_us":437064,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":2919168,"thread_start_us":123091,"threads_started":1,"wal-append.queue_time_us":159609}
I20260709 13:05:32.521301   630 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:32.522404 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2117) took 1992 ms. Trace:
I20260709 13:05:32.522822 29071 rpcz_store.cc:276] 0709 13:05:30.529706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.521561 (+1991855us) service_pool.cc:224] Handling call
0709 13:05:32.521870 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.522062 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.522090 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:32.522137 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:32.522172 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.522231 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.522347 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.366091   697 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:05:32.524160 29243 debug-util.cc:398] Leaking SignalData structure 0x7b0800300580 after lost signal to thread 29097
I20260709 13:05:32.524255 29071 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:32.524587 29243 debug-util.cc:398] Leaking SignalData structure 0x7b0800300540 after lost signal to thread 29098
I20260709 13:05:32.525084 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2200) took 1890 ms. Trace:
W20260709 13:05:32.525698 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08002f7380 after lost signal to thread 29172
I20260709 13:05:32.525442 29071 rpcz_store.cc:276] 0709 13:05:30.634942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.523780 (+1888838us) service_pool.cc:224] Handling call
0709 13:05:32.524083 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.524841 (+   758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.525028 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.377425 29217 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:32.533041 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.534484 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.535143 29217 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
W20260709 13:05:32.379989 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.538026 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:32.539048   369 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:32.539453   369 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.539844   369 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:32.540040 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:32.384500 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.469558   623 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:31.983760 29064 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:32.545363   584 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.545743 29214 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
W20260709 13:05:32.546057 29064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2204) took 1893 ms (client timeout 1544 ms). Trace:
W20260709 13:05:32.546473 29064 rpcz_store.cc:269] 0709 13:05:30.652253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.971131 (+1318878us) service_pool.cc:224] Handling call
0709 13:05:32.545983 (+574852us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.546264   584 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.545758   623 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.547727 29064 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:32.548381 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2202) took 1907 ms. Trace:
I20260709 13:05:32.548672 29064 rpcz_store.cc:276] 0709 13:05:30.641235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.547262 (+1906027us) service_pool.cc:224] Handling call
0709 13:05:32.547569 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.548132 (+   563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.548328 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.549479   623 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.549815 29064 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:05:32.549993   585 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:32.550743 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:32.550707   585 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.551708   928 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:32.552134   928 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.480654   740 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.552517   928 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:32.488181   673 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.553293   585 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:31.602918 29875 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.011s	sys 0.000s
I20260709 13:05:32.556850 29907 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:05:32.557368 29875 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.681s	user 0.014s	sys 0.000s
W20260709 13:05:32.557821 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 1.68211e+06us
I20260709 13:05:32.491248 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2107) took 2469 ms. Trace:
I20260709 13:05:32.560225 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2119) took 1904 ms. Trace:
I20260709 13:05:32.559279 29145 rpcz_store.cc:276] 0709 13:05:30.021810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.469125 (+447315us) service_pool.cc:224] Handling call
0709 13:05:30.469547 (+   422us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 13:05:30.635618 (+166071us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:05:30.635647 (+    29us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:05:30.636499 (+   852us) log.cc:844] Serialized 104 byte log entry
0709 13:05:32.490915 (+1854416us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:05:32.490954 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.491018 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.491061 (+    43us) raft_consensus.cc:1685] finished
0709 13:05:32.491071 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.491195 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1735,"raft.queue_time_us":30622,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3257,"thread_start_us":1887372,"threads_started":2,"wal-append.queue_time_us":136451,"wal-append.run_cpu_time_us":2170,"wal-append.run_wall_time_us":1003676}
I20260709 13:05:28.579974 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:32.560597 29071 rpcz_store.cc:276] 0709 13:05:30.655683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.559368 (+1903685us) service_pool.cc:224] Handling call
0709 13:05:32.559686 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.559865 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.559888 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:32.559988 (+   100us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:32.560026 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.560080 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.560174 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.391819   873 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.569445 29216 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:32.475742   608 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.570561   608 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.494381   951 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.572274   951 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.459522 29070 rpcz_store.cc:276] 0709 13:05:30.635184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.457756 (+1822572us) service_pool.cc:224] Handling call
0709 13:05:32.459045 (+  1289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.575318 29070 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 13:05:32.576040 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2207) took 1896 ms. Trace:
I20260709 13:05:32.576368 29070 rpcz_store.cc:276] 0709 13:05:30.679891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.574701 (+1894810us) service_pool.cc:224] Handling call
0709 13:05:32.575073 (+   372us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:32.575705 (+   632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.575949 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.581204 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:32.494458   615 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:32.297084 29069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2198) took 1665 ms (client timeout 1941 ms). Trace:
I20260709 13:05:32.297075 29140 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:32.495560   806 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:32.491832 29065 rpcz_store.cc:269] 0709 13:05:30.631187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.756102 (+1124915us) service_pool.cc:224] Handling call
0709 13:05:32.294816 (+538714us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:32.487861 29039 connection.cc:256] RPC call timeout handler was delayed by -1.7772743719997379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 13:05:32.587879 29069 rpcz_store.cc:269] 0709 13:05:30.631719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.290190 (+1658471us) service_pool.cc:224] Handling call
0709 13:05:32.297016 (+  6826us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:32.588239 29140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2111) took 2015 ms (client timeout 2249 ms). Trace:
W20260709 13:05:32.590159 29875 leader_election.cc:336] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.758s (SENT)
W20260709 13:05:32.590281 29140 rpcz_store.cc:269] 0709 13:05:30.572802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.613395 (+ 40593us) service_pool.cc:224] Handling call
0709 13:05:32.588181 (+1974786us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.497622 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:32.588402 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:32.588796 29039 connection.cc:256] RPC call timeout handler was delayed by -1.7662979149999956s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
W20260709 13:05:32.592358 29039 connection.cc:256] RPC call timeout handler was delayed by -1.7356979149999461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
W20260709 13:05:32.526155 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08002fa280 after lost signal to thread 29173
W20260709 13:05:32.592710 29039 connection.cc:256] RPC call timeout handler was delayed by -1.6827743719995851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18520000000000003s for a response
I20260709 13:05:32.593115 29064 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:32.500825 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2114) took 1972 ms. Trace:
I20260709 13:05:32.482998   948 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.502241   813 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:32.502394   876 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.504150   880 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.505260 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.506214 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.639309 29213 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:32.491606   512 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.508415 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.318794   884 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:32.495627 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2113) took 1967 ms. Trace:
I20260709 13:05:32.646852 29073 rpcz_store.cc:276] 0709 13:05:30.528611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.494829 (+1966218us) service_pool.cc:224] Handling call
0709 13:05:32.495151 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.495331 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.495357 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:32.495397 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:32.495430 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.495478 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.495572 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.503643 29902 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:32.513680   963 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.372622 32354 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.657402 32354 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:32.524428   697 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:32.415638   886 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:32.531090 29222 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:32.554479   942 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:32.561419 29139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2094) took 4004 ms (client timeout 1999 ms). Trace:
W20260709 13:05:32.668234 29139 rpcz_store.cc:269] 0709 13:05:28.556588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.579706 (+ 23118us) service_pool.cc:224] Handling call
0709 13:05:32.561329 (+3981623us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:32.582087 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 151563us
I20260709 13:05:32.493597   947 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.497825 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:32.593189 29039 connection.cc:256] RPC call timeout handler was delayed by -1.488958198000546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
I20260709 13:05:32.669440 29186 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:32.669879 29039 connection.cc:256] RPC call timeout handler was delayed by -1.315397915000176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
W20260709 13:05:32.670336 29039 connection.cc:256] RPC call timeout handler was delayed by -1.3041242690005674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 13:05:32.670712 29039 connection.cc:256] RPC call timeout handler was delayed by -1.1613060039999255s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20350000000000001s for a response
W20260709 13:05:32.671087 29039 connection.cc:256] RPC call timeout handler was delayed by -1.1550581980000061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
I20260709 13:05:32.500651   892 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:32.586148 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 46214.6us
W20260709 13:05:32.672508 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 174789us
I20260709 13:05:32.588009 29067 rpcz_store.cc:276] 0709 13:05:30.634271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.438685 (+1804414us) service_pool.cc:224] Handling call
0709 13:05:32.514490 (+ 75805us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:32.673276 29222 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2250) took 3199 ms (client timeout 1860 ms). Trace:
W20260709 13:05:32.593988 29243 debug-util.cc:398] Leaking SignalData structure 0x7b080035b240 after lost signal to thread 29246
W20260709 13:05:32.673686 29222 rpcz_store.cc:269] 0709 13:05:29.474197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.375083 (+900886us) service_pool.cc:224] Handling call
0709 13:05:32.673110 (+2298027us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54002837d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:32.673208 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4556635008}
I20260709 13:05:32.594229 29216 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:32.636521   879 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:32.680593   879 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.682833   879 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.683495 29906 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:32.647174   880 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.687017   880 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.636806   813 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.690732   813 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.662146   697 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:32.692315 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:32.601369 29143 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:32.589182   584 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:32.616369   615 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:32.617090   740 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.619505   909 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:32.621475   951 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.621994   673 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.588277   525 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:32.706892   967 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:05:32.712399 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.713610 29221 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:32.623549 29218 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: [11.8-15.10]
I20260709 13:05:32.713863 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:32.626065 29875 connection.cc:256] RPC call timeout handler was delayed by -1.3892149569992398s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
W20260709 13:05:32.513170 29112 connection.cc:256] RPC call timeout handler was delayed by -2.3770943979998265s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
I20260709 13:05:32.702859   813 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:32.664291 28971 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 6 to 7, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 7 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:32.636032   883 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:32.708881   880 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.716498   525 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:32.721082 29142 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:32.720748   880 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.722132   879 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.721889   525 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.722653   879 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.663192   963 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.724648   879 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.593660 29072 rpcz_store.cc:276] 0709 13:05:30.528903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.492756 (+1963853us) service_pool.cc:224] Handling call
0709 13:05:32.500145 (+  7389us) spinlock_profiling.cc:243] Waited 5.58 ms on lock 0x7b5800005198. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:05:32.500339 (+   194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.500514 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.500538 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:32.500579 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:32.500613 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.500669 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.500770 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12828928}
I20260709 13:05:32.643669   948 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.725723   948 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.726393   963 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.674132 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:32.718132   697 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:32.728495 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 54464us
I20260709 13:05:32.729023   697 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:32.729212 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:32.730381 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:32.731093   877 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:32.598548 29215 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:05:32.731810   877 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:32.732201 29215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2355) took 2051 ms (client timeout 2030 ms). Trace:
I20260709 13:05:32.719708 29220 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:32.720534 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:32.709278   369 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.733562 29214 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:32.718501 29221 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:05:32.733771 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2327) took 3377 ms. Trace:
I20260709 13:05:32.734221 29220 rpcz_store.cc:276] 0709 13:05:29.356838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.626024 (+2269186us) service_pool.cc:224] Handling call
0709 13:05:31.626479 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.732958 (+1106479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.733565 (+   607us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540023bc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:32.733716 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2513721088}
I20260709 13:05:32.676636   892 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:32.671458 29039 connection.cc:256] RPC call timeout handler was delayed by -1.0731581979998737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
I20260709 13:05:32.734959   369 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.678483   873 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:32.735265 29039 connection.cc:256] RPC call timeout handler was delayed by -1.0425581979998242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
I20260709 13:05:32.679062 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.735991 29213 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 7.
I20260709 13:05:32.736024 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:32.737663   892 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.738158   892 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.737540 28971 catalog_manager.cc:5697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 4 to 6, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 6 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:32.738572   892 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:32.678969   947 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.679733   895 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:32.720173 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.740163   623 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:32.740747 29218 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:32.588564   806 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:32.720297 29875 connection.cc:256] RPC call timeout handler was delayed by -1.0355535829994551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 13:05:32.742576 29215 rpcz_store.cc:269] 0709 13:05:30.680741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.680920 (+   179us) service_pool.cc:224] Handling call
0709 13:05:32.732130 (+2051210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:32.674223 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08003fcfe0 after lost signal to thread 29247
I20260709 13:05:32.700484 32354 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.660108   630 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:05:32.621910   878 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.660986 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:32.632601 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:32.715597 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 345080us
I20260709 13:05:32.594822 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2203) took 1947 ms. Trace:
W20260709 13:05:32.745677 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 84807.1us
I20260709 13:05:32.731766 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:32.746021 29064 rpcz_store.cc:276] 0709 13:05:30.647796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.549459 (+1901663us) service_pool.cc:224] Handling call
0709 13:05:32.549722 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.594534 (+ 44812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.594764 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:32.722936   880 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.732834   951 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:32.676553   916 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:32.720438   883 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.761124 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:32.744354 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08001edb80 after lost signal to thread 29935
I20260709 13:05:32.694991 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:32.762158 29243 debug-util.cc:398] Leaking SignalData structure 0x7b0800230ec0 after lost signal to thread 674
I20260709 13:05:32.712832 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 505) took 2185 ms. Trace:
I20260709 13:05:32.744884 29140 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:32.762897 29243 debug-util.cc:398] Leaking SignalData structure 0x7b0800318b80 after lost signal to thread 797
I20260709 13:05:32.727654   948 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:32.746960 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.764529 29038 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:32.766341   947 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:32.767575   947 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:32.768055   947 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:32.769217   947 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.748945 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.749352 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.750658   806 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.750799 29219 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:32.771724   947 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:32.751096 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.749002 28971 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 7 to 8, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 8 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:32.775797 28971 catalog_manager.cc:5697] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:32.776937 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:32.778793   512 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:32.756847 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.780596   952 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:32.753314 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.759006 32354 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.781857 32354 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:32.782245 32354 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.782653 32354 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:32.753830 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.747895 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:32.756855 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.752074 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.752951 29903 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:32.752859 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.738698 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:32.751906   878 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.787650 29215 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
W20260709 13:05:32.707198 29143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 599) took 5231 ms (client timeout 1758 ms). Trace:
I20260709 13:05:32.742183 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2208) took 2062 ms. Trace:
I20260709 13:05:32.791757 29070 rpcz_store.cc:276] 0709 13:05:30.680198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.577211 (+1897013us) service_pool.cc:224] Handling call
0709 13:05:32.577534 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:32.583286 (+  5752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.583318 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:32.583788 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:05:32.633284 (+ 49496us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:32.642674 (+  9390us) log.cc:844] Serialized 14 byte log entry
0709 13:05:32.642958 (+   284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.643034 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.741997 (+ 98963us) raft_consensus.cc:1685] finished
0709 13:05:32.742018 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.742129 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2018,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":13632,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":12996,"thread_start_us":13287,"threads_started":2,"wal-append.queue_time_us":10062}
I20260709 13:05:32.755664 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.795568 29216 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
W20260709 13:05:32.790469 29143 rpcz_store.cc:269] 0709 13:05:27.466032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.484662 (+ 18630us) service_pool.cc:224] Handling call
0709 13:05:31.895793 (+4411131us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b5800003098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:05:32.697084 (+801291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13717248}
I20260709 13:05:32.806773 29066 rpcz_store.cc:276] 0709 13:05:30.527699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.465433 (+1937734us) service_pool.cc:224] Handling call
0709 13:05:32.465795 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.466143 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:32.636726 (+170583us) log.cc:844] Serialized 14 byte log entry
0709 13:05:32.712318 (+ 75592us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:32.712397 (+    79us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:32.712466 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.712575 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.712743 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":168694,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":1209,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":5868,"thread_start_us":216483,"threads_started":2}
W20260709 13:05:32.810850 29169 debug-util.cc:398] Leaking SignalData structure 0x7b0800221d40 after lost signal to thread 27937
W20260709 13:05:32.812093 29169 debug-util.cc:398] Leaking SignalData structure 0x7b080015da40 after lost signal to thread 29005
I20260709 13:05:32.812355 32354 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:32.813527 29169 debug-util.cc:398] Leaking SignalData structure 0x7b08003e1100 after lost signal to thread 29097
W20260709 13:05:32.815898 29169 debug-util.cc:398] Leaking SignalData structure 0x7b08003e1080 after lost signal to thread 29098
W20260709 13:05:32.817035 29169 debug-util.cc:398] Leaking SignalData structure 0x7b08003b44c0 after lost signal to thread 29172
W20260709 13:05:32.817353 29169 debug-util.cc:398] Leaking SignalData structure 0x7b0800058a60 after lost signal to thread 29173
W20260709 13:05:32.818648 29169 debug-util.cc:398] Leaking SignalData structure 0x7b08003f0400 after lost signal to thread 29246
W20260709 13:05:32.818933 29169 debug-util.cc:398] Leaking SignalData structure 0x7b08003d9be0 after lost signal to thread 29247
I20260709 13:05:32.819190 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:32.820863 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.822373 29218 raft_consensus.cc:2410] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:05:32.823099 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:32.824054 29145 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:32.827028 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.827275 32354 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.827883 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:32.828020 32354 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:32.828599 29217 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:32.827335 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.828709 32354 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.829937 32354 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:32.831377 29169 debug-util.cc:398] Leaking SignalData structure 0x7b08001b5180 after lost signal to thread 29935
W20260709 13:05:32.832113 28967 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 239) took 31667 ms (client timeout 15000 ms). Trace:
I20260709 13:05:32.833029 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.834014 29909 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:32.732661   608 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:32.827462 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:32.840560 29142 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:32.838945 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:32.841085 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.841423 29213 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:32.718505   813 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:32.858508 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2112) took 2330 ms. Trace:
W20260709 13:05:32.865669 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 88862.8us
I20260709 13:05:32.858816 29910 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:32.761058   895 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:32.860761 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 122127us
I20260709 13:05:32.867453 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:32.861702 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 97266.7us
I20260709 13:05:32.857286 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.825852   963 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:32.761277   916 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:32.878439 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:32.820279   883 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:32.882441 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:32.884882 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:32.887547 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.879416 32354 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:32.891063 29143 raft_consensus.cc:2376] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9. Re-sending same reply.
I20260709 13:05:32.894136 29145 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:32.903596 29039 leader_election.cc:400] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11.
I20260709 13:05:32.904735 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:32.914597 32354 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:32.859510   630 consensus_queue.cc:1243] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:05:32.832660 28967 rpcz_store.cc:269] 0709 13:05:01.164453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.164630 (+   177us) service_pool.cc:224] Handling call
0709 13:05:32.832011 (+31667381us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:26.491278 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:26.504470 (+ 13192us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:26.504519 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:05:26.504538 (+    19us) tablet.cc:662] Decoding operations
0709 13:05:26.511055 (+  6517us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:26.511156 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:26.511183 (+    27us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:05:26.512038 (+   855us) tablet.cc:701] Row locks acquired
0709 13:05:26.512086 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:05:26.512266 (+   180us) write_op.cc:270] Start()
0709 13:05:26.512386 (+   120us) write_op.cc:276] Timestamp: P: 1783602326512202 usec, L: 0
0709 13:05:26.512402 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:05:26.513077 (+   675us) log.cc:844] Serialized 3626 byte log entry
0709 13:05:26.979244 (+466167us) op_driver.cc:464] REPLICATION: finished
0709 13:05:27.441504 (+462260us) spinlock_profiling.cc:243] Waited 40.1 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f7833a030ac 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f783a543ef0
0709 13:05:27.869386 (+427882us) write_op.cc:301] APPLY: starting
0709 13:05:27.869512 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:27.870331 (+   819us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:27.870395 (+    64us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:27.871285 (+   890us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:27.871577 (+   292us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:05:27.871625 (+    48us) write_op.cc:312] APPLY: finished
0709 13:05:27.921741 (+ 50116us) log.cc:844] Serialized 71 byte log entry
0709 13:05:28.373218 (+451477us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:28.403634 (+ 30416us) write_op.cc:454] Released schema lock
0709 13:05:28.428918 (+ 25284us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:28.472991 (+ 44073us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:28.485234 (+ 12243us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":504752,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":861569,"apply.run_cpu_time_us":65558,"apply.run_wall_time_us":592456,"num_ops":9,"prepare.queue_time_us":84553,"prepare.run_cpu_time_us":33936,"prepare.run_wall_time_us":223768,"raft.queue_time_us":147540,"raft.run_cpu_time_us":32974,"raft.run_wall_time_us":952244,"replication_time_us":432930,"spinlock_wait_cycles":92117888,"thread_start_us":1481415,"threads_started":5,"wal-append.queue_time_us":686404,"wal-append.run_cpu_time_us":13217,"wal-append.run_wall_time_us":2226047}]]}
I20260709 13:05:32.914358 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:32.859688   878 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.929145 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:32.929653 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:32.877528   813 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:32.879323   900 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:32.914264   806 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:32.889853   963 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.989425 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:32.991460 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 84291.3us
I20260709 13:05:32.980793 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:32.864149   873 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 13:05:32.968652 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:32.999008 29222 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:32.889978   615 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:32.880256   895 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:32.909865   916 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:32.979645 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:32.965260 32354 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:32.967492 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62852.9us
I20260709 13:05:33.012612 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:33.002830 29142 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:33.003702 29213 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:33.015780 29222 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:33.020325 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:32.922480   942 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:05:32.979101   369 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:32.981956   630 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:32.973878   877 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:05:33.024350 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:33.028712 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:32.993674   963 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:32.865564 29068 rpcz_store.cc:276] 0709 13:05:30.528306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.474548 (+1946242us) service_pool.cc:224] Handling call
0709 13:05:32.474872 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.475074 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:32.692333 (+217259us) log.cc:844] Serialized 14 byte log entry
0709 13:05:32.858134 (+165801us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:32.858192 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:32.858244 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.858325 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.858449 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96172,"prepare.queue_time_us":160029,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":97064,"thread_start_us":285923,"threads_started":2}
W20260709 13:05:33.021647 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 326738us
W20260709 13:05:33.098780 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 74623.8us
I20260709 13:05:33.099166 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:33.007236   615 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.009835   892 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.011565   895 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:33.021004   608 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:05:33.039891   870 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:33.017810   673 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:05:32.975287   878 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.114113 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:33.114768   972 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:05:33.115720   878 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:33.116221   880 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:33.116321 29217 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:33.116770   880 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:33.117621   969 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:05:33.117688   880 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.120002   888 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:33.120225 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.106980   615 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:33.121783 29143 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:05:33.122145 29143 pending_rounds.cc:85] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 7
I20260709 13:05:33.122401 29143 pending_rounds.cc:107] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:05:33.122674 29143 raft_consensus.cc:2889] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:33.124643   983 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:33.112808   877 consensus_queue.cc:1243] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:05:33.098806 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 279680us
I20260709 13:05:33.103600 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:33.105569 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 76917.9us
I20260709 13:05:33.030977   711 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:32.997000   909 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:33.023586   951 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:33.031431   369 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.991704   976 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.125s
W20260709 13:05:33.110603 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 89106.6us
I20260709 13:05:33.039059   630 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:33.040448   928 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.165598 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:33.134747 29217 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:33.123984   888 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:33.168987 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:33.168632 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 43607.2us
W20260709 13:05:33.176770   966 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:05:33.168717   880 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:33.165861 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:33.110733   870 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:33.213137   870 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.179440 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:33.214165   870 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:33.184265   630 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:33.216056 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:33.175714   909 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.217657   623 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:33.218487   623 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:33.218914   623 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:33.219292   623 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:33.219394   909 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:33.186421 32354 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:33.275861 32354 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:33.195006   948 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:33.195204 29902 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:05:33.196305   981 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 13:05:33.201591   974 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:05:33.204393   951 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.211556   888 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.219954 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:33.220844   979 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:33.169327 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:33.188915   976 consensus_queue.cc:1243] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:05:33.182144   983 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:33.184819   711 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:33.229369   623 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.231323   928 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:33.232790 29139 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:33.237726 29142 raft_consensus.cc:2410] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:05:33.238880 27939 debug-util.cc:398] Leaking SignalData structure 0x7b080000fe20 after lost signal to thread 29875
I20260709 13:05:33.164906   985 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:33.179209   369 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.242754 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:33.243578 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.243906 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:33.243602 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:33.245812 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.246977   608 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:33.186363   776 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:33.251535 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:33.256866   884 consensus_meta.cc:287] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.599s	user 0.010s	sys 0.000s
I20260709 13:05:33.265416 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:33.267165 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:33.339113 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 72074.1us
I20260709 13:05:33.271351 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:33.340229 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:33.340854 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 69631.2us
I20260709 13:05:33.277354 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:33.279781 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 180672us
I20260709 13:05:33.280392 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:33.342960 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 65709.4us
W20260709 13:05:33.343307 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 63002us
I20260709 13:05:33.281217 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:33.344460 29217 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.281977 32354 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.287748 29140 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:33.290098 29875 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.562s (SENT) [suppressed 28 similar messages]
W20260709 13:05:33.311050 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 93869.8us
I20260709 13:05:33.346205 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:33.316601   989 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:33.290280   979 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:33.316875   711 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.318290   983 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.336190   776 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:33.331226 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:33.361572 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:33.362682 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:33.362958 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:33.363595 29214 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
W20260709 13:05:33.336122 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 236200us
I20260709 13:05:33.363987 29218 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:33.349184 29141 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:33.282553   984 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:05:33.265457   813 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.354067 29147 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:05:33.356016 29186 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:33.366107   884 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:33.352970   985 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:33.368083   947 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:33.370347   512 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:33.372404 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:33.372963 29068 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:05:33.376228 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:33.390944 29222 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:33.391291 29070 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: [6.6-7.7]
I20260709 13:05:33.377815 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:33.379117 29217 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:33.355780   989 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:33.355630   878 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:33.384677   961 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:33.353219   928 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:33.355296   948 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:33.392838 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.409118 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.393082 29219 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:33.409613 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.376016   949 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.370810   984 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:33.397647 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.398707 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.401294 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:33.402266 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.438031 29073 raft_consensus.cc:2410] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:05:33.403345 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.438984 29066 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:33.400306 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:33.410531 29067 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:33.415899 29140 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:33.442986 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2114) took 2868 ms. Trace:
I20260709 13:05:33.433693   994 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:33.423333   900 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:33.429764 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:33.430751 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:33.447862 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 46692.4us
I20260709 13:05:33.449445 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:33.432150   979 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.443406 29147 rpcz_store.cc:276] 0709 13:05:30.574508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.346659 (+1772151us) service_pool.cc:224] Handling call
0709 13:05:32.346961 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:33.374998 (+1028037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.442732 (+ 67734us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003a57d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:33.442925 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2315094784}
I20260709 13:05:33.455586 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:33.456157 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 166101us
I20260709 13:05:33.460472 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:33.464939 29065 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:33.396379 32572 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:33.406862   961 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:33.413261   955 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:33.435173   978 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.480780 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:33.482882 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 71790.6us
I20260709 13:05:33.407588   989 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:33.499349 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:33.499460 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.413753   884 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:33.498178   948 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:33.441390   949 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.509568   979 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:33.509465   991 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:05:33.510603   991 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:33.511010   991 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:33.434352   984 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:33.519006 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:33.455233   877 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.460057   977 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.469926   876 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:33.455844   900 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:33.462435   878 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:33.455108   976 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:33.472533 32572 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.541379 29213 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:05:33.506412   961 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:33.547744 29064 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:33.442533   990 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 13:05:33.516536   584 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:33.583812 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 64952.6us
I20260709 13:05:33.550257   876 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:33.479207   776 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.553237   900 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:33.582435 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 174137us
I20260709 13:05:33.575277 32572 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.548332   879 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:05:33.529861   895 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.548928   525 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:05:33.540424   877 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.595304   982 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:33.603191   876 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.602006   892 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:33.623320 29145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:39218
W20260709 13:05:33.625240 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.554565   630 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.633860 29145 raft_consensus.cc:606] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:33.634814 29145 raft_consensus.cc:614] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:33.605711   999 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.520412   978 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.642655 29148 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:33.553985   977 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.593081   984 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.526147   985 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.617831   884 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.580562   976 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:33.642207   615 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.002s
I20260709 13:05:33.638581   892 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:33.714316   615 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.714956   892 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.716189   892 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:33.694805 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:33.718199   883 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.788s	user 0.008s	sys 0.003s
I20260709 13:05:33.718624   883 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.652366   949 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.655530   776 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.588096   991 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.721505 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:33.721974   991 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.722738 29213 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:33.626341   982 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:33.687400 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:33.724468   977 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.724665   982 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.725751   982 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.687820 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:33.676887 29222 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:33.676226 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:33.639146   623 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:33.731202 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:33.737422 29219 raft_consensus.cc:2410] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:05:33.737221 29065 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:33.645531 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:33.739934 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 142561us
I20260709 13:05:33.740964 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:33.756496 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 228783us
W20260709 13:05:33.668442   963 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.000s
W20260709 13:05:33.679198  1001 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:05:33.693464   996 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:05:33.692759   995 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:05:33.776584 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.777362 29072 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:33.766786 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.768628 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:33.769387 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.693301   720 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:33.757606   608 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:33.782065 29073 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:33.770695 29064 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
W20260709 13:05:33.747184   916 consensus_meta.cc:287] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.708s	user 0.008s	sys 0.003s
I20260709 13:05:33.771143 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:33.669723   877 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.677317   630 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:33.772836   963 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.671658   813 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:33.743564   870 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.000s
I20260709 13:05:33.774324 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.716694   615 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.669584   978 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.775828   998 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:33.780817   720 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:33.784559 29148 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9574de46fcd490e84208e41907200b0"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:05:33.795979 29148 raft_consensus.cc:606] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:33.796413 29148 raft_consensus.cc:614] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:33.680336   783 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 13:05:33.725728 32572 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.784458   608 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.801997 29146 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:33.802589 29066 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:33.790875 29072 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:05:33.783166 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.793460 29067 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:33.794929 29068 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:05:33.799292 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.787189   883 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.781018   949 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.804937 29064 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:33.806668 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.829285 29145 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:33.830237 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.806972 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:33.831373 29147 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:33.832046 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:33.800212   948 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:33.809006 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:33.809695 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.809826 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:33.810530 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.835691 29143 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:33.810662 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:33.836067 29139 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:33.810982 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.836738 29072 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:33.838322 29072 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:33.811903 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.813019 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.813274 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.815711 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:33.839241 29069 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:33.820787 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.821584  1012 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:33.823066   525 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:33.790282   916 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.814172   877 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.807585 29140 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:05:33.832829 29219 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:33.830833 29145 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:33.845989 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.846517 29145 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:33.847127 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.840128 29070 raft_consensus.cc:2410] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:05:33.814553   608 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.795667   998 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:33.859983 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.860177 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.860661 29140 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:33.860913 29146 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:33.861583 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.862073 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:33.848721 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.848485 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.863332 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.863786 29148 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:33.864660 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.864732   740 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:33.865161   740 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:33.865356 29148 raft_consensus.cc:2410] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:05:33.866389 29148 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:33.866086   740 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.867426   740 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:33.870790 29146 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:33.849226 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.871424 29143 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:33.871992 29146 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:33.851696   916 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:33.873356   963 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:33.847965   947 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.874612   947 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.876803   999 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:33.878276   947 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.879120 29142 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 13:05:33.843663   369 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.004s
I20260709 13:05:33.884800   369 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.886797   369 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:33.852402 29215 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 9: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:33.852927 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:33.853099   641 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:33.889053   641 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:33.890026   641 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.891374   641 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.853179   976 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.854574 29222 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:05:33.855805   877 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.893939   877 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.857256 32572 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.858459   525 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:33.894469   876 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.895198   525 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.896534 29144 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:33.898687   895 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:33.899263   900 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.834957   949 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:33.900557   948 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.901870   900 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:33.902045   949 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:33.902606 29139 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:33.904663 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:33.860114   998 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:33.905821   984 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:33.798058   584 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.910652   584 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.848754 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:33.852053  1004 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:05:33.883482   720 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:33.862178 29140 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:33.914834 29068 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:33.867599 29148 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:33.920733 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:33.921382 29217 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:33.921759 29147 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:33.921519 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:33.922793 29215 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:33.919579 29219 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:33.924342 29140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:05:33.926026 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:33.926856 29213 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.929359 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:33.930876 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:33.925158 29140 raft_consensus.cc:606] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:33.932734 29140 raft_consensus.cc:614] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:33.938493 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:33.940064 29145 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:33.943058 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:33.943668 29148 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:33.944023 29067 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:33.946193 29910 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:33.949904 29113 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:33.960263   900 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:33.961179   900 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:33.961494   900 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:33.959890 29213 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:33.962996 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:33.964879 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 179378us
I20260709 13:05:33.963017 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.965744 29147 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:33.968350 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:33.970474 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:33.973618 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.976254   584 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:33.981443 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:33.981685 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.982611 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:33.982940 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:33.983564 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:33.984042 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:33.985167 29904 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:33.956983  1017 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.113s	user 0.003s	sys 0.000s
I20260709 13:05:33.985817 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:33.986734 29908 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:33.988631 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.999432 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:33.999709 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:34.000193 29904 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:33.987576 29141 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:05:33.998745 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:33.999862 29139 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:33.997573 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:34.001461 29148 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:34.002993 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:34.006495 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:33.985293   979 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:05:34.017127 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:34.008394 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.008221 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.017652 29067 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:34.019594 29140 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:33.978317   720 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:34.008934 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:34.009127 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:34.009377 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:34.021409 29142 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:34.013479 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:34.015094 29146 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:34.010257 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.024803 29148 raft_consensus.cc:2410] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:33.957587   877 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:05:33.937196   961 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.029564 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:34.030831 29064 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:34.124358   989 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.600s	user 0.012s	sys 0.000s
W20260709 13:05:34.177767 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.182163 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.017017   625 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:05:34.083757   608 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.082756 29903 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:34.189908 29140 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:34.189638 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:34.190699 29067 raft_consensus.cc:2376] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8. Re-sending same reply.
W20260709 13:05:34.189661 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.199260 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:34.201650 29069 raft_consensus.cc:2368] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12.
I20260709 13:05:34.204337 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:34.163218 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.205142 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:34.140755 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:34.206038 29073 raft_consensus.cc:2410] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:05:34.140759 29071 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:33.962395   900 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.083508 29219 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:34.028707   982 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:34.002445   947 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:34.004859   697 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:34.015321  1016 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:34.053046 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.007968  1009 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.137s
I20260709 13:05:34.180899   989 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:34.154059  1019 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.117s	user 0.003s	sys 0.000s
I20260709 13:05:34.173909 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.049064   740 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:34.183241 29142 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:34.199447 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:34.235028 29910 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:34.204155 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:34.234326 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.099872 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:34.206895 29145 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:34.122665 29143 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:34.215641 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:34.182387   998 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.222707 29144 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:34.084868   876 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:34.223592  1009 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:34.229164 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.227547 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.229729 29140 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:05:34.232724 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.237493 29213 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:34.228161   697 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.239584 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.259033 29145 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:34.236155   947 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:34.225647   982 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:34.262632  1022 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:05:34.262842   982 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.264396 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.264930   783 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:34.265202 29068 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:34.265323   783 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:34.266232 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:34.244252 29871 leader_election.cc:336] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.994s (SENT)
I20260709 13:05:34.225250  1016 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.256562 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.273173 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.274698 29072 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:34.275820 29213 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:34.276304 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:34.277560 29068 raft_consensus.cc:1217] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[4.3-11.5]   Dedup: 8.4->[11.5-11.5]
I20260709 13:05:34.278309 29068 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: [4.3-11.5]
I20260709 13:05:34.256788 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:34.280390 29142 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:34.257452 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:34.224743   900 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:34.256254 29146 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:34.260083 29065 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:34.258518   877 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:34.261520   947 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.243860 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.293342 29143 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:34.293165 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.294155 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.267292 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.267691 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.268724 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.295212 29069 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:34.269039 29910 raft_consensus.cc:2410] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:34.271142 29064 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:34.296644 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.298151 29143 raft_consensus.cc:2410] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:34.298681 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.254647   990 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:34.285071 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:34.282876 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:34.288255 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:34.291551 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.287217 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.299719 29144 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:34.300942 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:34.310176 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:34.255546   961 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:34.312711   625 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:34.259578   584 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:05:34.314332   608 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:34.314749   998 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:34.315326 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:34.316107   991 consensus_meta.cc:287] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.594s	user 0.013s	sys 0.000s
I20260709 13:05:34.307260 29146 raft_consensus.cc:2376] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12. Re-sending same reply.
W20260709 13:05:34.317070  1026 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:05:34.290680   892 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.319077   892 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.320703 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:34.321514   892 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:34.322808   737 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:34.323501   737 consensus_queue.cc:1243] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:05:34.324116 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.316051 29070 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
W20260709 13:05:34.326581 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 313476us
I20260709 13:05:34.299459 29064 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:34.259155  1009 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:34.330582 29215 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:34.332185 29072 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:34.332870 29066 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:34.328549 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:34.341562 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.342275 29069 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:34.344972 29875 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:34.349485 29066 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:34.291153   955 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:34.290251  1016 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:34.354480 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:34.352598 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:34.361364 29905 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
W20260709 13:05:34.361685 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.362015 29219 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:34.367401 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:34.368533 29908 raft_consensus.cc:2410] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:05:34.350656 29142 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:34.344377   886 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:34.370072   886 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:34.370452   886 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:34.344509   977 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:34.374120  1016 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.374751   977 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:34.375195   977 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:34.376374  1016 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:34.379802 29019 auto_rebalancer.cc:893] Removing replace marker failed: Replica 730186a8c70f43f2a57d80163bbc37d2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } } pending_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:05:34.380467 29019 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:05:34.380860 29019 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:05:34.382090 29902 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:34.343427   870 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:05:34.386358   884 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:34.386818   884 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:34.387187   884 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:34.329716   963 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:34.344786   978 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:05:34.393275   989 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:34.400614 29139 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:34.395602 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:34.402848 29066 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:34.362823 29070 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:34.363580 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:34.403575 29070 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:34.402848 29142 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:34.406916 29142 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:05:34.407334 29142 pending_rounds.cc:85] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 9
I20260709 13:05:34.407434 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:34.407650 29142 pending_rounds.cc:107] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:05:34.408054 29142 raft_consensus.cc:2889] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:34.404314 29070 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.412251 29070 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:34.408622   951 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:34.408985   979 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:34.409044 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56678.4us
I20260709 13:05:34.409799  1011 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:34.348141   876 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:34.419242 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:34.401033  1020 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:34.415906 29909 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:34.468924 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:34.298394   877 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.470439  1020 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.471520  1020 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:34.472020   884 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.464165   876 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:34.467111 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:34.473111   951 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:34.474042 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 56461.8us
I20260709 13:05:34.475509   877 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:34.586205 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 119170us
I20260709 13:05:34.482075   697 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:34.419201   949 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:34.484144   916 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:34.612360 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.613029 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.489861 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.492380  1020 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.496122 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:34.514323 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.535774 29072 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:05:34.364307   711 consensus_queue.cc:1059] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:05:34.616143 29905 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:05:34.562979 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59849.7us
W20260709 13:05:34.583959 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 165296us
I20260709 13:05:34.618584 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:34.620654 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:34.609432 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:34.610306   870 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.485456   873 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:34.486733 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:34.613792 29070 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:34.553570   886 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.625022 29146 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:34.631580 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.555256   641 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:05:34.582356   369 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:34.594058   525 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:34.467628  1009 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.640995 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:34.641641 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.408145   783 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:34.643862 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 157252us
I20260709 13:05:34.651223 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.659000 29072 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:34.652199 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:34.633405   978 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:34.627300   949 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:34.624715   697 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:34.665062  1020 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:34.665159   697 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.656419   977 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.625443   916 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.633119   955 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:05:34.672693 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.675164 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:34.676497 29068 raft_consensus.cc:2410] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:05:34.623823 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:34.659751 29065 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:34.656776 29066 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:34.633253   948 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:34.672611 29070 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:05:34.634258   873 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:34.684844 29142 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:34.642251   870 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.661379   978 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:34.692452 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 386797us
I20260709 13:05:34.661737   584 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:34.708854  1029 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:05:34.686833   525 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:05:34.709408   886 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:34.670856   990 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:34.685528   948 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.687081   369 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:34.712286 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:34.713217 29909 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:34.714892 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:34.661978   888 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
W20260709 13:05:34.679209 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55486.6us
I20260709 13:05:34.710335   783 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:34.708904  1020 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:05:34.768265 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:34.772528   984 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:34.772632   985 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:34.773649   985 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:34.773991   985 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:34.730397   949 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.774768   985 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.778429   525 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:34.779230   525 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:34.780725 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:34.782699   884 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:34.772677   870 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:34.784716   525 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.791019   697 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:34.792018   641 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:34.792328 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:34.792517   641 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:34.793172   884 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:34.793619 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:34.793474   641 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.794452   884 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:34.794836   884 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.794988   641 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:34.795188   884 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:34.797766 29039 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.020s	sys 0.029s
W20260709 13:05:34.798147 29039 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.020s	sys 0.029s
W20260709 13:05:34.798413 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 770836us
I20260709 13:05:34.671255   876 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.786110 29069 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:34.767390 29902 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:34.737838   873 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:34.803117 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:34.817778 29143 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
W20260709 13:05:34.818163 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.818426 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:34.818949 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:34.797686   984 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:34.804044   963 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:34.808013 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:34.819696 29143 raft_consensus.cc:2376] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8. Re-sending same reply.
W20260709 13:05:34.823382 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 233323us
I20260709 13:05:34.824076 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:34.820927 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 162754us
I20260709 13:05:34.755713   978 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:34.770160   948 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:34.832684   870 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:34.826013 29143 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:34.814721   985 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:34.820313  1016 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.859759 29902 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:05:34.823628   584 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:05:34.853104   990 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:34.868423   977 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:34.860718  1036 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:34.862506  1011 consensus_queue.cc:1059] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:05:34.861876   720 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:05:34.875612   955 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:34.868438  1016 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:34.900043  1030 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.408s	user 0.003s	sys 0.003s
W20260709 13:05:34.904981   900 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.609s	user 0.008s	sys 0.004s
I20260709 13:05:34.904981   916 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:34.906432  1016 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:34.907158   955 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.898275  1049 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:34.912421  1049 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.914710  1049 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:34.876827   873 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.916379   873 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:34.920449  1033 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:05:34.920341   948 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.921211  1036 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.921731  1036 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.922927   948 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:34.924120   916 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:34.925601   916 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:34.929162  1046 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:34.934938   884 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.936499  1049 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:34.936982  1049 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:34.936690   978 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:34.937871  1049 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:34.939107  1036 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:34.939267   978 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:34.944424   884 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:34.946377  1009 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:34.987076 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.995138 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 184716us
W20260709 13:05:35.005064 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.005594 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.997537 29067 raft_consensus.cc:2410] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:05:35.011643 29065 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:34.998577 29144 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:05:35.012259 29065 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:35.013921 29065 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.017436 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:35.018298 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 199809us
I20260709 13:05:35.019168 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.019586 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.020905 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:35.021178 29220 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:35.021215 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.022109 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:35.001315 29147 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.015s	sys 0.035s
W20260709 13:05:35.042335 29147 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.015s	sys 0.035s
I20260709 13:05:35.022225 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.074210 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:35.075510 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 642) took 1068 ms. Trace:
I20260709 13:05:35.076112 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:35.077062 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:35.081955 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:35.083096 29213 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:35.083483 29065 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:35.084089 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:35.075529 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:35.079541 29220 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:35.073197 29219 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:35.079375 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:35.105993 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:35.107787   985 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.108860   985 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:35.111042   979 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:35.112452   584 consensus_queue.cc:1243] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:05:35.112636 29071 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:35.113628 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:35.114835 29220 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:05:35.095324   999 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:35.142262 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.130029 29219 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:35.116830 29220 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:35.117878 29070 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:35.121260 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:35.125550 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.121906 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.115839 29217 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:35.122645 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.159957 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:35.121644   886 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:05:35.156756   999 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:35.139129  1036 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:05:35.141420 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:35.129597 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:35.128341 29064 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:35.133352 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:35.116153 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 243161us
I20260709 13:05:35.168727 29140 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:35.170208 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:35.132674 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:35.126349 29142 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:35.173269 29143 raft_consensus.cc:2410] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:05:35.173857 29185 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:35.175037 29143 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:05:35.175724 29217 raft_consensus.cc:1217] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 5.7->[8.8-13.11]   Dedup: 8.8->[10.9-13.11]
I20260709 13:05:35.181864 29217 pending_rounds.cc:85] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 8
I20260709 13:05:35.176231   985 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:35.076258 29147 rpcz_store.cc:276] 0709 13:05:34.007463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.007611 (+   148us) service_pool.cc:224] Handling call
0709 13:05:34.007959 (+   348us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:34.008410 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.008439 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:34.008999 (+   560us) log.cc:844] Serialized 52 byte log entry
0709 13:05:34.218638 (+209639us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b180013b5e0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:05:34.352227 (+133589us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:34.821482 (+469255us) spinlock_profiling.cc:243] Waited 6.44 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:35.074336 (+252854us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.074929 (+   593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.075069 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:35.075172 (+   103us) raft_consensus.cc:1685] finished
0709 13:05:35.075186 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.075431 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65,"raft.queue_time_us":5278,"raft.run_cpu_time_us":612,"raft.run_wall_time_us":17416,"spinlock_wait_cycles":22657664,"thread_start_us":767539,"threads_started":2,"wal-append.queue_time_us":461888}
W20260709 13:05:35.179699 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 74084.3us
I20260709 13:05:35.133682 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:35.175796 29218 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:05:35.182581 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:35.192204 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:35.192701 29214 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:35.181285 29143 pending_rounds.cc:85] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 9
I20260709 13:05:35.130226 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.182148 29217 pending_rounds.cc:107] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:05:35.130324 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:35.208673 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 104675us
I20260709 13:05:35.203715 29217 raft_consensus.cc:2889] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:35.215319  1012 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:35.134773   525 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:35.217142  1012 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:35.219101 29216 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
I20260709 13:05:35.167754   999 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:35.225041 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.225973 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:35.226807 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:35.226945   720 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:35.230505 29143 pending_rounds.cc:107] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:05:35.246726 29143 raft_consensus.cc:2889] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:35.231006 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:35.234175 29910 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:35.238374 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:35.239302   783 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:35.241757   873 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:05:35.241948 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 108578us
I20260709 13:05:35.242390 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:05:35.243376 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 69740.8us
I20260709 13:05:35.230036   985 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:35.244065  1056 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:35.241974  1055 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:35.270605   873 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:35.277730 32572 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:35.278771 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:35.283897 29065 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:05:35.288815 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 336042us
I20260709 13:05:35.288913   873 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.289585 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:35.290629   883 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.291256   873 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:35.286931 32572 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:35.292205  1036 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:35.293185 32572 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:35.294068 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 51927.1us
I20260709 13:05:35.294502 32572 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:35.296380   900 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:35.296973   900 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:35.297293 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:35.297554   900 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:35.299052 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:35.300312 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:35.282824  1055 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:35.291345   883 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.322866 29220 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:35.291842 29185 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:35.280543   985 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:35.304513  1055 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:35.336654 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 45082.7us
I20260709 13:05:35.337728 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:35.316253  1056 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:35.326350   720 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:35.326861   985 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:35.342029 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.324753  1036 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:35.349715 29220 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:05:35.350831   999 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:35.351536   999 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.353855   999 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:35.357264   990 raft_consensus.cc:2764] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:05:35.358408   641 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.563s	user 0.015s	sys 0.005s
W20260709 13:05:35.359349 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 131123us
I20260709 13:05:35.359997   916 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:35.360888 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62058.6us
I20260709 13:05:35.364797 29067 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:05:35.360972   883 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:35.367542 29214 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:35.361804   990 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:05:35.368819 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:35.368634   990 consensus_queue.cc:1243] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:35.370258 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:35.367450 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:35.370739   711 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:35.366746 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.341981   720 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:35.375284   720 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:35.376745   720 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:35.377210 29215 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:35.378302 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:35.378340 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.379335 29216 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:05:35.379659   984 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:35.380643   984 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.381968 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:35.383577   916 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:35.384294   916 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:35.385492   916 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.389243 29067 pending_rounds.cc:85] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 10
I20260709 13:05:35.389719 29067 pending_rounds.cc:107] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:05:35.389990 29067 raft_consensus.cc:2889] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:35.300446   697 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:05:35.395794 29219 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:35.400637 29139 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:05:35.411355 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 65181.3us
I20260709 13:05:35.398311 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:35.417111 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 121554us
I20260709 13:05:35.406250 29220 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:35.391906   876 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:35.398106  1063 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:35.425722 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:35.405512   984 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:35.448108  1063 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:35.448359 29073 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:35.449748  1063 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.452262 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:35.453200   697 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:35.427990   886 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.460441   886 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:35.461189 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.462891   886 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:35.465534   979 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:35.466209  1056 raft_consensus.cc:2764] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:05:35.467195 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 58639.8us
I20260709 13:05:35.467409 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:35.468070   883 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:35.450364 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:35.469565 29072 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:35.467718   979 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:05:35.422014   876 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:35.472970   876 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:35.474494   876 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:35.468175 29070 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:35.484937  1049 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:35.485493 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.486006 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.488535 29071 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:35.488950 29071 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:35.489316 29143 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:35.488950 29068 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:35.489071 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:35.490694   979 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.491501   979 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.492281 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:35.493825 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:35.494189 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 52576.7us
I20260709 13:05:35.495486   697 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:35.495980   697 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.495853 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:35.496649   697 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:35.497180  1036 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:35.500149  1068 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:35.502696 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.511121   883 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:35.468364 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:35.534369 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 58110.2us
I20260709 13:05:35.541993 29218 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
W20260709 13:05:35.568113 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.579083 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 150766us
W20260709 13:05:35.580575  1052 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.364s	user 0.001s	sys 0.008s
I20260709 13:05:35.581456   984 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:35.583117   984 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:35.586719 29068 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:05:35.589466 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:35.590986 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:35.591991 29066 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 9: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:35.592327 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:35.593279 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:35.591015 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 95358.2us
I20260709 13:05:35.594902 29218 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:35.595136 29185 leader_election.cc:400] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12.
I20260709 13:05:35.595434 29185 leader_election.cc:403] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:35.596501 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:35.597446  1046 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12.
I20260709 13:05:35.600656 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:35.601984 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:35.603499  1046 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:35.602728 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:35.605230  1057 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:35.605677  1057 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:35.606350 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:35.606988  1057 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.579141 29073 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:35.608821  1054 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:35.609017   979 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:35.609272  1054 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:35.609724  1054 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:35.609962 29874 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:35.611224 32572 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:35.612231   955 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:35.612751   955 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:35.613158   955 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:35.614573   900 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.620430   641 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.621188 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:35.622268   948 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:35.622985 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:35.624329   369 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:35.624711   369 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:35.625065   369 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:35.621049   641 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:35.628760 29113 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:35.629670 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 90684.3us
I20260709 13:05:35.629750   948 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:35.631634   948 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.631505   984 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:35.632115   948 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:35.632592 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 164374us
I20260709 13:05:35.633949 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:35.635560   584 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:35.636152   584 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.636624   584 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:35.637279 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.637684 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.638041 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.612331   985 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:35.611848 32572 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:05:35.604295   625 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:35.635615 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:35.660243 29215 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:35.660624 29145 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:35.662509 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:35.640040 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47880.9us
I20260709 13:05:35.588234 29071 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.656497 29072 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
W20260709 13:05:35.665609 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.670686 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:35.670888 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.674563 29068 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:35.677152  1057 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:35.663718  1068 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.680505  1068 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.681396 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:35.683312  1068 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:35.667022  1056 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:05:35.689358 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 83117.2us
I20260709 13:05:35.689697  1056 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:35.688146 29145 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:35.692416   641 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:35.689378   625 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:35.693810   625 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:35.688500   900 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:35.698035 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62527.4us
W20260709 13:05:35.699647 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.700488   886 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:35.701289   985 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:35.701324   886 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:35.704151 29069 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:35.704346   886 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:35.705397   886 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.706032   999 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:35.706699   886 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:35.708235   873 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.708774   873 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.710673   873 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:35.712834 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:35.668959 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:35.716034 29070 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:35.725999   979 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:05:35.730288 32572 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:35.730650  1067 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:35.731185  1067 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:35.731587  1067 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:35.731769 32572 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:35.732187 32572 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:35.733049 32572 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:35.734212 32572 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:35.735102   625 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.736135   900 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:35.746219   884 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:05:35.613072   883 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:35.752378 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:35.711509   584 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.691308  1056 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:35.768349 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:35.743649  1068 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.742746  1046 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:35.772843 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60137.3us
I20260709 13:05:35.760509   985 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.756919   979 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:35.761849   625 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:35.786046 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 127169us
I20260709 13:05:35.783685   999 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:35.789713 28971 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 162) took 39536 ms (client timeout 15000 ms). Trace:
I20260709 13:05:35.793417 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:35.741984   955 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.802812 29071 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:35.823807 29218 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:35.839675 32572 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:35.840559 32572 consensus_queue.cc:1243] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:35.843394  1068 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:35.845597 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.845952  1068 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:35.857115 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:35.791895   985 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:35.875538 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 124504us
I20260709 13:05:35.837530   984 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:35.790129 28971 rpcz_store.cc:269] 0709 13:04:56.253344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.253705 (+   361us) service_pool.cc:224] Handling call
0709 13:05:35.789424 (+39535719us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:32.807511 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:32.808362 (+   851us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:32.808413 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:05:32.808436 (+    23us) tablet.cc:662] Decoding operations
0709 13:05:32.809245 (+   809us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:32.809351 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:32.809392 (+    41us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:05:32.809909 (+   517us) tablet.cc:701] Row locks acquired
0709 13:05:32.809943 (+    34us) write_op.cc:260] PREPARE: finished
0709 13:05:32.810147 (+   204us) write_op.cc:270] Start()
0709 13:05:32.810291 (+   144us) write_op.cc:276] Timestamp: P: 1783602332810078 usec, L: 0
0709 13:05:32.810320 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:05:32.811125 (+   805us) log.cc:844] Serialized 1686 byte log entry
0709 13:05:33.664302 (+853177us) op_driver.cc:464] REPLICATION: finished
0709 13:05:33.881609 (+217307us) write_op.cc:301] APPLY: starting
0709 13:05:33.881729 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:34.411480 (+529751us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:34.411584 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:34.753954 (+342370us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:34.754256 (+   302us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:05:34.754325 (+    69us) write_op.cc:312] APPLY: finished
0709 13:05:34.754726 (+   401us) log.cc:844] Serialized 49 byte log entry
0709 13:05:34.896269 (+141543us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:34.898073 (+  1804us) write_op.cc:454] Released schema lock
0709 13:05:34.898130 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:34.904930 (+  6800us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:34.904961 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1506651,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":216917,"apply.run_cpu_time_us":66626,"apply.run_wall_time_us":1023345,"cfile_cache_hit":14,"cfile_cache_hit_bytes":14630,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6217,"cfile_init":2,"lbm_read_time_us":223412,"lbm_reads_10-100_ms":6,"lbm_reads_lt_1ms":6,"num_ops":4,"prepare.queue_time_us":9535,"prepare.run_cpu_time_us":11066,"prepare.run_wall_time_us":93458,"raft.queue_time_us":240114,"raft.run_cpu_time_us":11939,"raft.run_wall_time_us":192507,"replication_time_us":832912,"thread_start_us":544029,"threads_started":5,"wal-append.queue_time_us":748548,"wal-append.run_cpu_time_us":16884,"wal-append.run_wall_time_us":1601044}]]}
I20260709 13:05:35.910100   985 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:35.911217 29904 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:05:35.911541 29905 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:35.914132   948 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:35.917448  1061 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:35.887043   641 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:35.965257   584 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:35.989171 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:35.989534 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 308195us
W20260709 13:05:35.991482 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.981581 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:35.993353   985 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:35.994408   985 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:35.994779   985 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:35.996162  1009 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.928s	user 0.010s	sys 0.012s
I20260709 13:05:35.996857   990 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.997073 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:35.997191 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:35.997370   990 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:35.998708 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64824us
I20260709 13:05:36.000159  1068 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:36.000772 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:35.985754 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:35.985893   369 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:35.991659   876 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.008s
I20260709 13:05:35.900569  1036 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:36.000722  1068 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:35.984246   948 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:35.999800 29039 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:36.006172 29902 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
W20260709 13:05:35.938591  1055 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.593s	user 0.006s	sys 0.006s
I20260709 13:05:36.014652 29905 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:36.024324 29037 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:35.980777   641 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:36.016260   720 consensus_meta.cc:287] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.003s
I20260709 13:05:36.013281   999 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:36.044322 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:36.018024   900 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.938158   697 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.023605  1068 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:35.966269  1054 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.023864  1061 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:36.068001 29071 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:05:36.075604 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.043139   641 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:36.053771   916 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.011s	sys 0.023s
W20260709 13:05:36.083637 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 83948.9us
I20260709 13:05:36.084416 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:35.995755   985 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.118242 32572 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.121443 32572 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.068146   883 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:36.093578   916 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.012s	sys 0.026s
I20260709 13:05:36.031041  1055 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.133126 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:36.139361  1078 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 13:05:36.140440 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 336213us
I20260709 13:05:36.110198  1057 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:36.138953 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 356884us
I20260709 13:05:36.067729   999 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.068545  1067 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.133847   955 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:36.160641 29214 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:36.081861   900 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.204201 32572 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:36.093247  1061 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.216163 29905 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:36.217434 32572 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.218073 32572 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.160132   916 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:36.246953 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:36.249954  1075 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 13:05:36.250844   916 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.252115  1089 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:36.250876   985 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:36.252372 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:36.253425 29216 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:36.256415  1054 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:36.258647 32572 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:36.259759  1093 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:36.260015  1088 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:36.260313   916 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:36.260855 29216 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:05:36.263485   697 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:36.264163   369 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:36.264853   369 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:36.266781   984 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:36.267568   984 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:36.281788   900 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:36.284219  1061 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:36.286478  1061 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:36.286814 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:36.286787   947 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:36.287690 29905 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:36.289710 29113 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:36.290277 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:36.265061  1055 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:36.167203  1057 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.299961 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:36.298699 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:36.300163  1079 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
W20260709 13:05:36.315079 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 270866us
I20260709 13:05:36.317293   697 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:36.301111   916 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:05:36.324987   999 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:36.325248   641 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.323820  1068 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.297416  1088 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:36.334722 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:36.339962 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:36.340996 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:36.341574 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:36.341940 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.260525  1093 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.350987 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:36.317781  1055 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:36.357647 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 232841us
I20260709 13:05:36.340040   883 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:36.317924   900 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:36.339455  1088 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.343319  1068 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:36.361472 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:36.371444 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 318654us
I20260709 13:05:36.354420  1094 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:36.380805 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 46214.5us
I20260709 13:05:36.381608  1092 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:36.382287  1092 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:36.383224  1092 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.385002  1067 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:36.358366  1055 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:36.387357  1055 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:36.387660  1076 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.399s	user 0.002s	sys 0.000s
I20260709 13:05:36.389166  1057 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:36.372570  1088 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:36.390656  1085 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:05:36.350107   984 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.393939   641 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:36.395406  1074 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.552s	user 0.002s	sys 0.005s
W20260709 13:05:36.364355  1049 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.879s	user 0.010s	sys 0.004s
I20260709 13:05:36.397192 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2377) took 1386 ms. Trace:
I20260709 13:05:36.397136 29221 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:36.373303 29222 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:36.399003 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 693) took 1325 ms. Trace:
I20260709 13:05:36.408998 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:36.409895 29905 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:36.403177 29217 rpcz_store.cc:276] 0709 13:05:35.093660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.174732 (+ 81072us) service_pool.cc:224] Handling call
0709 13:05:35.175195 (+   463us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:05:35.245532 (+ 70337us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:35.340013 (+ 94481us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.763567 (+423554us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:35.782517 (+ 18950us) log.cc:844] Serialized 78 byte log entry
0709 13:05:35.782733 (+   216us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:35.783039 (+   306us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.785394 (+  2355us) log.cc:844] Serialized 14 byte log entry
0709 13:05:36.082746 (+297352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.082847 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.281639 (+198792us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b18000460c0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:36.386904 (+105265us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:36.398760 (+ 11856us) raft_consensus.cc:1685] finished
0709 13:05:36.398788 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.398940 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32824,"prepare.queue_time_us":271060,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":113,"raft.queue_time_us":343957,"raft.run_cpu_time_us":547,"raft.run_wall_time_us":70697,"spinlock_wait_cycles":74779008,"thread_start_us":768095,"threads_started":3,"wal-append.queue_time_us":618165}
I20260709 13:05:36.407094  1089 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:36.397763 29213 rpcz_store.cc:276] 0709 13:05:35.010840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.084181 (+ 73341us) service_pool.cc:224] Handling call
0709 13:05:35.084680 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:35.206034 (+121354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.206425 (+   391us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:35.318791 (+112366us) log.cc:844] Serialized 26 byte log entry
0709 13:05:35.569291 (+250500us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:35.569342 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.569442 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.394679 (+825237us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:36.396898 (+  2219us) raft_consensus.cc:1685] finished
0709 13:05:36.396919 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.397115 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80232,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":247,"spinlock_wait_cycles":23736192,"thread_start_us":149567,"threads_started":1,"wal-append.queue_time_us":418127}
I20260709 13:05:36.406491   948 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.425191 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:36.426268   948 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.428136  1105 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:05:36.402853   883 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:36.432700 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:36.417187  1093 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:36.453392 32572 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.454118 32572 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.456689  1068 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:36.458122  1093 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:36.463614 32572 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:36.466988 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.441300  1049 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:36.454245  1056 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.008s	sys 0.023s
W20260709 13:05:36.472832 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.482636  1094 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:36.488512   883 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:36.492100 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 67110.5us
I20260709 13:05:36.493235 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:36.494773 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:36.495955  1112 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:36.496521  1112 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.496608  1113 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:36.496968  1112 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:36.476758  1056 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.009s	sys 0.024s
I20260709 13:05:36.463164  1068 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:36.502102 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 298598us
W20260709 13:05:36.504354 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 139122us
I20260709 13:05:36.474232  1049 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:36.500795 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 95383us
W20260709 13:05:36.506129 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 98062us
I20260709 13:05:36.514303 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:36.516086 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:36.518175 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:36.519017 29215 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:05:36.524395 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.513952  1049 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:36.530879 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:36.532823 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.534057 29143 raft_consensus.cc:1217] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 7.8->[8.9-14.12]   Dedup: 8.9->[11.10-14.12]
I20260709 13:05:36.534579 29143 pending_rounds.cc:85] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 9
I20260709 13:05:36.534714 29143 pending_rounds.cc:107] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:05:36.534845 29143 raft_consensus.cc:2889] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:36.539013 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 697) took 1017 ms. Trace:
I20260709 13:05:36.539489 29214 rpcz_store.cc:276] 0709 13:05:35.521229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.521357 (+   128us) service_pool.cc:224] Handling call
0709 13:05:35.572278 (+ 50921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.207294 (+635016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.538757 (+331463us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b54003552d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:36.538945 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1352334592}
I20260709 13:05:36.532442   948 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:36.541311  1055 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.542136   947 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:36.542652   947 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:36.545324 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.546155   900 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:36.546736   900 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.548138   884 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:36.548274 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:36.548588   884 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.548980   884 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:36.551837 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.553216 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:36.554325 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:36.554852 29218 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:36.555540 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:36.556357 29910 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:36.556892 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2386) took 1542 ms. Trace:
I20260709 13:05:36.557271 29221 rpcz_store.cc:276] 0709 13:05:35.014814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.086094 (+ 71280us) service_pool.cc:224] Handling call
0709 13:05:36.556732 (+1470638us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540007add4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:36.556816 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2806603904}
I20260709 13:05:36.558243 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:36.558409 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.558071  1068 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:36.562067 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.562868 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.564724 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.565750 29071 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:36.567716 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:36.570672 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.571744 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.576772 29213 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:36.558738   876 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:05:36.581715 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.584405 29071 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: [9.7-12.9]
I20260709 13:05:36.584470 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:36.585100 29145 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:36.588218 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 69938.8us
I20260709 13:05:36.588972 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:36.590340 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59543.7us
I20260709 13:05:36.565253   900 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:36.597865  1055 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:36.593535 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:36.593912 29221 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: [10.8-16.10]
I20260709 13:05:36.568290  1113 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:36.613337 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:36.619192 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:36.620146 29070 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:05:36.620471 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.621542 29069 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:36.621614  1120 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:36.622195  1120 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:36.622550 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.622879 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.623214 29065 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:36.623885 29072 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:36.624812 29070 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:36.625222  1113 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:36.543570   947 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.631179 29910 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:36.627552 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.521373  1089 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.640334  1099 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:36.641059  1099 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:36.642284  1108 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:05:36.633455 29067 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:36.653863 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 699) took 1132 ms. Trace:
I20260709 13:05:36.654273 29219 rpcz_store.cc:276] 0709 13:05:35.521893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.522028 (+   135us) service_pool.cc:224] Handling call
0709 13:05:35.575790 (+ 53762us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:36.061260 (+485470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.061293 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.117379 (+ 56086us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.435880 (+318501us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:36.435943 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.436022 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.653616 (+217594us) raft_consensus.cc:1685] finished
0709 13:05:36.653640 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.653795 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18580,"raft.queue_time_us":21717,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"thread_start_us":646379,"threads_started":2,"wal-append.queue_time_us":355201}
W20260709 13:05:36.657358  1110 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
W20260709 13:05:36.658877 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 65640.7us
I20260709 13:05:36.659988 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:36.661906 29071 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: [11.7-14.9]
I20260709 13:05:36.662129 29064 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:36.660645 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:36.666328 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 51178.4us
I20260709 13:05:36.666651 29218 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:36.666808 29066 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:36.626327  1113 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.622593  1120 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:36.671037 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:36.656575  1088 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.675968 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:36.679390 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:36.679798 29068 raft_consensus.cc:2368] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12.
I20260709 13:05:36.691143 29145 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:36.691664 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.690230 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:36.693965 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.694643 29064 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:36.696693  1098 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 13:05:36.698655  1116 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:05:36.700150 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:36.668344   883 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:36.723376 29185 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:36.723515 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:36.725098   985 raft_consensus.cc:2764] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:05:36.725119   999 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:36.725557 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:36.726184   999 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:36.726585   985 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:36.727164   985 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:36.720952 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:36.727757   985 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:36.726922 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:36.728889   999 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:36.730455 29067 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:36.731195 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.731443 29141 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:36.730170 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:36.700594   625 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:36.734186   625 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:36.735224  1088 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:36.735122   625 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.736196 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.736910 29071 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
W20260709 13:05:36.737227  1122 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:05:36.737821 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.731813 29217 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:36.738533 29071 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:36.739022 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.742174 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:36.739688 29220 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:36.747565 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.749116 29065 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:36.750839 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:36.757027 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:36.687554  1099 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:36.759763 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.762806 29069 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:36.739430  1067 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:36.769990 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:36.772116 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:36.773476 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:36.776688 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.777509 29067 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:36.778249 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:36.778820 29067 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:36.755409   641 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:05:36.840080 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50966.9us
I20260709 13:05:36.759626  1055 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:36.888857 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.888933 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:36.760353   883 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:36.893239 29071 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:36.893350 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.945564 29065 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:36.839905  1105 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:36.959712 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:36.841771   900 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:36.897264  1067 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:36.992314 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 262281us
W20260709 13:05:37.001570 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 228165us
I20260709 13:05:36.946158  1098 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.945005   999 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:37.020020   979 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.144s	user 0.015s	sys 0.050s
W20260709 13:05:37.021498   979 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.157s	user 0.016s	sys 0.051s
I20260709 13:05:37.022943   979 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:37.024983   979 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:36.955883  1055 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:36.969424   525 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:37.033490 29185 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:37.001983  1056 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:37.038890 29065 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:05:37.046731 29215 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.000s
I20260709 13:05:37.047252 29215 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:36.961879  1036 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:05:37.049875  1097 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:37.050387  1097 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.050774  1097 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:37.051234   979 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:37.051846   979 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:37.051723  1106 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.254s
W20260709 13:05:37.055579 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 52486.9us
I20260709 13:05:36.982131  1057 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:05:37.035702   525 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:37.046017  1100 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:37.044394   984 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:37.070880 29071 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:37.048660  1056 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:37.085244 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:05:37.067487  1112 consensus_meta.cc:287] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.570s	user 0.006s	sys 0.005s
I20260709 13:05:37.033082   999 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:37.074471  1134 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:05:37.066892   525 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:37.032678  1098 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.081676  1046 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:37.115377 29072 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:37.091290  1056 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:37.058044   979 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:37.140187 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.208523   641 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:05:37.211076  1117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.007s	sys 0.013s
I20260709 13:05:37.210227   641 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:37.211571  1117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.007s	sys 0.013s
I20260709 13:05:37.112814   737 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:37.226264 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 156998us
I20260709 13:05:37.191594  1068 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:37.117933  1046 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:37.226755 29871 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:37.227058 29221 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.005s	sys 0.009s
W20260709 13:05:37.270092 29221 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.005s	sys 0.009s
W20260709 13:05:37.260242 29218 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.000s	sys 0.015s
W20260709 13:05:37.271803 29218 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.000s	sys 0.016s
W20260709 13:05:37.212180 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 155545us
I20260709 13:05:37.092674  1112 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.069159   883 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:37.274591  1089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.008s	sys 0.012s
I20260709 13:05:37.331172 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:37.323269   720 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.011s	sys 0.028s
W20260709 13:05:37.326200 29875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.009s	sys 0.030s
I20260709 13:05:37.334177  1092 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:37.336717  1099 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:37.339632 29875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.009s	sys 0.031s
I20260709 13:05:37.257285   984 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:37.347558 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 600157us
W20260709 13:05:37.330802  1089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.008s	sys 0.012s
I20260709 13:05:37.347671 29073 raft_consensus.cc:2368] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10.
I20260709 13:05:37.118345  1067 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:37.343842   720 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.011s	sys 0.029s
W20260709 13:05:37.364921 29066 consensus_meta.cc:287] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.697s	user 0.012s	sys 0.000s
I20260709 13:05:37.328650  1068 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:37.369191 29066 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:37.372618  1129 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:37.373227  1136 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.420s	user 0.001s	sys 0.000s
W20260709 13:05:37.373548  1121 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.446s	user 0.000s	sys 0.001s
I20260709 13:05:37.373440  1158 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:37.374573  1144 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:05:37.374750  1128 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
W20260709 13:05:37.375470  1139 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:05:37.341876  1057 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.207391  1055 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.345969   990 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:37.356364   900 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:37.402623  1165 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:37.363461   984 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.405668  1168 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:05:37.470232 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 243585us
I20260709 13:05:37.386242 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:37.348881  1092 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:37.420372  1141 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
I20260709 13:05:37.420725  1133 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:37.421939  1120 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.702s	user 0.010s	sys 0.000s
I20260709 13:05:37.413206  1158 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.424571  1166 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.425405 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:37.425669   979 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:37.426522   999 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.351151  1099 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:37.403223  1137 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 13:05:37.396763   737 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.471302 29142 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:37.418026 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:37.473091 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:37.485459 29070 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.010s	sys 0.024s
I20260709 13:05:37.506507 29066 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:37.431236  1057 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:37.506536 29070 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.011s	sys 0.024s
I20260709 13:05:37.508356 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:37.397940  1129 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.493189 29909 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:37.485980  1132 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.520781 29146 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:37.417975  1068 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.521814 29145 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:05:37.523502 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:37.521857 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:37.528332  1131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.019s	sys 0.030s
W20260709 13:05:37.528803  1131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.020s	sys 0.031s
W20260709 13:05:37.532786  1159 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:05:37.534162  1140 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:37.534937  1140 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.528872 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:37.537071   525 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.537456  1140 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.538236  1149 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:37.538837  1149 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.539371   525 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.540030   525 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:37.540072  1152 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.540918  1152 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:37.543085  1111 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.006s	user 0.005s	sys 0.008s
I20260709 13:05:37.543161  1152 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:37.544963  1131 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:05:37.545316  1157 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:05:37.545380  1131 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:37.546397  1131 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.548295 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:37.520608 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:37.577069 29139 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:37.516341   979 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:37.587507 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 160290us
I20260709 13:05:37.590857 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.512246  1171 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:37.492307  1120 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.477768  1046 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.639809 29068 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:37.525504  1154 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.642557 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:37.643388 29140 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:37.479173  1165 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:37.645660 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 57321us
I20260709 13:05:37.644836 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:37.648581 29072 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:37.648496 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:37.653275 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:37.587579   999 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.577140  1098 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:37.502312  1166 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.514673   737 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.673009 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:37.675621 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:37.675750 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:37.673676 29909 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
W20260709 13:05:37.671497  1049 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.111s	user 0.007s	sys 0.007s
I20260709 13:05:37.668171   720 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:37.709401 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:37.708609 29146 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:37.667958   948 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:05:37.712754 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 435452us
I20260709 13:05:37.708041 29072 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:37.664320  1112 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.549746  1132 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:37.714807 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 388596us
I20260709 13:05:37.637964   979 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.698450  1046 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:37.719519 29903 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:37.700817   999 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.725988  1165 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:37.726696  1165 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.727202  1165 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.657485  1171 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.729084  1099 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.729694  1171 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:37.730432  1167 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:05:37.730610 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:37.731716 29146 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:37.714534  1049 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.732928  1170 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:37.733091 29185 leader_election.cc:400] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12.
I20260709 13:05:37.733384  1170 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:37.733507 29185 leader_election.cc:403] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:05:37.734028  1049 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:37.734465  1170 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.727499   999 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.664842  1154 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.738320  1154 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:37.738909  1165 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:37.739815  1165 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:37.740720  1165 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12.
I20260709 13:05:37.717945  1182 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.741729  1165 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:37.742151  1165 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:37.741941  1182 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.743155  1165 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.744570  1165 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:37.714125   720 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.740617   984 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.735546  1170 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:37.754825 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2254) took 1085 ms. Trace:
I20260709 13:05:37.755722 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.759814 29071 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:37.760824 29142 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
W20260709 13:05:37.762705 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 112475us
I20260709 13:05:37.757203 29070 rpcz_store.cc:276] 0709 13:05:36.660935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.661423 (+   488us) service_pool.cc:224] Handling call
0709 13:05:36.681532 (+ 20109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:36.692661 (+ 11129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.692690 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.708879 (+ 16189us) log.cc:844] Serialized 26 byte log entry
0709 13:05:37.512598 (+803719us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:37.579319 (+ 66721us) log.cc:844] Serialized 14 byte log entry
0709 13:05:37.584169 (+  4850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.584267 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:37.745584 (+161317us) raft_consensus.cc:1685] finished
0709 13:05:37.745611 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:37.745809 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4810,"prepare.queue_time_us":66423,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":5350,"raft.queue_time_us":23418,"raft.run_cpu_time_us":402,"raft.run_wall_time_us":44635,"thread_start_us":747865,"threads_started":1,"wal-append.queue_time_us":967109}
W20260709 13:05:37.771955 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48073.4us
W20260709 13:05:37.747795   884 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.198s	user 0.010s	sys 0.004s
I20260709 13:05:37.775205 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:37.777655 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.777925  1092 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.778825 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:37.755023   720 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:37.779178 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:37.780051 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 77726.3us
I20260709 13:05:37.775204   884 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.788079 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:37.788599 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55664.2us
I20260709 13:05:37.789770 29072 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:37.787015 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:37.792685  1182 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:37.792927  1166 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:37.785288   979 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:37.795552 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.796553 29066 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:37.796658 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:37.797488  1177 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:05:37.797919 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.798686 29066 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:37.799049 29068 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:37.799314 29068 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:37.799883 29068 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.800479 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.801116 29068 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:37.801169 29148 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:37.796526 29071 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:37.802234   884 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.810889 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.813737 29909 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:37.815618 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:37.816753   979 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:37.800431 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:37.818178 29907 raft_consensus.cc:2410] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:37.820288  1129 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:37.821652  1132 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.822288  1057 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:37.813604   999 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:05:37.823110  1097 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.772s	user 0.007s	sys 0.005s
I20260709 13:05:37.824261 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.825001 29072 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:37.824803 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:37.825812 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.826015 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:37.827426 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:37.826375 29071 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
W20260709 13:05:37.828899 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 49837.5us
I20260709 13:05:37.831353 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.829936   984 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:37.832053  1171 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:37.832361   984 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:37.832777   984 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:37.823613  1097 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.843641 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:37.844390   985 consensus_meta.cc:287] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.116s	user 0.013s	sys 0.000s
I20260709 13:05:37.841082  1099 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.832727   737 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:37.849558  1175 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.146s	user 0.000s	sys 0.002s
I20260709 13:05:37.857123 32572 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:37.878484 32572 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:37.858156  1120 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.858888  1183 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:37.861104  1166 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.863121  1149 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.863217 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:37.886813 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:37.863801 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:37.831046  1182 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:05:37.828370   979 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:37.863801 29073 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2166) took 1204 ms (client timeout 1567 ms). Trace:
I20260709 13:05:37.866899 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.890203 32572 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:37.872046 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55385.6us
I20260709 13:05:37.884052 29140 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:37.890995 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.888274 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:37.895642 29072 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 8: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:37.887969 29146 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:37.883116  1099 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.893921 29071 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:37.889730  1057 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:37.893110 29073 rpcz_store.cc:269] 0709 13:05:36.659314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.659700 (+   386us) service_pool.cc:224] Handling call
0709 13:05:37.863593 (+1203893us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54002ad854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:37.863715 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579596544}
I20260709 13:05:37.885221   999 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:37.898633 32572 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:37.902212 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 58677.4us
I20260709 13:05:37.899224 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.889925   737 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:37.881995   985 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:37.904232 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.912734 29066 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:37.900274  1182 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.905875  1097 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:37.908021 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:37.914340 29072 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:37.909264 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.893330  1068 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:37.911453 29071 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:37.915262 29073 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:37.921578 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.924575 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.925534 29071 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:37.928045 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:37.928900 29072 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:37.930864 29073 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:37.898604  1181 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:05:37.931025  1176 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:05:37.892824  1166 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.941908 29064 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:37.939121 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:37.948428 29910 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:37.929101 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.952281 29069 raft_consensus.cc:2410] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:37.949357 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:37.928061  1068 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:37.955088 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:37.920718   737 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:37.978749 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 76896.2us
I20260709 13:05:37.920503  1182 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:37.973299  1056 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.765s	user 0.007s	sys 0.004s
W20260709 13:05:37.981137 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.919834  1057 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:37.955655  1094 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:37.981844 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:37.972517  1192 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:05:37.945524   947 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:05:37.986871 29072 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:37.987783  1172 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:37.961244  1150 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:37.990170 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:37.995680 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.979274  1158 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:37.985935  1094 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.000726   985 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:37.990849   999 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:37.991503  1150 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.005142 29068 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
W20260709 13:05:38.009217 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 61291.1us
I20260709 13:05:38.005082 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:37.979986  1181 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:05:38.010627  1099 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:37.994076  1097 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:37.987048  1056 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.026437 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:37.997534  1098 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:38.027433 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.030838 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.028029 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:38.023885 29068 pending_rounds.cc:85] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 9
I20260709 13:05:38.032114 29068 pending_rounds.cc:107] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:05:38.032354 29068 raft_consensus.cc:2889] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:38.006783  1094 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:38.035245 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:38.038973  1182 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:38.039306  1105 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:05:38.039897  1098 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:38.041066  1098 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.044277   737 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:38.039007  1187 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.319s	user 0.001s	sys 0.001s
I20260709 13:05:38.043581  1171 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:38.011391  1057 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:37.993762  1172 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.058351  1056 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:38.024698  1068 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.050611  1195 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:38.051705 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:38.054476 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.055984  1193 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:05:38.056089  1158 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:38.056747 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:38.068291 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68256.5us
I20260709 13:05:38.069023 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:38.069650 29907 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:38.073172 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.073925 29216 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:38.045586  1166 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:38.065955  1195 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.079133 29213 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:05:38.080551  1195 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:38.080811 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:38.081663 29071 raft_consensus.cc:2368] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10.
I20260709 13:05:38.077445 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:38.082417 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:38.083034 29071 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:38.084452 29066 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:38.073959  1131 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:38.027357  1150 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:38.083824 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:38.088454 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.089196  1172 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:38.087272  1112 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:38.091933 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.084623   720 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.093459 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.094281 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:38.095018 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:38.086640 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.028956   984 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.100907 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:38.095120   720 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:38.099141   900 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:38.104449 29214 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:38.106892 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.108376 29213 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:38.107700 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.109575 29214 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:38.111908   984 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:38.112635  1143 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.993s	user 0.003s	sys 0.000s
I20260709 13:05:38.112836 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:38.111862 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:38.114646 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:38.115269 29070 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:38.117975 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:38.124215 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:38.126030 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56410.6us
I20260709 13:05:38.137810 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:38.136232  1165 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:05:38.148599  1165 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:38.149046  1165 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:38.150262 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56075.8us
I20260709 13:05:38.150007  1165 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.151439  1165 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:38.153185 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:38.156533 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 545) took 1456 ms. Trace:
I20260709 13:05:38.156915 29067 rpcz_store.cc:276] 0709 13:05:36.699665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.088624 (+388959us) service_pool.cc:224] Handling call
0709 13:05:37.088896 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.089077 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:37.119747 (+ 30670us) log.cc:844] Serialized 14 byte log entry
0709 13:05:38.044741 (+924994us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b180048c6a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a 00007f7836b17a2a
0709 13:05:38.138032 (+ 93291us) log.cc:844] Serialized 14 byte log entry
0709 13:05:38.156163 (+ 18131us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.156216 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:38.156258 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.156326 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.156474 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17871,"prepare.queue_time_us":30590,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":10533,"raft.queue_time_us":18787,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":87,"spinlock_wait_cycles":3791104,"thread_start_us":72946,"threads_started":1,"wal-append.queue_time_us":1028023}
I20260709 13:05:38.161314 29214 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:05:38.161755 29214 pending_rounds.cc:85] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 8
I20260709 13:05:38.161985 29214 pending_rounds.cc:107] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:05:38.162230 29214 raft_consensus.cc:2889] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:38.155496 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.164304 29213 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
W20260709 13:05:38.165944 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.165933 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:38.172412 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.173091 29222 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:38.173197 29113 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:38.174220 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:38.174846 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.167776 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:38.175024 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.179383 29070 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:05:38.181773 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:38.184466 29147 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:05:38.185091 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.164580  1036 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:38.166869  1099 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.187875 29220 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:05:38.189663 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.192549 29142 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:38.190009 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.168793  1166 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:38.193744  1166 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.194057  1166 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:38.197254  1170 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:38.197842  1170 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.199517 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.168352   873 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:05:38.181310  1162 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.732s	user 0.000s	sys 0.003s
I20260709 13:05:38.177467  1140 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:38.213104   979 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:38.188917   947 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:38.189888  1056 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:38.189710  1097 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:38.227984 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 46265.5us
I20260709 13:05:38.228456 29213 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:38.229126 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.222394 29144 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:05:38.229306 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.234817 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.201891  1036 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:38.199220  1120 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:05:38.207681  1140 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:38.220604   947 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:38.233000  1097 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.244119 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.241429 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.248438 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.245236  1093 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:38.268368  1158 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:38.268872 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.269063  1158 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.270175 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.271322 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.271513  1158 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:38.272773 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:38.280107 29139 raft_consensus.cc:2393] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:38.280505  1036 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.281119 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.282325 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.286211 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.287467 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.287968 29146 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:38.297972  1046 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:38.298540  1046 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:38.299577  1046 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:38.301350  1202 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 13:05:38.302977  1204 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:05:38.305446 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:38.305610 29069 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:05:38.306164 29066 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:38.306473 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.307133   984 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.307264   883 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.307114 29141 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:38.308005   883 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.202169  1099 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.310757   883 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:38.249039  1140 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:38.312249  1089 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:38.312840  1089 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.315011  1089 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:38.307660   984 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.317533 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:38.317793 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 665) took 1795 ms. Trace:
I20260709 13:05:38.318423 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.319105   984 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:38.319329  1170 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:38.318451 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:38.250465   947 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:38.320252 29905 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 8.
I20260709 13:05:38.310779  1099 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:38.321521 29145 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:38.318783 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:38.322441 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.322767 29902 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:38.323001 29145 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:38.323832  1152 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:38.324265  1152 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.324592  1152 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:38.325968 29142 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:38.330739 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:38.332264  1201 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:38.333144  1201 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:38.333621 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2168) took 1652 ms. Trace:
I20260709 13:05:38.333750  1201 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:38.334244  1201 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:38.334091 29064 rpcz_store.cc:276] 0709 13:05:36.681192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.709091 (+ 27899us) service_pool.cc:224] Handling call
0709 13:05:36.745628 (+ 36537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.953294 (+1207666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.333362 (+380068us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54003bbfd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:38.333547 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2738173824}
I20260709 13:05:38.335316 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.335204  1201 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.336392  1099 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:38.338610  1099 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:38.338686  1201 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:38.339076  1099 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:38.339733 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.340080   984 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:38.319562 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.279465  1166 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.337394 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:38.339087 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 97754.3us
I20260709 13:05:38.350678 29113 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.342875  1093 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:38.344681  1097 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:38.318200 29143 rpcz_store.cc:276] 0709 13:05:36.522468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.522605 (+   137us) service_pool.cc:224] Handling call
0709 13:05:36.529939 (+  7334us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:05:36.587673 (+ 57734us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:36.740591 (+152918us) log.cc:844] Serialized 14 byte log entry
0709 13:05:37.334807 (+594216us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:37.335658 (+   851us) log.cc:844] Serialized 78 byte log entry
0709 13:05:37.335826 (+   168us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:37.336094 (+   268us) log.cc:844] Serialized 14 byte log entry
0709 13:05:37.336274 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.336396 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:38.159617 (+823221us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:38.317573 (+157956us) raft_consensus.cc:1685] finished
0709 13:05:38.317595 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.317737 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":201042,"prepare.queue_time_us":130445,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":103389,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":25996,"spinlock_wait_cycles":54653696,"thread_start_us":377230,"threads_started":2,"wal-append.queue_time_us":583587}
I20260709 13:05:38.369625 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:38.348446 29112 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.377801 29147 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:38.378489 29142 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:38.348526 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:38.349172 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 74711.2us
I20260709 13:05:38.349828 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:38.380550 29146 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:38.341851 29216 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:38.337318 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:38.340354 29141 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:38.368731 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.372105 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.380052 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.404088   984 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:38.404469  1191 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:38.404865  1191 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:38.405020 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.378372 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:38.409373 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:38.410323 29068 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:38.412191 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 43524.9us
I20260709 13:05:38.413901 29113 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.416617 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.417994 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.420451 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.417272 29144 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:38.339578 28969 catalog_manager.cc:5697] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 6 to 9, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 9 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.423254 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.424198 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.428038 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:38.440531 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.453461 32572 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:38.453613 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:38.453962 32572 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.454295 32572 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:38.460255 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.409480  1149 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:38.422116  1093 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:38.423797  1089 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:38.494650 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:38.467778 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:38.299535  1094 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.468556 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.500125 29146 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:38.476866 29143 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:38.476724 28969 catalog_manager.cc:5697] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 2 to 9, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 9 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:05:38.480499 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 148548us
I20260709 13:05:38.503038 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:38.490559   947 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.426977   737 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:38.492229 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.496670 32572 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:38.462395 29185 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.310s (SENT) [suppressed 27 similar messages]
I20260709 13:05:38.498186 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.439075   979 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:38.506409 29145 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:05:38.502442 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:38.528475 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:38.529825 29139 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:38.530771 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:38.509994 29905 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:05:38.511965 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet ac412099b9f64d4e8bcceebe1c7ffe87: Illegal state: Replica a936878f59c840a6bd176a1d2d7347b1 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:38.512658 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:38.513424 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 51081.2us
I20260709 13:05:38.513701 28969 catalog_manager.cc:5697] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:38.537266 29065 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:38.516642 29066 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:05:38.518738 29072 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:05:38.465605  1205 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:38.519811 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:38.507854   737 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.440157   883 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:05:38.464933  1149 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.469249  1093 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:38.508315 29141 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:38.497865   625 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:38.501120  1182 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:38.427459  1150 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.547024 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.550665 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:38.552232 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.565601 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.578779 29213 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:38.576390 29072 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:05:38.576534 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.578662 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:38.467141  1152 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.555073 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:38.584594 29146 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:38.585873 29141 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:05:38.586742 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:38.526682  1181 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:38.591223 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:38.592473 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.524693   979 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:38.597954 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:38.599900 28969 catalog_manager.cc:5697] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 8, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 8 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:38.594461  1036 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:38.545779   737 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:38.551785  1056 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:38.575465  1089 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:38.553500  1093 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:38.549592  1195 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:38.626243 28969 catalog_manager.cc:5697] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 7 to 10, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 10 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:38.554000  1165 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.549034  1149 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:38.549319  1170 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:38.598934  1113 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:38.634114 28969 catalog_manager.cc:5697] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 7 to 8, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 8 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:38.615300 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.553676   625 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:38.613155 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.548115  1140 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.577746  1168 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:38.598783  1068 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:38.612075   985 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:38.617050  1182 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:38.649411 29222 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:05:38.724086 29222 pending_rounds.cc:85] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 8
I20260709 13:05:38.724332 29222 pending_rounds.cc:107] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:05:38.724562 29222 raft_consensus.cc:2889] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:38.643862   900 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.003s
I20260709 13:05:38.684250  1152 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:38.672062  1206 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:05:38.641547  1098 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.733991  1098 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.736761  1098 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:38.670873  1197 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.162s
I20260709 13:05:38.739125  1197 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:38.739668  1197 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:38.682089  1099 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.743247  1099 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:38.670367  1150 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:38.676975  1147 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.151s
I20260709 13:05:38.625288  1158 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:05:38.678035 32572 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:38.631388  1170 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.707780  1168 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.556926 29218 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:05:38.715377 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 102301us
I20260709 13:05:38.717602 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:38.639775  1165 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.681895   979 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.760811 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.761123 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:38.772357 29217 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:38.772668 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55165.9us
I20260709 13:05:38.762405 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.763749 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:38.773336 32572 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:38.773752 32572 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.765453 29185 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.765594 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.767937  1158 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:38.768765  1057 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:38.723640  1182 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.774061 32572 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:38.778373  1211 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:38.771278 28969 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 8 to 12, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 12 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:38.802716 29219 raft_consensus.cc:2410] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:05:38.779846 29185 leader_election.cc:400] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10.
W20260709 13:05:38.780628 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55413.3us
I20260709 13:05:38.783219 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:38.805081 29185 leader_election.cc:403] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:05:38.796591  1158 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.796859  1097 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.816393  1094 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:38.818367  1212 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:05:38.802682 28969 catalog_manager.cc:5697] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 3 to 10, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 10 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:38.822703 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:38.784895 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 290765us
I20260709 13:05:38.727367  1208 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:38.793591   947 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:38.770113  1168 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:38.685006  1056 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.752214   900 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:38.759121  1170 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:38.826589 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.827684 29217 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
W20260709 13:05:38.827068 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 229205us
I20260709 13:05:38.803814  1182 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:38.826381 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.830870 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:38.831944 32572 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.807075  1067 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:38.836870 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:38.837801 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:38.787871  1152 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:05:38.837196  1208 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:38.843884 29222 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:05:38.515023  1191 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.837813 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:38.845070 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:38.740785  1197 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.827006  1061 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:05:38.849931 32572 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:38.853298 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:38.858544 29217 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 13: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:38.820328   948 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:05:38.807282  1057 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.831698  1150 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:05:38.873623  1230 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:38.848098   369 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:38.853876 29216 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:38.856350 28969 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:38.847987   999 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:38.878747 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:38.838469  1195 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10.
I20260709 13:05:38.881315 29222 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:38.859858  1208 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:38.888773 28969 catalog_manager.cc:5697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:05:38.886281 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55243.8us
I20260709 13:05:38.891742 28969 catalog_manager.cc:5697] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:38.876595  1150 consensus_queue.cc:1243] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:05:38.900499 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55543.6us
I20260709 13:05:38.877905   900 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:38.902019 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:38.857825   873 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:05:38.904415 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:38.881258  1067 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.900722   984 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:38.910753 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 84865.9us
W20260709 13:05:38.914403 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 76702us
I20260709 13:05:38.915287 29185 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:38.915688 28969 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:38.917135 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 77325.5us
I20260709 13:05:38.929430 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.932766 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:38.934190 29144 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:38.908107  1089 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:38.936863 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.905113  1219 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:38.914462   999 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:38.914283   900 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:38.953747   900 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:38.956146  1164 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.486s	user 0.004s	sys 0.006s
I20260709 13:05:38.912814  1067 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.961704  1236 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:38.962589  1093 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:38.965750   737 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.966626  1219 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.918192  1105 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:38.941450 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.979476 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:38.931908  1181 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:05:38.971827 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:38.944211  1226 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:05:38.925510  1208 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:38.948530 29140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:05:38.938849  1227 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:38.952047  1152 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:38.970345 29222 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:38.936753 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:38.941406 28969 catalog_manager.cc:5697] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:38.987183 29140 raft_consensus.cc:606] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:38.971385 32572 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.967883  1217 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:38.967168  1219 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.979460 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:38.989526 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 76251.2us
I20260709 13:05:38.992607 29141 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:38.934252  1209 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:05:38.984870 29147 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 9: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:38.980432  1105 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.918052   876 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:05:38.990212 29140 raft_consensus.cc:614] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:38.991019 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:38.990684 32572 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:38.994096 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:38.998829 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:39.000592 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:39.002519 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:38.998343 28969 catalog_manager.cc:5697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 10, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 10 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:38.989284  1208 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:39.007035 29907 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 9: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:38.986696  1205 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:39.008220 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:39.008649 29903 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:05:39.010149 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 68806us
I20260709 13:05:38.989544  1228 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:39.010313 29910 raft_consensus.cc:2368] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11.
I20260709 13:05:38.987414  1232 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:38.995157  1233 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.009207  1208 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:39.013386 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:39.011494 28969 catalog_manager.cc:5697] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 13. New cstate: current_term: 13 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:39.046098 29142 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:39.019120 29141 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:38.989346   525 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:05:39.058437  1105 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.059283 29140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:05:39.075886 29140 raft_consensus.cc:606] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:39.076354 29140 raft_consensus.cc:614] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:39.061830  1233 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.065086  1093 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:38.996392   985 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:05:39.009868   999 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:39.068926 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:39.086822   999 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:39.025871   737 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:39.025956  1219 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:39.078332 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 65359.7us
W20260709 13:05:39.083704 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 92799us
I20260709 13:05:39.026676  1057 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.075205  1131 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:39.102625  1246 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:39.049278 28969 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 7 to 11, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 11 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:39.045207  1232 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.115617  1232 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:39.115490 28969 catalog_manager.cc:5697] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 7 to 10, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 10 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.103242  1246 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:39.118067  1246 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.117924 28969 catalog_manager.cc:5697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 7 to 9, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 9 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:39.051407   720 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:05:39.047753  1152 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:39.133239 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:39.110707 29222 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:39.137832 28969 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 6 to 10, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 10 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.142328 28969 catalog_manager.cc:5697] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 10, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 10 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.148241 29221 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:39.149973 29221 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 6. (term mismatch)
I20260709 13:05:39.150380 29221 pending_rounds.cc:85] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 5
I20260709 13:05:39.150578 29221 pending_rounds.cc:107] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 13.6
I20260709 13:05:39.150789 29221 raft_consensus.cc:2889] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:39.152226 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:39.152832 29908 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:39.109970   999 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.141113  1182 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.156247 29218 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:05:39.156947  1057 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:39.157567  1057 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.158722  1197 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:39.152823 28969 catalog_manager.cc:5697] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 10, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 10 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:39.161317 28969 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 9 to 11, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 11 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.162905 29214 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:39.144867  1228 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:39.167521 29142 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:39.146209  1243 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:39.170584 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:39.171661 28969 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.144112  1152 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.144634  1170 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:39.197845 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 187081us
I20260709 13:05:39.200153 28969 catalog_manager.cc:5697] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 3 to 7, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 7 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:05:39.198444 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 267367us
I20260709 13:05:39.202250 28969 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 3 to 8, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 8 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.169559  1182 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:39.205972 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.207787 29066 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:39.208561 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.209187 29073 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:39.212131 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.213642 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.214212 29065 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:05:39.213020 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.192567  1243 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:39.216349 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:39.216851 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:39.216454 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.222746 29072 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:39.222931 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.223599 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:39.223743 29068 raft_consensus.cc:2376] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11. Re-sending same reply.
I20260709 13:05:39.224453 29069 raft_consensus.cc:2410] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:05:39.218559 28969 catalog_manager.cc:5697] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 7 to 11, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 11 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:39.201732  1057 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.222267 29067 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:39.231297 29904 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:39.210441  1209 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:39.220350  1241 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:05:39.237855 29073 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
W20260709 13:05:39.226547  1149 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.000s
I20260709 13:05:39.240697 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.200520  1168 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.236074 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:39.242354 29068 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:39.244653 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:39.244910 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.248301 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:39.249459 29902 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:39.232699 28969 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 3 to 12, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 12 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:39.234153  1209 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:39.260226 29072 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 9.
I20260709 13:05:39.251984  1249 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.264778 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:39.265722 29072 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:39.266417 29066 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:39.244043   999 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:05:39.267905  1249 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.271471  1165 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:39.276906 29222 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:39.278623 29065 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:39.289175 29066 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:39.299019 29072 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:39.291393 29871 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:39.300657 29069 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:39.290620 29064 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:39.300868 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:39.305703 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.305958 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:39.306351 29066 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:39.306507 29065 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:39.264123  1209 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:39.309044 29068 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:39.307353 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:39.250563  1149 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.293183  1089 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:39.309303   985 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.312671  1250 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:39.323086 29066 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
W20260709 13:05:39.323648 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 386973us
I20260709 13:05:39.324929 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.326551 29142 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:39.328109 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.312259  1251 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:39.328390 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:39.329119 29146 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:39.315049  1233 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:39.331023 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 255654us
I20260709 13:05:39.240720  1228 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.336941 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.338793 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:39.339495 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.340092 29142 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:39.340610 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.340982 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.338898 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.342702 29146 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:39.344044 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:39.347271  1218 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:39.347949  1218 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:39.348599 29908 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:39.348867  1218 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.310362  1243 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.337726   873 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:39.350569   873 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:39.350967   873 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:39.352057 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 216908us
I20260709 13:05:39.352842 29112 leader_election.cc:400] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11.
I20260709 13:05:39.353142 29112 leader_election.cc:403] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:05:39.329272   369 raft_consensus.cc:2764] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:05:39.356953  1061 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:39.356973 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.324311  1089 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.341039  1251 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.362553  1208 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.363651 29222 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:39.344470 29147 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:05:39.366529 29140 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:39.373466 29875 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:39.330257 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:39.390949  1061 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.398211 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.403017  1215 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:39.336227  1227 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:05:39.331175   984 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.073s
W20260709 13:05:39.408656 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 69956.5us
W20260709 13:05:39.409349 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 79027us
I20260709 13:05:39.340590  1250 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:39.390618   884 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11.
I20260709 13:05:39.389636  1231 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:39.416963 29113 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:39.420179 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:39.336563   985 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.421994 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:39.423364 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.425683   979 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:39.426242   979 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:39.424973 29141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:05:39.427052  1208 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.427327 29141 raft_consensus.cc:606] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:39.432260  1165 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:05:39.433951 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:39.411219  1215 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:39.433320 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:39.391793  1233 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:05:39.415373  1150 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:39.427157   979 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.423338  1089 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.469946  1252 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:05:39.470681  1254 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:39.435285  1105 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:39.423054   369 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
W20260709 13:05:39.476240  1263 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:05:39.497684   873 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.477619 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 731) took 1060 ms. Trace:
I20260709 13:05:39.499104  1061 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.498835 29905 rpcz_store.cc:276] 0709 13:05:38.417189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.438873 (+ 21684us) service_pool.cc:224] Handling call
0709 13:05:39.477376 (+1038503us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b54006c02d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:39.477522 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119724416}
I20260709 13:05:39.467542 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:39.479583 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2412) took 1265 ms. Trace:
I20260709 13:05:39.499814  1105 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.500876 29139 rpcz_store.cc:276] 0709 13:05:38.203658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.282067 (+ 78409us) service_pool.cc:224] Handling call
0709 13:05:39.468567 (+1186500us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54000ea6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:39.468724 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567427456}
I20260709 13:05:39.502713  1105 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.506176  1097 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:39.483575  1195 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:05:39.483560  1254 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.507146  1254 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:39.481436 29141 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:05:39.465703  1208 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.488574 32572 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:39.490736  1131 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:39.491273  1250 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.515172 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:39.492182  1225 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:39.517439 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.484139  1150 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:39.523077   525 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.513932   873 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:39.529909  1217 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.004s
I20260709 13:05:39.540252 29909 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:39.533838   948 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.534204  1181 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:39.528024  1208 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.567610 29220 raft_consensus.cc:2410] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:05:39.556675 29070 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:39.542387  1217 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:39.545164  1149 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.602047   625 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.603384   625 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:39.604707  1262 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:05:39.605486  1099 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:39.606073  1099 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:39.610329 29903 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:05:39.590291  1197 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:39.620847  1158 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:39.551849   948 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.625893   948 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.618775  1105 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.626700  1105 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.628825  1105 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:39.620538  1197 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.631412 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.532599  1215 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.633049  1197 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:39.633342  1215 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:39.612426  1099 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.636389  1099 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:39.639987  1209 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.641073   985 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.642335  1209 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.647089 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:39.648108  1249 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:39.654105  1251 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.655519  1067 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:39.656950 29904 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:39.658452 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:39.660141 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:39.666522 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:39.668299 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.669286 29072 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 13: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:39.669736 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:39.670249 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.671069 29066 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:05:39.671155 29072 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:39.683125 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:39.673013 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.686290 29072 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:39.685191 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.688031 29070 raft_consensus.cc:2368] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11.
I20260709 13:05:39.690006 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:39.684926  1209 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:39.692490  1209 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:39.693059 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.693871 29067 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 9: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:39.693754 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.542135   525 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.711583   525 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:39.712984 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.716521 29909 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:39.704293 29070 raft_consensus.cc:1217] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 5.5->[7.6-17.10]   Dedup: 9.7->[11.8-17.10]
I20260709 13:05:39.717509 29070 pending_rounds.cc:85] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 7
I20260709 13:05:39.717751 29070 pending_rounds.cc:107] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:05:39.717962 29070 raft_consensus.cc:2889] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:39.718232 29070 pending_rounds.cc:107] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:05:39.718451 29070 raft_consensus.cc:2889] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:39.695657  1067 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:05:39.719424  1201 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:39.720402 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:39.720645  1113 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:05:39.721509  1113 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:39.721839  1113 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:39.721969 29216 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:05:39.722308 29216 pending_rounds.cc:85] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 7
I20260709 13:05:39.722505 29216 pending_rounds.cc:107] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:05:39.722719 29216 raft_consensus.cc:2889] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:39.722671  1113 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.694000  1209 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.724157  1113 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:39.707423  1150 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.708823  1191 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:39.695449   985 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:39.709654  1208 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.716629  1170 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:39.701550 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:39.785434 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 83979.2us
W20260709 13:05:39.732890 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 49884.7us
I20260709 13:05:39.787709 29111 leader_election.cc:400] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11.
I20260709 13:05:39.788960 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:39.740530 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.753623 29067 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:05:39.591974  1181 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.796715 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:39.739089  1092 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:39.765077  1094 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.807058 29067 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
W20260709 13:05:39.798851  1133 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.984s	user 0.007s	sys 0.022s
I20260709 13:05:39.800595 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:39.820452 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2366) took 1053 ms. Trace:
I20260709 13:05:39.821013 29218 rpcz_store.cc:276] 0709 13:05:38.766682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.775664 (+  8982us) service_pool.cc:224] Handling call
0709 13:05:38.776035 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.156685 (+380650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:39.818761 (+662076us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54001338d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:39.820372 (+  1611us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":873767168}
I20260709 13:05:39.801441  1099 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:05:39.826274 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.833424 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:39.816411  1133 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.008s	sys 0.023s
W20260709 13:05:39.839884 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.839974 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:39.841024 29902 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:39.846060  1056 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:39.846766  1056 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:39.832315  1171 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:39.849550  1181 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.856506  1046 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:39.860533 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.863452 29064 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:39.864729 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.865614 29141 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:39.867826  1201 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:39.869041  1120 raft_consensus.cc:993] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: : Instructing follower a936878f59c840a6bd176a1d2d7347b1 to start an election
I20260709 13:05:39.869506  1120 raft_consensus.cc:1081] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Signalling peer a936878f59c840a6bd176a1d2d7347b1 to start an election
I20260709 13:05:39.873492 29064 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:39.808790  1254 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.809432  1105 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:05:39.807791   985 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.876068 29111 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:39.877578 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:39.880154  1254 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:39.880839 29112 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:39.882296 29064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:47960
I20260709 13:05:39.817428  1094 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.896843 29064 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:39.896188 29902 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:39.900322 29064 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:39.900481 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:39.872298  1181 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.838711   948 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:05:39.917268 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:39.897233  1120 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:39.902894  1205 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:39.931886 29143 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:39.943274  1166 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:39.929767  1067 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:39.944402   999 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:39.944653  1067 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.947367  1067 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:39.935263  1120 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:39.916862  1092 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:39.954756 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.934233 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:39.964089 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.965433  1061 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:39.965974  1061 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.966351  1061 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:39.927385  1237 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:05:39.969079  1237 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:39.969424  1237 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:39.969796 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:39.970250  1237 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.971555  1237 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:39.958148  1092 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:39.972852  1092 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:39.963526 29064 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.980634 29215 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:39.982538  1049 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.983120  1049 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.985357 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:39.986482   985 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:39.986742  1191 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:39.987591 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:39.989352 29064 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:39.989400 29139 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:39.989945 29139 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:39.990955 29144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2178) took 1618 ms (client timeout 2044 ms). Trace:
I20260709 13:05:39.992447 29145 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:39.991473 29144 rpcz_store.cc:269] 0709 13:05:38.372571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.423025 (+ 50454us) service_pool.cc:224] Handling call
0709 13:05:39.990738 (+1567713us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b5400190854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:39.990886 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173593344}
I20260709 13:05:39.998891 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:39.999568 29064 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:39.951813  1133 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:39.999491 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:39.917236  1181 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:39.985435  1094 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:39.949132  1120 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.014772 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:40.016618 29216 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:40.016741 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.030117 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.008843 29139 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:39.912436  1056 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.002439  1133 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.047132 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.054280 29220 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:40.055346 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2285) took 1882 ms. Trace:
I20260709 13:05:40.058220 29071 rpcz_store.cc:276] 0709 13:05:38.173125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.174071 (+   946us) service_pool.cc:224] Handling call
0709 13:05:38.174362 (+   291us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:38.186068 (+ 11706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.186098 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:38.221634 (+ 35536us) log.cc:844] Serialized 78 byte log entry
0709 13:05:38.666717 (+445083us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:39.630677 (+963960us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b180031a500. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f783a5053c7 00007f7836b179a7 00007f7836b270bf 00007f7836b27038
0709 13:05:39.842368 (+211691us) log.cc:844] Serialized 14 byte log entry
0709 13:05:40.018459 (+176091us) log.cc:844] Serialized 14 byte log entry
0709 13:05:40.018759 (+   300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.018882 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:40.055043 (+ 36161us) raft_consensus.cc:1685] finished
0709 13:05:40.055070 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.055273 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1204,"prepare.queue_time_us":765455,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":7978,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":4410,"spinlock_wait_cycles":46435584,"thread_start_us":697104,"threads_started":3,"wal-append.queue_time_us":500572,"wal-append.run_cpu_time_us":2286,"wal-append.run_wall_time_us":1039601}
I20260709 13:05:40.066658 29064 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:40.071755 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.072918 29218 raft_consensus.cc:2410] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:05:40.032867  1219 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:40.083088 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.083993 29216 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:40.061715  1133 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:40.099272 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:40.102182 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:40.100144 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:40.115278 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:40.078177  1191 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.121966 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:40.088037  1254 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:40.089829  1140 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:40.109310  1056 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:40.098948  1209 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.104169   625 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:40.141345 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2363) took 1523 ms. Trace:
I20260709 13:05:40.104614  1208 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:40.141853 29213 rpcz_store.cc:276] 0709 13:05:38.617955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.618090 (+   135us) service_pool.cc:224] Handling call
0709 13:05:38.618413 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:38.665738 (+ 47325us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:38.785928 (+120190us) log.cc:844] Serialized 14 byte log entry
0709 13:05:38.885307 (+ 99379us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:38.885361 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:38.886921 (+  1560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.141041 (+1254120us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b5400132ed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:40.141086 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.141262 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10337,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":39487,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":137,"spinlock_wait_cycles":75401472,"thread_start_us":194209,"threads_started":3,"wal-append.queue_time_us":87611,"wal-append.run_cpu_time_us":1168,"wal-append.run_wall_time_us":1003555}
I20260709 13:05:40.087052  1219 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:40.153456 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.120716  1150 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:40.104007  1046 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:05:40.129001  1113 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:40.129715  1170 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:40.108232  1105 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:05:40.112357  1217 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:05:40.105064   999 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:05:40.157361  1209 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:40.169703  1274 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:05:40.130064  1191 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.164366   737 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:40.147981  1254 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.214550  1254 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:40.195331 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:40.201006  1268 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:05:40.207791 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:40.209019 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:40.186477  1231 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:40.162685  1150 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.232015 29037 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:40.190826  1113 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:05:40.224215  1231 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:40.222538  1254 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:40.228945  1181 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:40.234313 29188 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:40.226609  1092 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:40.250084  1281 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.123s	user 0.004s	sys 0.000s
I20260709 13:05:40.264561 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:40.226294  1061 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:40.266372 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 145759us
W20260709 13:05:40.291126  1276 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:05:40.305374 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 182931us
I20260709 13:05:40.267762  1172 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:40.260428  1181 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:05:40.324857 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2410) took 1559 ms. Trace:
I20260709 13:05:40.325372 29215 rpcz_store.cc:276] 0709 13:05:38.765229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.765348 (+   119us) service_pool.cc:224] Handling call
0709 13:05:40.324633 (+1559285us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54000be854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:40.324778 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2773556608}
W20260709 13:05:40.327227 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.329035  1198 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:40.329756  1198 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:40.330925  1198 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.270844  1170 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:40.333323  1092 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:40.336627 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2375) took 1409 ms. Trace:
I20260709 13:05:40.337522  1191 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:40.343966  1231 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.345057 29139 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:40.401971  1231 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:40.348495  1254 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:40.356341  1166 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:40.360584 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.417824 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 185910us
I20260709 13:05:40.335696   985 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:40.363293  1209 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:40.368081  1219 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:40.368413  1150 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:40.370775  1277 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
W20260709 13:05:40.372567  1279 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:05:40.373303  1061 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:40.306396 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:40.446872 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 140562us
I20260709 13:05:40.447665 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:40.377120 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.384776 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.391475 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.390439 29219 rpcz_store.cc:276] 0709 13:05:38.927301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.161564 (+234263us) service_pool.cc:224] Handling call
0709 13:05:39.161893 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.162288 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:39.168689 (+  6401us) log.cc:844] Serialized 14 byte log entry
0709 13:05:39.914914 (+746225us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b18003ed3a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a 00007f7836b17a2a
0709 13:05:40.246439 (+331525us) log.cc:844] Serialized 14 byte log entry
0709 13:05:40.248046 (+  1607us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:40.248505 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:05:40.248640 (+   135us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:40.248669 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.261888 (+ 13219us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:40.336416 (+ 74528us) raft_consensus.cc:1685] finished
0709 13:05:40.336438 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.336567 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":175473,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":92,"raft.queue_time_us":11554,"raft.run_cpu_time_us":494,"raft.run_wall_time_us":38812,"spinlock_wait_cycles":3611392,"thread_start_us":324360,"threads_started":2,"wal-append.queue_time_us":1055931}
I20260709 13:05:40.394847 29147 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:40.399556 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:40.453055 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:40.453878 29140 raft_consensus.cc:2410] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:40.393437  1149 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:40.399747 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:40.400578 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:40.457445 29143 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:40.400643 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:40.458449 29910 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:40.400873 29146 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:40.459821 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:40.460691 29072 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:40.461138 29071 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:40.410797 29139 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:40.372588  1133 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.433111  1166 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.466171  1089 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:40.466838  1089 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:40.467550  1133 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:40.416894 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.363080 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2421) took 1037 ms. Trace:
I20260709 13:05:40.418679 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:40.470520 29904 raft_consensus.cc:2368] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13.
I20260709 13:05:40.408259  1254 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.431248 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 730) took 1192 ms. Trace:
I20260709 13:05:40.432366 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:40.437741 29073 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:40.454612 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:40.456231 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.465303 29072 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:40.465516 29064 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:40.476212 29071 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:40.469238 29145 rpcz_store.cc:276] 0709 13:05:39.325336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.325461 (+   125us) service_pool.cc:224] Handling call
0709 13:05:39.992214 (+666753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.993007 (+   793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.362862 (+369855us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b5400190850. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:40.363015 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500766464}
I20260709 13:05:40.483038 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:40.484105 29139 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:40.485546 29139 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:40.487057 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.489328 29148 raft_consensus.cc:2368] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14.
I20260709 13:05:40.477807 29903 rpcz_store.cc:276] 0709 13:05:39.224620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.224764 (+   144us) service_pool.cc:224] Handling call
0709 13:05:39.235610 (+ 10846us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.611988 (+376378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.399893 (+787905us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b540061fdd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:40.416434 (+ 16541us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860571392}
I20260709 13:05:40.434415  1105 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.494383 29139 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:40.496703 29071 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:40.456032  1198 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:40.498441 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:40.496263 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:40.495607 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.505766 29904 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:40.500288 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.501423 29145 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:40.506917 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.499388 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:40.512216 29213 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:40.467959  1166 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:40.467434  1089 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:05:40.521041 29064 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:40.513509  1036 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.524897  1036 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.525513 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.526299 29220 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:40.495134  1209 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:40.527256  1036 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:40.527632  1209 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.529676  1056 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.530169  1290 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:05:40.530323  1056 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.532903  1168 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:40.534516  1286 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.281s	user 0.000s	sys 0.000s
I20260709 13:05:40.535502 29112 leader_election.cc:400] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13.
I20260709 13:05:40.535820 29112 leader_election.cc:403] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:05:40.535992  1191 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:40.536409  1191 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.536767  1191 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:40.541685  1049 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:40.543517 29038 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:40.529758  1209 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:40.513645  1170 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:40.553085  1170 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.524931 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.562430 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.570178 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:40.481940  1149 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.582926 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:40.584164 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 64658.5us
I20260709 13:05:40.591187 29188 leader_election.cc:400] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14.
I20260709 13:05:40.594602 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.596505 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.593114 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:40.595570 29219 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:40.574534  1225 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:40.605774 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:40.505349  1105 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.618227 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:40.620493 29220 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:40.623373 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.623215 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 694) took 1254 ms. Trace:
I20260709 13:05:40.629954 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.611459 29904 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:40.617475 29222 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:40.617580 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.587114  1170 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:40.649323 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 61872.7us
I20260709 13:05:40.649478 29144 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:40.628132 29142 rpcz_store.cc:276] 0709 13:05:39.368475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.368618 (+   143us) service_pool.cc:224] Handling call
0709 13:05:39.369078 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:39.408661 (+ 39583us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:39.727647 (+318986us) log.cc:844] Serialized 14 byte log entry
0709 13:05:40.007980 (+280333us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:40.029129 (+ 21149us) log.cc:844] Serialized 26 byte log entry
0709 13:05:40.029324 (+   195us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:40.029372 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.029464 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:40.371908 (+342444us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:40.605546 (+233638us) raft_consensus.cc:1685] finished
0709 13:05:40.623012 (+ 17466us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:40.623159 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32729,"prepare.queue_time_us":58326,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":42056,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":28141,"spinlock_wait_cycles":3565312,"thread_start_us":545859,"threads_started":2,"wal-append.queue_time_us":466074}
I20260709 13:05:40.584900  1289 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:40.657732 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.658664 29144 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:40.659777 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.668032 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.669075 29142 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:40.601142  1056 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:40.624599  1172 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:40.627759  1036 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:40.624423  1195 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.710449  1195 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.705065  1149 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:40.707103  1105 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:40.670820  1289 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:40.601998  1049 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:05:40.647799  1046 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:40.669790  1092 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:05:40.713014  1089 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:40.657099  1120 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:05:40.602192  1209 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:05:40.681347 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.688854 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:40.775167 29142 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:40.738338 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.766259   999 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:05:40.767153 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 76419.6us
I20260709 13:05:40.678114   948 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.639329  1253 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.077s
W20260709 13:05:40.759822  1282 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:05:40.774400 29144 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:40.757804  1181 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.746264 29143 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:40.768010   525 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:05:40.758057  1113 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:40.786798 29148 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: [9.5-13.7]
I20260709 13:05:40.758934  1289 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:40.680660  1292 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 13:05:40.791669 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:40.735489  1217 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:05:40.792352 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:40.794471 29216 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:40.796322 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:40.799289 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:40.786594  1253 consensus_queue.cc:1243] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:40.796751 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:40.777640   999 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:40.770105  1046 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.787774  1067 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:40.776685  1089 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.789712   884 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:40.821674 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 170735us
I20260709 13:05:40.806555  1036 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.801708  1105 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:40.804782  1140 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:40.785976   948 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.801988  1298 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:40.792542  1113 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.790225  1181 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.808845  1293 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:40.884788 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:40.809864  1300 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:40.888620 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.869195 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51814 (request call id 9) took 1611 ms. Trace:
W20260709 13:05:40.877489 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 57898.5us
I20260709 13:05:40.805817  1094 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:05:40.823859  1092 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:05:40.900345  1055 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:40.805884  1219 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:05:40.825289  1112 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:40.920694 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:40.927404 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.928364 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:40.896150 29221 rpcz_store.cc:276] 0709 13:05:39.259799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.494856 (+235057us) service_pool.cc:224] Handling call
0709 13:05:40.868909 (+1374053us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b54003557d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b13735 00007f783bad3d29 00007f78350521d4 00007f7835052162 00007f7835052091 00007f783505200c 00007f78350512c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:40.869098 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1709890304}
I20260709 13:05:40.931324 29216 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:40.932399 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.933053 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:40.937253 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:40.938354 29904 raft_consensus.cc:2410] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:05:40.940907 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:40.859097  1254 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:40.942131 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.864319  1298 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:40.863600  1036 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.945391  1292 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:40.946161  1292 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:40.946307 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.946945 29216 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:05:40.947203  1292 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.947963 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.949122 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.949370 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:40.949666 29216 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:40.945564  1298 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:40.929913  1055 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:40.952711  1280 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.793s	user 0.003s	sys 0.000s
I20260709 13:05:40.952613 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:40.954689 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:40.956609 29219 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:40.828570   884 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.959796  1298 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.962766 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:40.965372 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.966454 29221 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:40.968597  1303 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:05:40.971537  1298 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:40.968521  1172 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:40.972388  1172 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:40.973399  1172 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.930284  1291 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.975641  1291 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.977900  1291 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:40.931875  1191 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:40.978724  1181 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:40.981336   948 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:40.981987  1113 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:40.924374   999 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:40.987124 29217 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:05:40.988395 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47602.3us
I20260709 13:05:40.961606   884 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.002146 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:40.989468 29222 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:40.989833   985 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:40.989223  1170 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:40.859987  1140 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:40.991039  1305 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:05:40.990394  1195 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:40.924177  1300 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.034255 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:40.993096 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:41.039706 29910 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:40.983085 29218 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:40.993472 29903 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:40.924068   525 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:40.997644   737 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:40.998157  1295 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:40.998975  1112 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:40.983249  1089 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:41.001026 29213 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:41.059152 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:40.990382  1201 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:40.819271  1067 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.004871   985 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:41.080435   985 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.081804   985 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.007174  1166 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:41.032917  1181 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:40.860069  1133 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.143s
I20260709 13:05:41.037108 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:41.059154 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:41.058264 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 249557us
W20260709 13:05:41.089444 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 290250us
I20260709 13:05:41.004151 29185 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:41.078750  1140 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.079453  1067 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.044479  1170 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:41.093361 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:41.035063  1217 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:05:41.101603 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.096558 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:41.122581 29147 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:41.126744 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:05:41.127835 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 302519us
I20260709 13:05:41.128671 29139 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:41.128773 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:41.129086 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:41.095027  1092 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.094894  1171 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:41.057827   525 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.096987  1293 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.097126  1295 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:41.097213  1149 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.126874  1191 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:41.135713  1120 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:05:41.135531 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:41.096972  1094 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:41.099437  1067 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.105769  1036 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:41.143963 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:41.150597 29909 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.110816  1227 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:41.092315  1181 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.126343  1140 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:41.096733  1089 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:05:41.120307  1298 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:41.154822 29038 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:41.151562  1227 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.126447  1237 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:05:41.176935  1205 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:41.177083  1227 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:41.186533  1172 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.139860  1092 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.152928  1067 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.188380  1055 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:41.189713   999 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:41.190312  1067 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.190923  1170 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.152760  1094 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.147308  1293 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.194198  1094 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.151695  1113 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:41.195214  1293 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:41.152585  1098 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:41.196818  1098 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:41.143687  1112 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.197907  1300 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.197777  1098 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:41.199849  1198 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.684s	user 0.012s	sys 0.000s
I20260709 13:05:41.201920   525 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:41.189805  1092 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.205224  1046 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:41.206033 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:41.206735 29911 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:41.211973  1181 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.212378  1166 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:41.212990 29222 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:41.213457  1166 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:41.213980 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 209933us
I20260709 13:05:41.147713  1149 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.214689  1195 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:41.215345 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:41.147809  1295 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:41.218623 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 63929us
I20260709 13:05:41.221169 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:41.206312  1225 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.588s	user 0.009s	sys 0.000s
I20260709 13:05:41.218775 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:41.226433 29909 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:41.220345 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:41.235009 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 106325us
I20260709 13:05:41.236441 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:41.235316 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:41.262123 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 133118us
I20260709 13:05:41.263224 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:41.265921 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 44876us
I20260709 13:05:41.266808 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:41.267584 29222 pending_rounds.cc:85] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 11
I20260709 13:05:41.233122  1298 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:41.233841  1225 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13.
I20260709 13:05:41.267894 29222 pending_rounds.cc:107] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:05:41.274318 29222 raft_consensus.cc:2889] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:41.234627  1219 consensus_queue.cc:1243] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:05:41.235255  1149 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.282420 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.294822 29139 raft_consensus.cc:2393] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:41.296702 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.298560 29148 pending_rounds.cc:85] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 5
I20260709 13:05:41.298954 29148 pending_rounds.cc:107] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:05:41.299198 29148 raft_consensus.cc:2889] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:41.300408 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.301203 29144 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:41.303330 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.306294 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.307049 29142 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:41.310235 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:41.316256  1046 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:05:41.266959  1249 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.013s	sys 0.019s
I20260709 13:05:41.318236 29911 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:41.262387  1056 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:41.293512  1061 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:41.326581 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.241969  1314 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:41.321902 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:41.320199  1249 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.014s	sys 0.020s
W20260709 13:05:41.293699  1305 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:05:41.266976  1300 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:41.329337 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.330405 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.235882  1289 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.331089 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.342623 29142 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:41.268081  1191 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:41.348867 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.343647 29143 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:41.316717  1046 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.276651  1198 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:41.312849  1253 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:41.368541 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.369832 29139 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:05:41.341907 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 105562us
I20260709 13:05:41.372049 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.372696 29145 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:41.372969 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:41.340957  1323 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:05:41.383343 29140 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:41.372434 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:41.375007 29142 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:41.384356 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.326607  1061 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:41.369278  1046 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:41.329312  1295 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:41.318840  1313 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:05:41.396157  1227 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.396906  1061 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.398288  1061 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.398435  1227 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.401772  1205 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.402196 29217 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:05:41.402585  1150 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:41.402866  1036 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:41.403076  1150 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.403340  1036 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:41.403611  1150 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:41.407383  1198 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:41.385051 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:41.339336   737 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:41.346347  1300 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:41.408407 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 98292.6us
I20260709 13:05:41.415277   737 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.418022   737 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:41.419423 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:41.423964 29215 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:41.424998 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:41.426964  1089 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:41.427484  1089 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:41.427965  1089 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:41.428047  1113 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:41.428669  1113 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.430982  1113 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.431526  1113 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:41.432098 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 180796us
I20260709 13:05:41.433104  1105 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:41.433507  1105 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:41.433869 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:41.426887 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 208179us
I20260709 13:05:41.438011  1105 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.438575 29070 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:41.340461  1217 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:05:41.444684  1211 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.445225  1314 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:41.410044  1287 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:05:41.348773  1225 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:05:41.403831  1036 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:41.408017  1198 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:41.411305  1300 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:41.450736 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:41.451363 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 66374.7us
I20260709 13:05:41.459882 29188 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:41.461396 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48208.7us
I20260709 13:05:41.464789 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.465467 29069 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:41.469655 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.470547 29064 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:41.442879   737 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.448823  1171 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.448982   948 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.453459  1105 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.488489 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:41.489130 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55326.7us
W20260709 13:05:41.489578 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 226422us
I20260709 13:05:41.402290  1205 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.490326 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:41.450979  1253 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:41.490688 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:41.489732 29065 raft_consensus.cc:2410] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:05:41.492151 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet 1fdfda5dcdd94083afe1620564f9d376: Illegal state: Replica 6ad9faa1c14444b29c9db8e71f46e2d6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:05:41.495167 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.496380 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.496564 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.497434 29064 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:05:41.498127 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.502112 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:41.504657 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:41.443085  1113 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.506791 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:41.473510  1341 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:41.453841  1211 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.515781 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:41.501145  1337 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.499738  1320 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.500216  1330 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:41.505088  1201 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:41.520609 29064 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:41.511767  1322 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:41.514081  1298 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:41.536311 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.537580 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:41.509763  1343 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:41.492725  1171 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.509447  1092 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:05:41.513980  1295 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.514168  1254 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:41.532815  1201 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.542861 29142 raft_consensus.cc:2368] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10.
I20260709 13:05:41.515625  1341 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.544140 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.544762 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.509099  1181 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:05:41.545449 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.489327   737 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.546159 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.523909  1249 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:41.539707  1322 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:41.447082  1318 consensus_queue.cc:1175] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "730186a8c70f43f2a57d80163bbc37d2" responder_term: 14 status { last_received { term: 7 index: 4 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: [9.5-13.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:41.522020  1293 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:41.532477  1314 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.525756  1198 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.550123 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.550743  1056 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:41.551154  1332 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:05:41.525254  1330 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.545599  1343 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.557876 29064 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:41.500581   948 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.540520  1166 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.568420 29039 leader_election.cc:400] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10.
I20260709 13:05:41.597844 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.601171 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:41.603551 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.605038 29039 leader_election.cc:403] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:05:41.555483 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:41.602927 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.606235 29065 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:41.606372 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 50986us
I20260709 13:05:41.603684 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.608676 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.609170 29071 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
W20260709 13:05:41.577677  1345 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:05:41.602514  1330 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:41.611534   948 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.612093  1329 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:41.612394  1219 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10.
W20260709 13:05:41.573462  1339 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:05:41.614511 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.617955 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:41.584189   985 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.004s
I20260709 13:05:41.618664 29146 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:41.619170 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.620286 29065 raft_consensus.cc:2410] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:05:41.614693 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:41.620924 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.623961 29188 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:41.624982 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.625150  1195 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:41.569428  1201 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.579350  1341 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:41.625670  1195 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.578955  1254 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:41.601076  1056 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:41.600072  1322 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:41.590543  1211 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:05:41.569199   999 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.627297 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:41.589269  1205 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.548486  1298 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.629072 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.647207 29065 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:41.648257 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.599870  1249 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.581271  1171 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:41.601600  1089 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.532603  1337 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.629071  1195 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:41.615180  1289 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.606189  1343 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:41.651993 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:41.650274  1350 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:41.654127  1254 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.656225  1089 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:41.660295 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:41.680333 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.620069  1166 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.631246   948 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:05:41.665171  1331 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.261s	user 0.003s	sys 0.000s
I20260709 13:05:41.602882  1293 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.556959  1253 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.669329   999 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.651684  1249 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.690394   948 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:41.673450  1350 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.715252 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:41.715348 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:41.636014  1150 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.717548 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:41.661965  1113 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.715948 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:41.729619  1338 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.337s	user 0.000s	sys 0.003s
I20260709 13:05:41.749387 29072 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:41.733734 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.679045  1171 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:41.735404 29909 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:41.745370 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:41.750535 29141 raft_consensus.cc:2410] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:05:41.751080 29113 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:41.751202 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:41.741735 29064 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:41.742991 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:41.745260 29185 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:41.722674 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:41.723315  1089 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:41.734997  1342 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.328s	user 0.002s	sys 0.002s
I20260709 13:05:41.761322 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:41.762024 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 49672.4us
I20260709 13:05:41.763381  1344 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:41.739725  1350 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.818698  1067 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:41.825009  1347 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:05:41.828583   737 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:41.832172   948 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.833585   999 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:41.835141  1067 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:41.837219  1112 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.837766  1112 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.838581  1298 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:41.840580  1067 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.842044  1300 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.757370  1168 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:41.748045  1292 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:41.811045 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 49850.9us
I20260709 13:05:41.748834  1092 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:41.815990 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 64862us
W20260709 13:05:41.816587 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 97256.3us
I20260709 13:05:41.762147  1171 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.760452  1061 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:41.848141 29067 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:41.850247 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:41.858687 29143 raft_consensus.cc:2410] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:41.851665 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:41.758287  1321 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:41.853677 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.854285 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:41.859577 29147 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:05:41.860992 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:41.861047 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.865097 29071 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:41.873548 29215 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:05:41.873692 29185 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:41.882956 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:41.845436  1112 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:41.866354  1356 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:05:41.762650  1089 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.858999  1340 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:41.899734 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:41.861938  1171 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:41.876883  1172 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:41.879848  1055 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:41.861693  1321 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:41.922148 29147 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:05:41.864295  1330 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.924613  1036 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:41.975008  1036 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:41.928287  1344 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:41.977409 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:41.977684 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:41.978199 29064 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:41.978576   999 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:41.979147   999 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:41.980072   999 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.982447  1337 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:41.984674 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:41.984968 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:41.984671  1360 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:05:41.986179 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:41.986902 29222 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 13: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:41.986325  1350 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:41.987741 29148 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:41.991230 29069 raft_consensus.cc:2368] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11.
I20260709 13:05:41.988409 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:41.995042 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:41.995628 29216 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
W20260709 13:05:41.997314 29140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 705) took 1434 ms (client timeout 1694 ms). Trace:
I20260709 13:05:41.999361 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:41.909914  1300 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:42.023748 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:42.000813 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:41.987689 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:41.997845 29140 rpcz_store.cc:269] 0709 13:05:40.562989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.582599 (+ 19610us) service_pool.cc:224] Handling call
0709 13:05:41.997102 (+1414503us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b54002e8954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:41.997229 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1621831552}
I20260709 13:05:42.004693  1150 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:42.005236  1350 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:42.009037 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:42.027647 29141 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:42.027757 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:42.009763  1354 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:05:42.010244 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:42.010542  1359 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:05:42.011008 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:42.011962  1351 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 13:05:41.924042  1321 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:41.913265  1166 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:42.035247 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:42.032087 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:42.032930 29214 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 13: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:42.033787 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 219645us
I20260709 13:05:42.044618 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:42.050597 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:42.051594 29144 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:05:42.052439  1361 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:05:42.053285 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:42.053881 29068 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:05:42.056242 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:42.057906 29215 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:41.990119  1330 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:42.023852  1318 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:42.044351  1205 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:42.091360 29068 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:42.116360 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:42.117376 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.137339 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:42.163259 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47039.3us
I20260709 13:05:42.164178 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:42.165997 29214 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:42.165786 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:42.101491  1046 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.710s	user 0.011s	sys 0.001s
I20260709 13:05:42.079622  1067 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:42.177560 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:42.164052  1181 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.162390  1105 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:42.070186  1343 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:42.234364 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 56894.1us
I20260709 13:05:42.109280  1318 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:42.184233  1067 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:42.180981 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:42.225052 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 351418us
I20260709 13:05:42.277609 29068 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:42.277967 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:42.169874  1227 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:42.191066  1329 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:42.107825  1205 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.295902 29219 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:42.245317 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:42.297276 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 52109.6us
W20260709 13:05:42.235667 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 258922us
I20260709 13:05:42.094245  1293 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:42.304873 29038 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:42.189531  1046 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.303602 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:42.235731  1219 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:42.334328 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:42.334110  1321 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.310459 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:42.305258  1205 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:42.318030  1089 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:42.323210  1295 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.764s	user 0.008s	sys 0.000s
I20260709 13:05:42.234380  1181 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.310428  1112 raft_consensus.cc:2764] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:05:42.324188  1055 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:42.324703  1322 consensus_meta.cc:287] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.655s	user 0.008s	sys 0.003s
W20260709 13:05:42.325076  1320 consensus_meta.cc:287] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.785s	user 0.010s	sys 0.001s
I20260709 13:05:42.272572  1166 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:42.384604 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 74220.9us
I20260709 13:05:42.327538 29222 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:42.330713  1056 consensus_meta.cc:287] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.679s	user 0.012s	sys 0.000s
I20260709 13:05:42.289800  1067 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:42.290728  1314 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:42.326972  1046 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:42.384298  1329 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:42.390352 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:42.362735  1249 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.000s
I20260709 13:05:42.352929  1293 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:42.388553  1295 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:42.370735 29068 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:42.409075 29147 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:42.335063  1227 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:42.391831  1320 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:42.416921 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:42.427037 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:42.440883 29147 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
W20260709 13:05:42.421870  1170 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.000s	sys 0.015s
I20260709 13:05:42.392444  1055 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:42.454713  1366 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:05:42.454638  1227 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.459610  1293 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:42.400902  1056 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.507449  1293 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:42.509099  1056 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:42.400081  1166 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.509680  1293 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:42.511171  1166 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:42.500038 29068 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:42.514118 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:42.516645 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:42.433382   999 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:42.444715  1350 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:05:42.451571  1341 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.823s	user 0.005s	sys 0.011s
I20260709 13:05:42.505191  1343 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:42.514312  1172 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.601s	user 0.005s	sys 0.005s
I20260709 13:05:42.533289  1172 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:42.534245 29216 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:42.392709  1322 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:42.450397  1170 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.000s	sys 0.016s
I20260709 13:05:42.536051 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:42.519263 29068 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:42.446779  1112 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:42.537438 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:42.537400  1322 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:42.545439 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:42.546883 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:42.555523 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:42.566700 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:42.565825 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 394690us
I20260709 13:05:42.585865 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:42.587831 29217 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:42.592902 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:42.445158  1249 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.582830  1120 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:42.582996  1046 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:42.591289  1170 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.591851  1314 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:42.591507  1350 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:42.647003  1201 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.021s	user 0.007s	sys 0.007s
I20260709 13:05:42.684989  1055 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:42.686705  1195 consensus_meta.cc:287] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.021s	user 0.011s	sys 0.001s
W20260709 13:05:42.688539  1171 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.000s
I20260709 13:05:42.666065  1120 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:42.692330  1289 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.680789  1046 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:42.696465  1369 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
I20260709 13:05:42.697005  1112 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:05:42.698074  1363 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.876s	user 0.000s	sys 0.002s
I20260709 13:05:42.699052 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:42.700372  1181 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:42.700944 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:42.700990  1318 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:42.584800  1341 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.702406  1321 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:42.680765   999 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:42.702977 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:42.703635  1219 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.702157 29144 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:05:42.715410 29214 raft_consensus.cc:2368] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:05:42.716580 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:42.665421  1364 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.715745  1170 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.748657 29141 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:42.719695  1201 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:42.713608  1350 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.725250  1171 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:42.741606 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 437261us
I20260709 13:05:42.738790  1120 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:42.712913  1314 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:42.711870  1168 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.868s	user 0.010s	sys 0.000s
I20260709 13:05:42.715165  1249 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:42.759851  1140 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.364s	user 0.051s	sys 0.054s
I20260709 13:05:42.786362 29039 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:42.817973 29065 raft_consensus.cc:2368] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14.
I20260709 13:05:42.721755  1195 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.814313 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 576) took 1210 ms. Trace:
I20260709 13:05:42.804724 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:42.807893 29144 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:42.744576  1289 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:42.815574  1168 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:42.816447  1385 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:42.790638  1321 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:42.821295 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 235852us
I20260709 13:05:42.821506 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:42.818894 29071 rpcz_store.cc:276] 0709 13:05:41.603959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.873838 (+269879us) service_pool.cc:224] Handling call
0709 13:05:41.874177 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:41.891938 (+ 17761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.891984 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:41.899337 (+  7353us) log.cc:844] Serialized 26 byte log entry
0709 13:05:42.220396 (+321059us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:42.220459 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.220676 (+   217us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.814007 (+593331us) raft_consensus.cc:1685] finished
0709 13:05:42.814038 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.814226 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":319,"raft.queue_time_us":468,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":7470,"thread_start_us":307749,"threads_started":1,"wal-append.queue_time_us":494984}
I20260709 13:05:42.783087  1235 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:42.824856 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:42.828588 29186 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.032s	sys 0.040s
I20260709 13:05:42.779433   999 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.860772 29218 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:05:42.822929  1140 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.422s	user 0.052s	sys 0.054s
I20260709 13:05:42.872589 29221 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:42.862833 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:42.868033 29186 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.033s	sys 0.040s
I20260709 13:05:42.782166  1219 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.785760  1318 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:05:42.864051 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:42.784878  1364 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.872627 29144 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:42.899067 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:42.873960 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 575283us
I20260709 13:05:42.826848  1181 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:42.864064  1249 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:42.922116 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:42.907709  1055 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:42.875998 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:42.910497  1330 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:42.917636 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 467971us
I20260709 13:05:42.834038 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:42.923153 29146 raft_consensus.cc:2410] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:05:42.906701 29217 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:05:42.925364 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2457) took 1254 ms. Trace:
I20260709 13:05:42.875557  1314 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:42.896705  1235 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:42.973801 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:42.987325 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:42.988512 29144 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:42.937803  1341 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:42.939244  1055 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:42.953127 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:42.918256  1386 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.956485  1046 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:42.957489  1371 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 13:05:42.918077  1322 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.959146  1170 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:42.874303  1321 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:42.917622  1140 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:42.916501  1291 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
W20260709 13:05:42.936488  1358 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:05:42.930485  1249 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:42.945237  1383 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:05:42.943760  1382 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:05:42.961513 29142 rpcz_store.cc:276] 0709 13:05:41.670542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.671054 (+   512us) service_pool.cc:224] Handling call
0709 13:05:41.671876 (+   822us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:41.672430 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.672484 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:41.673086 (+   602us) log.cc:844] Serialized 52 byte log entry
0709 13:05:41.759943 (+ 86857us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:41.849959 (+ 90016us) log.cc:844] Serialized 14 byte log entry
0709 13:05:41.855165 (+  5206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.855266 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.642048 (+786782us) spinlock_profiling.cc:243] Waited 38.4 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:42.914305 (+272257us) raft_consensus.cc:1685] finished
0709 13:05:42.924900 (+ 10595us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.925259 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7343,"prepare.queue_time_us":63977,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":37540,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":13970,"spinlock_wait_cycles":88235264,"thread_start_us":32598,"threads_started":1,"wal-append.queue_time_us":138703}
I20260709 13:05:43.091346 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:42.912918  1181 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.998184 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.000049 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:42.970793  1373 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:43.021688 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:42.971772  1340 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:42.962298  1330 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.116596 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.117470 29142 raft_consensus.cc:2410] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:05:43.117842  1330 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:42.937501 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:43.120208 29145 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:43.122412  1385 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:43.123006  1385 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.123459  1385 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:43.042918 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 243282us
W20260709 13:05:43.021981  1253 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.050s	user 0.013s	sys 0.042s
I20260709 13:05:43.002661  1314 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:05:43.134384 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 210187us
W20260709 13:05:43.077854 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 213950us
I20260709 13:05:43.030161  1372 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.021569  1386 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.114424  1343 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:43.115897 29140 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:05:43.116050  1357 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.295s	user 0.006s	sys 0.003s
W20260709 13:05:43.126487 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2225) took 1581 ms (client timeout 1559 ms). Trace:
I20260709 13:05:43.107631  1181 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:43.133919 29145 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:43.000576  1055 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:43.108243 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 108338us
I20260709 13:05:43.193022 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:43.195379 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.119715  1330 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:43.157044 29067 rpcz_store.cc:269] 0709 13:05:41.544637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.544760 (+   123us) service_pool.cc:224] Handling call
0709 13:05:43.126184 (+1581424us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54003edfd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:43.126390 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577572096}
I20260709 13:05:43.208351 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:43.132612 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:43.130522  1253 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.158s	user 0.013s	sys 0.042s
I20260709 13:05:43.051728  1322 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.170317 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:43.211300 29143 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:43.170823 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.200493 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.212713 29145 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:43.202888 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 63698.2us
I20260709 13:05:43.076859  1140 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.136224  1289 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.136677  1219 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.206357 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.209537 29071 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:43.257040 29140 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:05:43.264353 29216 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:05:43.142931  1195 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:05:43.156623   737 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.290891 29143 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:05:43.303273 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2233) took 1456 ms. Trace:
I20260709 13:05:43.304260 29148 rpcz_store.cc:276] 0709 13:05:41.846916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.847283 (+   367us) service_pool.cc:224] Handling call
0709 13:05:41.847561 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:41.993302 (+145741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.302963 (+1309661us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54003a4dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:43.303195 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321851136}
I20260709 13:05:43.321594 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2239) took 1464 ms. Trace:
I20260709 13:05:43.322031 29072 rpcz_store.cc:276] 0709 13:05:41.856828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.542575 (+685747us) service_pool.cc:224] Handling call
0709 13:05:42.542874 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.543093 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:42.931740 (+388647us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.321176 (+389436us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:43.321231 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:43.321322 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.321407 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.321537 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86034,"prepare.queue_time_us":564887,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":108,"spinlock_wait_cycles":3328,"thread_start_us":734514,"threads_started":2}
I20260709 13:05:43.323712 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:43.322844  1295 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:43.338938  1406 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:05:43.340191  1410 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.340754  1410 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:43.341964  1392 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:05:43.131462  1340 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.343098  1410 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.330492  1209 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.344424  1209 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.345180  1340 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.212324  1364 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.345688 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2330) took 1909 ms. Trace:
I20260709 13:05:43.192138  1056 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.137s
I20260709 13:05:43.346585  1209 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.266536  1330 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.348650  1386 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.350050  1330 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:43.350404  1067 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.918s	user 0.011s	sys 0.001s
I20260709 13:05:43.350612  1330 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.351145  1330 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.131258  1373 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.353430  1373 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:43.353505  1120 consensus_meta.cc:287] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.003s
I20260709 13:05:43.354132  1120 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:43.356473  1205 consensus_meta.cc:287] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.963s	user 0.013s	sys 0.000s
I20260709 13:05:43.357116  1205 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.358954 29144 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
W20260709 13:05:43.359560  1398 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:05:43.159070  1372 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.256142  1320 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.360219 29139 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:05:43.315656  1289 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.360685  1320 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.361337  1289 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.361725 29072 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:43.362785  1320 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.362968  1322 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.338594  1295 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:43.367635 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 241508us
I20260709 13:05:43.360173 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.368566 29216 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:43.333168 29148 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79cdfc6a725145348b70034422ef477b"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:39218
W20260709 13:05:43.341217  1395 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.250s	user 0.000s	sys 0.000s
W20260709 13:05:43.337553 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50376.9us
I20260709 13:05:43.343122 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 582) took 1707 ms. Trace:
I20260709 13:05:43.302130  1253 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:43.370858 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 177928us
I20260709 13:05:43.346082 29066 rpcz_store.cc:276] 0709 13:05:41.436101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.437198 (+  1097us) service_pool.cc:224] Handling call
0709 13:05:41.437628 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:41.509822 (+ 72194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.509853 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:41.510339 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 13:05:41.736122 (+225783us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:41.736160 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.736233 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:43.115448 (+1379215us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:43.345395 (+229947us) raft_consensus.cc:1685] finished
0709 13:05:43.345416 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.345628 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69314,"raft.queue_time_us":74462,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":8496,"spinlock_wait_cycles":56553728,"thread_start_us":200871,"threads_started":1,"wal-append.queue_time_us":300284}
I20260709 13:05:43.369798 29148 raft_consensus.cc:606] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:43.350978  1067 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.373167 29871 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:43.388527 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:43.363649  1289 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.400471 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:43.401971 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:43.402614 29066 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:43.408593 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 79258.7us
I20260709 13:05:43.395932 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:43.422043 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.331315  1254 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.361703  1372 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.436223 29067 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:43.432843  1386 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.438060  1386 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.323792   737 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.442239   737 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.376364  1205 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.453912 29073 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:43.442967   737 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:43.509637  1391 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:05:43.421573  1049 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:43.517897  1397 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:43.435099  1254 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.518406 29067 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:43.521900  1254 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.524325  1273 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.525069  1273 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.528968  1295 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.536366  1429 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:43.384987 29071 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:43.373009 29068 rpcz_store.cc:276] 0709 13:05:41.636140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.538161 (+902021us) service_pool.cc:224] Handling call
0709 13:05:42.538446 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:42.539026 (+   580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.539053 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:42.539747 (+   694us) log.cc:844] Serialized 26 byte log entry
0709 13:05:42.921443 (+381696us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:42.921511 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.921601 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:43.342839 (+421238us) raft_consensus.cc:1685] finished
0709 13:05:43.342861 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.343039 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"thread_start_us":380953,"threads_started":1,"wal-append.queue_time_us":510174}
I20260709 13:05:43.539367 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:43.540027 29068 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:43.541173 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:43.542451 29068 raft_consensus.cc:2410] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:43.544698 29066 raft_consensus.cc:2410] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:05:43.434844  1195 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:43.433159  1320 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:43.523661  1411 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:43.567698 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.581918 29213 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
W20260709 13:05:43.550941 29064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.007s	sys 0.003s
W20260709 13:05:43.584535 29064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.008s	sys 0.003s
W20260709 13:05:43.572614 29872 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.006s	sys 0.007s
I20260709 13:05:43.444564  1289 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:43.587170 29872 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.007s	sys 0.007s
I20260709 13:05:43.536410  1415 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.547729  1219 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.565315 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2355) took 1460 ms. Trace:
I20260709 13:05:43.509717   737 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:43.583169 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.596899  1415 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.603971  1195 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:43.602276 29066 rpcz_store.cc:276] 0709 13:05:42.104705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.564120 (+1459415us) service_pool.cc:224] Handling call
0709 13:05:43.564802 (+   682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.564986 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.565011 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:43.565055 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:43.565088 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.565140 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.565252 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:43.609107  1417 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:43.583086  1321 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.597049  1414 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:43.629086 29066 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:43.633313 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2243) took 1182 ms. Trace:
I20260709 13:05:43.635267 29066 rpcz_store.cc:276] 0709 13:05:42.450965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.627639 (+1176674us) service_pool.cc:224] Handling call
0709 13:05:43.628844 (+  1205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.632908 (+  4064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.633234 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:43.601256  1120 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.537242  1429 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.445096  1055 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.599071  1219 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.584023  1320 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.618304 29213 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 8.
W20260709 13:05:43.632174 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 238041us
I20260709 13:05:43.608659  1407 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:43.618861 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:43.627173  1254 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:43.608012  1273 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.639376 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.640244 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:43.640739 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 313062us
I20260709 13:05:43.640978 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2357) took 1064 ms. Trace:
I20260709 13:05:43.595676  1289 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.624574  1417 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:43.619225 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:43.644923 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.659000 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.664923 29909 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
W20260709 13:05:43.665093 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 45990.3us
I20260709 13:05:43.656916 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.661926 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.660960 29066 rpcz_store.cc:276] 0709 13:05:42.576823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.640216 (+1063393us) service_pool.cc:224] Handling call
0709 13:05:43.640496 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.640665 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.640687 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:43.640726 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:43.640753 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.640811 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.640929 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:43.662809 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44087.1us
I20260709 13:05:43.665843 29903 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:43.667097 29038 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:43.667411  1421 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:43.637456  1140 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.589848  1411 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.663013 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.680085  1408 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:43.680219  1181 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.680475  1295 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:43.669255  1049 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:43.614727  1385 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.681066  1330 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.683401 29071 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:43.683776  1425 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
W20260709 13:05:43.683811 29113 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.829s (SENT) [suppressed 1 similar messages]
I20260709 13:05:43.664343  1386 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.667953  1421 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.680733  1253 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:43.662595  1254 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.680132  1067 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.687826 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:43.685889  1408 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.698046 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:43.705109  1426 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.714401 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.695217 29066 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:43.633687  1414 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.708626 29213 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:43.711884 29186 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:43.687274  1295 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.715906 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2358) took 1132 ms. Trace:
I20260709 13:05:43.722151 29066 rpcz_store.cc:276] 0709 13:05:42.584039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.670545 (+1086506us) service_pool.cc:224] Handling call
0709 13:05:43.694715 (+ 24170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.715656 (+ 20941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.715851 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:43.718421 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:43.718864  1411 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.718952  1386 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:43.726213 29186 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:43.714270  1253 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.731947  1049 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:43.658089  1407 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.731994 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.733008 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:43.714511  1408 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:43.732440  1049 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.723731  1295 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:43.727533  1420 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.735221 29903 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:43.729923  1386 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.730561  1415 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.758803 29186 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:43.760630 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.761328 29066 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:05:43.764986 29066 pending_rounds.cc:85] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 6
I20260709 13:05:43.751202 29213 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
W20260709 13:05:43.736512 29221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2436) took 1752 ms (client timeout 2097 ms). Trace:
W20260709 13:05:43.765939 29221 rpcz_store.cc:269] 0709 13:05:41.984246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.984401 (+   155us) service_pool.cc:224] Handling call
0709 13:05:43.736287 (+1751886us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b54000799d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:43.736451 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2011492608}
I20260709 13:05:43.767246 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.768468 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.734397  1056 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:43.754074 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.754033  1418 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:43.774966 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:43.772563  1429 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.765384 29066 pending_rounds.cc:107] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:05:43.775945 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.779953 29066 raft_consensus.cc:2889] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:43.752786  1049 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:43.777235 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 110206us
I20260709 13:05:43.770191 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:43.783018  1414 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:43.685146  1195 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:43.817170 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 102392us
I20260709 13:05:43.757400  1434 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.773705  1209 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:43.773455  1112 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:43.816108 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:43.831691 29066 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:43.720085  1426 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:43.755596  1289 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.764891  1386 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:43.849232 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:43.850237 29066 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:43.854066 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.855794 29902 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:43.778151  1340 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:05:43.860504 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:43.864970 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:43.866506 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.884824 29911 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:43.867287 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.885954 29903 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:43.868150 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.868111 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:43.760300  1420 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.837484  1209 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.887388 29066 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:43.869683  1430 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:43.873656 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.891952 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.892392 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.892772 29222 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:43.893985 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:43.876152 29147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2466) took 1867 ms (client timeout 1532 ms). Trace:
I20260709 13:05:43.820222  1428 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:43.879844 29037 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:43.881286 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 111143us
W20260709 13:05:43.874729 29069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2339) took 2398 ms (client timeout 2188 ms). Trace:
W20260709 13:05:43.895056 29147 rpcz_store.cc:269] 0709 13:05:42.008908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.010715 (+  1807us) service_pool.cc:224] Handling call
0709 13:05:43.875972 (+1865257us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54003d36d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:43.876086 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":870711552}
I20260709 13:05:43.901167 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.901823 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.787160  1418 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.903858 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:43.899735 29069 rpcz_store.cc:269] 0709 13:05:41.476152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.476347 (+   195us) service_pool.cc:224] Handling call
0709 13:05:43.874358 (+2398011us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54003bbfd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:43.874521 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136344320}
W20260709 13:05:43.911293 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 228049us
I20260709 13:05:43.896759 29222 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:43.915072 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.909122 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:43.916023 29222 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:43.918331 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.918990 29222 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:05:43.920209 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.918617 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:43.922240 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:43.862264  1425 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.898499  1209 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:43.930835 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.939618 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:43.848104  1112 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.905855  1094 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:43.906834  1205 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:43.899507  1385 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.950706 29148 raft_consensus.cc:2368] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
W20260709 13:05:43.951215 29073 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2235) took 2403 ms (client timeout 1816 ms). Trace:
W20260709 13:05:43.982271 29073 rpcz_store.cc:269] 0709 13:05:41.547359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.600912 (+ 53553us) service_pool.cc:224] Handling call
0709 13:05:43.951035 (+2350123us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54002ace54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:43.951147 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3951974272}
W20260709 13:05:43.983206 29219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2410) took 2237 ms (client timeout 1829 ms). Trace:
W20260709 13:05:43.983685 29219 rpcz_store.cc:269] 0709 13:05:41.714980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.715106 (+   126us) service_pool.cc:224] Handling call
0709 13:05:43.951908 (+2236802us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b54000be854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:43.952080 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1283158784}
I20260709 13:05:43.984011 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:43.985026 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:43.985030 29073 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:43.918728  1430 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:43.987128 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:43.987181 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:43.916384  1415 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:43.905553  1120 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:43.910698  1414 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:43.922267  1181 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:43.922930  1320 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:43.944664  1393 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.746s	user 0.000s	sys 0.001s
I20260709 13:05:43.951692 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.991674 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.930274  1105 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:43.991674 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:43.939407  1289 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:43.926062  1440 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:43.944376  1407 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.004525 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.039062 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:43.852835  1434 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:43.990701  1430 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:44.006423  1414 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.044831 29219 raft_consensus.cc:2368] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11.
W20260709 13:05:44.045876 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 92007.9us
I20260709 13:05:43.996934  1427 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.977124  1094 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.065053 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.065973 29219 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:44.067123 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.042132  1385 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:44.072273 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2448) took 1550 ms. Trace:
I20260709 13:05:43.912554  1428 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.043253 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:44.072790 29218 rpcz_store.cc:276] 0709 13:05:42.521736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.547788 (+ 26052us) service_pool.cc:224] Handling call
0709 13:05:42.580261 (+ 32473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.869510 (+289249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.072019 (+1202509us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54003cd2d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:44.072210 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642632576}
I20260709 13:05:44.074186 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.076414  1418 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.078701  1414 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.082135  1171 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.040625  1105 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:44.086962 29910 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:44.087072 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:44.088925 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.089627 29216 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:44.053113  1289 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:44.093504 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:44.090216 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:44.094743 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.088618 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:43.995883  1205 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.059137  1181 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:44.036865  1120 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.115442 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.122262 29216 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:44.060176  1320 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:44.051059  1440 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:44.113345 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:44.117170 29218 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:44.083869  1105 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:44.114343  1289 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:43.939803  1425 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.088752  1385 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:44.131215 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.112344  1237 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:44.125348   999 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.134526  1171 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:44.145148 29147 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:05:44.138752 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2349) took 2463 ms. Trace:
I20260709 13:05:44.210485 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.211455 29218 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:44.214319  1205 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.210354 29070 rpcz_store.cc:276] 0709 13:05:41.675250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.538787 (+863537us) service_pool.cc:224] Handling call
0709 13:05:42.539082 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:42.540157 (+  1075us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:42.540187 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:42.540621 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 13:05:42.904723 (+364102us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:42.904768 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.904837 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:43.876859 (+972022us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:44.126645 (+249786us) raft_consensus.cc:1685] finished
0709 13:05:44.126666 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.138674 (+ 12008us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":908,"raft.run_cpu_time_us":41,"raft.run_wall_time_us":41,"spinlock_wait_cycles":25177344,"thread_start_us":363694,"threads_started":1,"wal-append.queue_time_us":493522}
I20260709 13:05:44.222244 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:44.222883 29070 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:44.201900 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:44.135147  1441 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:05:44.132831  1320 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.239667  1428 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:44.240176  1320 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.134513  1253 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.240861  1426 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.125792  1195 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:44.209076  1171 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.232559  1420 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.220772  1237 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.124675  1181 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.209784  1385 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:44.132262  1414 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:44.244786  1425 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:44.246176 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44385.2us
I20260709 13:05:44.205021  1445 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:44.223320 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.205092  1450 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.110s
I20260709 13:05:44.248034 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.205863  1415 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.133337  1440 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:44.204973  1321 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:44.134939  1418 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 13:05:44.250212   999 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:44.254837  1414 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:44.261248 29145 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:44.250483  1171 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:44.268486 29145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:05:44.223806  1448 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:44.250310  1195 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.270002 29216 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:05:44.269695 29145 raft_consensus.cc:606] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:44.265295  1321 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.272141  1094 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:44.272854  1439 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 13:05:44.093336  1427 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.277343 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.269451  1414 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:44.265399   999 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:44.284654 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:44.284727 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:44.266669  1445 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:44.285079 29218 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:05:44.287961 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.289115 29216 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:44.264444  1415 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.275826  1321 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:44.316270 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.317099 29216 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:44.319114 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:44.320215  1049 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.512s	user 0.005s	sys 0.005s
I20260709 13:05:44.319885  1209 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.320619  1049 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.322568  1209 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.323017 29219 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:44.326233  1295 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.566s	user 0.005s	sys 0.004s
I20260709 13:05:44.326854  1295 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.329336  1105 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.331969  1105 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.339296 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:44.341121  1384 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.545s	user 0.008s	sys 0.000s
I20260709 13:05:44.288403   999 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:44.341624  1445 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.354553  1434 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.322597  1049 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:44.356101  1443 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:05:44.282063  1448 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.368567 29216 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:44.373333 29070 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:44.381258  1181 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.382777  1415 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.356755  1120 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.367678  1434 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.352783  1386 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:44.399292 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 309139us
I20260709 13:05:44.410043 29113 leader_election.cc:400] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11.
I20260709 13:05:44.410434 29113 leader_election.cc:403] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:05:44.412818 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:44.388827  1237 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:44.398672  1379 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.451s	user 0.022s	sys 0.047s
I20260709 13:05:44.416463 29221 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 12: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:44.427096 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2248) took 1468 ms. Trace:
I20260709 13:05:44.415822  1120 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11.
I20260709 13:05:44.424925  1237 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:44.431038 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 316767us
I20260709 13:05:44.383966  1430 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.433789 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:44.423867  1408 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.669s	user 0.006s	sys 0.006s
W20260709 13:05:44.421626  1379 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.482s	user 0.022s	sys 0.047s
I20260709 13:05:44.423243  1094 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:44.427999 29070 rpcz_store.cc:276] 0709 13:05:42.958435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.376804 (+1418369us) service_pool.cc:224] Handling call
0709 13:05:44.426431 (+ 49627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.426664 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.426701 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.426760 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:44.426805 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.426881 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.427022 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:44.422003  1450 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:44.393283  1434 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.476001 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2250) took 1517 ms. Trace:
I20260709 13:05:44.440357  1094 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.444972  1386 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.436815  1237 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.446523  1440 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.441339  1408 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.458514  1171 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:44.478014 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 387779us
I20260709 13:05:44.434391  1195 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:44.421501  1289 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:44.441309 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 398133us
I20260709 13:05:44.442801  1427 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.498165  1289 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.485435  1094 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:44.483577 29070 rpcz_store.cc:276] 0709 13:05:42.958981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.475129 (+1516148us) service_pool.cc:224] Handling call
0709 13:05:44.475428 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.475624 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.475646 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.475688 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:44.475719 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.475777 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.475893 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:44.499771 29073 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 15: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:44.477857  1450 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.503376 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:44.495643  1440 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.506906 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:44.507241 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:44.491422  1295 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.509449 29070 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:44.510689 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2251) took 1551 ms. Trace:
I20260709 13:05:44.511204 29070 rpcz_store.cc:276] 0709 13:05:42.959284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.508859 (+1549575us) service_pool.cc:224] Handling call
0709 13:05:44.509216 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.510371 (+  1155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.510619 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:44.513108 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:44.515244 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2252) took 1555 ms. Trace:
I20260709 13:05:44.515803 29070 rpcz_store.cc:276] 0709 13:05:42.959547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.514435 (+1554888us) service_pool.cc:224] Handling call
0709 13:05:44.514781 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.514969 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.514993 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.515029 (+    36us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:44.515059 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.515107 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.515192 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:44.522243 29065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 583) took 2882 ms (client timeout 2075 ms). Trace:
I20260709 13:05:44.522454 29070 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:44.523438 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2255) took 1563 ms. Trace:
I20260709 13:05:44.523968 29070 rpcz_store.cc:276] 0709 13:05:42.960481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.517191 (+1556710us) service_pool.cc:224] Handling call
0709 13:05:44.517562 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.523109 (+  5547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.523355 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:44.524972 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2473) took 1826 ms. Trace:
I20260709 13:05:44.517148 29038 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:44.502055  1198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.790s	user 0.035s	sys 0.027s
W20260709 13:05:44.522861 29065 rpcz_store.cc:269] 0709 13:05:41.640361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.648011 (+  7650us) service_pool.cc:224] Handling call
0709 13:05:44.521965 (+2873954us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400383954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:44.522150 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2687979264}
I20260709 13:05:44.550334  1430 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.551424  1440 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:44.551950  1440 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.552383  1440 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:44.525381 29146 rpcz_store.cc:276] 0709 13:05:42.698623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.700692 (+  2069us) service_pool.cc:224] Handling call
0709 13:05:44.524763 (+1824071us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400041854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:44.524913 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385935232}
I20260709 13:05:44.561544 29065 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:05:44.563315 29065 pending_rounds.cc:85] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 4
I20260709 13:05:44.564028 29065 pending_rounds.cc:107] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 10.5
W20260709 13:05:44.560715 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 43637us
I20260709 13:05:44.564457 29070 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:44.502316   948 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:05:44.565546 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:44.562819  1198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.850s	user 0.037s	sys 0.028s
I20260709 13:05:44.562219  1448 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:44.567625  1434 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.513198  1218 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:44.520764  1195 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.583598 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:44.594419  1218 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:44.594446  1195 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.595470  1218 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.559372  1425 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.565888  1450 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.585813 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:44.526670  1456 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:44.587337 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.599516 29903 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:44.596568 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:44.600571 29910 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:44.564795 29065 raft_consensus.cc:2889] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:44.588356  1171 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.578296 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:44.610212 29070 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:44.612053 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:44.512321  1385 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.621948 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51814 (request call id 10) took 3072 ms. Trace:
I20260709 13:05:44.520802  1433 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:05:44.577553  1408 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.630736 29220 rpcz_store.cc:276] 0709 13:05:41.536409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.536597 (+   188us) service_pool.cc:224] Handling call
0709 13:05:44.603021 (+3066424us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54001329d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b13735 00007f783bad3d29 00007f78350521d4 00007f7835052162 00007f7835052091 00007f783505200c 00007f78350512c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:44.608818 (+  5797us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2534963072}
I20260709 13:05:44.639945 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.641485 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.643204 29220 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:44.647797 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.648597 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.648820 29146 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 8.
I20260709 13:05:44.648936 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.651726 29910 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:44.652077 29220 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:44.653059 29186 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:44.649240 29143 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:44.649255 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:44.629290  1434 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:44.654973 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.666021 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:44.668568 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:44.670050 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.671087 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:44.672259 29070 raft_consensus.cc:2368] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14.
I20260709 13:05:44.673770 29871 leader_election.cc:400] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14.
I20260709 13:05:44.674072 29871 leader_election.cc:403] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:44.632978  1459 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:44.613085  1456 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.603681  1321 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.671357 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.677201 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:44.609089  1425 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:44.681613 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2249) took 1722 ms (client timeout 1855 ms). Trace:
I20260709 13:05:44.691157 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:44.691485 29067 rpcz_store.cc:269] 0709 13:05:42.958711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.208009 (+249298us) service_pool.cc:224] Handling call
0709 13:05:44.681390 (+1473381us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54002abf54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:44.681545 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521921280}
I20260709 13:05:44.693073 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:44.692509 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.696403 29221 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:44.696936 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:44.699106 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:44.653890  1462 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:44.698786 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:44.689285 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.700857 29112 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:44.683568  1112 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.667200  1434 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.711185  1434 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:44.712505 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:44.664276  1458 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:44.679538  1433 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:05:44.704661  1463 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:44.705113  1171 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:44.705819 29905 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:44.708254 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.720965 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:44.661312  1460 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:44.710534 29220 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:44.742974 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.743610 29220 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:44.713631 29142 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:44.677613  1440 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.687228  1459 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.749253 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:44.750130  1459 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.713814  1036 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:44.752187  1036 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:44.752599  1036 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:44.704054 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:44.710534 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:44.720754 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:44.723724 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2459) took 1156 ms. Trace:
I20260709 13:05:44.754329 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.714461 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:44.754382 29215 rpcz_store.cc:276] 0709 13:05:43.567496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.498463 (+930967us) service_pool.cc:224] Handling call
0709 13:05:44.498811 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.499120 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:44.672892 (+173772us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.722965 (+ 50073us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.723358 (+   393us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:44.723419 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.723503 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.723657 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":244068,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":91,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":11835,"thread_start_us":209718,"threads_started":2}
I20260709 13:05:44.757145 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.758104 29215 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:44.759054 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.759617 29220 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:44.702375  1337 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:44.761547 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.713807   948 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:44.767218 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.714422  1195 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.762461 29215 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:44.760872 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.778571 29220 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
W20260709 13:05:44.763689 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 49340.2us
I20260709 13:05:44.723014 29221 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:44.761610 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 57735.2us
I20260709 13:05:44.721818  1458 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:44.767598 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.780952 29875 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:44.720764  1201 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.737253  1237 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.789811 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:44.772280  1460 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:44.794834 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.750176  1440 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.767294   948 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:44.770203  1463 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:44.780042  1217 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:44.811036 29221 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:44.812841 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:44.817188 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:44.818730 29070 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:44.818925 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.790185  1458 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:44.788465  1300 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.820466 29038 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:44.821038 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.825198 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.827178 29220 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:44.828626 29221 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:44.829658 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.830554  1456 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:44.831657  1425 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.844777 29221 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:44.837963  1385 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.840305 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:44.817806  1217 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:44.845793  1217 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:44.845827 29906 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:44.840937  1461 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:44.847012  1461 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.847280 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:44.848626  1456 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:44.849040  1456 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:44.849255  1461 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.849464  1456 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:44.849408  1094 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.850862 29143 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:05:44.851827  1468 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:05:44.852440 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.844075 29220 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
W20260709 13:05:44.853575  1465 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:05:44.852722 29215 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:44.855129 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 758) took 1288 ms. Trace:
I20260709 13:05:44.795421  1460 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.856426 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:44.858052  1463 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:44.859293 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:44.836534  1471 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:05:44.799512  1459 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:44.860334 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:44.855675 29216 rpcz_store.cc:276] 0709 13:05:43.566959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.376050 (+809091us) service_pool.cc:224] Handling call
0709 13:05:44.395126 (+ 19076us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.395601 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.395644 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.396219 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 13:05:44.633440 (+237221us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:44.633501 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.633610 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.854894 (+221284us) raft_consensus.cc:1685] finished
0709 13:05:44.854919 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.855065 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28965,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":10369,"spinlock_wait_cycles":1036032,"thread_start_us":192145,"threads_started":1,"wal-append.queue_time_us":252583}
I20260709 13:05:44.861745 29220 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:44.863447 29903 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:44.870787 29221 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:05:44.877696 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:44.874233 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 588) took 1005 ms. Trace:
I20260709 13:05:44.878079 29904 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:05:44.791738  1337 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:05:44.876957 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:44.817965   948 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:44.881170 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:44.881079 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:44.887658 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.890303 29216 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:44.890816 29143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7517b9e93904801886467474f000b2d"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:05:44.878433 29065 rpcz_store.cc:276] 0709 13:05:43.868542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.607119 (+738577us) service_pool.cc:224] Handling call
0709 13:05:44.607540 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.607808 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.607844 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.613883 (+  6039us) log.cc:844] Serialized 26 byte log entry
0709 13:05:44.715401 (+101518us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:44.873624 (+158223us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.873913 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.873987 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.874026 (+    39us) raft_consensus.cc:1685] finished
0709 13:05:44.874038 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.874171 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":159324,"threads_started":2,"wal-append.queue_time_us":163236}
I20260709 13:05:44.892714 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:44.894819 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:44.860131  1300 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.900695  1448 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:44.900921 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.901317  1448 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:44.902314 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.903096 29216 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:44.905987 29065 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: [6.6-9.8]
I20260709 13:05:44.907066 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.907574 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:44.908536 29065 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:44.908437 29216 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:44.865657  1408 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:44.905750 29113 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:44.887972  1337 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:44.905750 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:44.889041  1293 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:44.888295  1440 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:44.824702   999 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:44.902242  1300 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.891717  1433 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14.
I20260709 13:05:44.890883  1459 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:44.893594  1403 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:44.893688  1430 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:44.902971  1201 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:44.916307 29070 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:44.889914  1434 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.902275  1448 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:44.930343 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:44.928472  1440 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:44.982847 29072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2253) took 2023 ms (client timeout 1938 ms). Trace:
I20260709 13:05:44.983278 29220 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:44.983340 29072 rpcz_store.cc:269] 0709 13:05:42.959803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.323324 (+363521us) service_pool.cc:224] Handling call
0709 13:05:44.982474 (+1659150us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b540012a554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:44.982773 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50035584}
I20260709 13:05:44.915436  1337 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:44.988229 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:44.989238 29072 raft_consensus.cc:2410] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:05:44.989789 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:44.993381 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:44.994077 29070 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:44.968878 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 593) took 1071 ms. Trace:
I20260709 13:05:44.919591  1171 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:44.973147 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 75658.4us
I20260709 13:05:44.997478 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:44.918478  1408 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:44.921893  1293 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:44.979400   999 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:44.919296  1112 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:44.991323 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:44.994671 29067 rpcz_store.cc:276] 0709 13:05:43.896999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.692398 (+795399us) service_pool.cc:224] Handling call
0709 13:05:44.692691 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.692884 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:44.799531 (+106647us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.968443 (+168912us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.968521 (+    78us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:44.968574 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.968657 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.968797 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62388,"prepare.queue_time_us":194778,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":49630,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":1448,"thread_start_us":210399,"threads_started":2,"wal-append.queue_time_us":182689}
W20260709 13:05:45.000842 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet cdb0c0ac78fc4de7a353a995edd44346: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:05:45.004815 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:45.002422  1112 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.013777  1417 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:45.014191  1456 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:45.020463 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:45.021603  1094 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:45.021466  1408 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.022794  1293 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.023834  1460 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:45.022742  1463 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:45.024945 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:45.026693  1235 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:45.027318 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:45.029729  1447 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:45.030694 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2453) took 2344 ms. Trace:
I20260709 13:05:45.074947 29217 rpcz_store.cc:276] 0709 13:05:42.686366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.777201 (+ 90835us) service_pool.cc:224] Handling call
0709 13:05:42.906227 (+129026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.961225 (+ 54998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.030398 (+2069173us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540009f1d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:45.030582 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295850240}
I20260709 13:05:45.036581 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 591) took 1152 ms. Trace:
I20260709 13:05:45.076761 29073 rpcz_store.cc:276] 0709 13:05:43.884204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.986851 (+102647us) service_pool.cc:224] Handling call
0709 13:05:45.036271 (+1049420us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b5400186854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:45.036494 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1157689216}
I20260709 13:05:45.078570 29073 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:45.038050  1434 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:45.039021 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 753) took 1172 ms. Trace:
I20260709 13:05:45.039348  1440 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.000954  1201 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:45.040977 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:45.011029  1459 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.047178 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:44.996641  1171 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:45.043354  1329 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.338s	user 0.008s	sys 0.035s
I20260709 13:05:45.048872  1112 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:45.070243 29220 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:45.059782  1417 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:45.084930 29070 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:45.087167 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:45.146224 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 144897us
I20260709 13:05:45.084973 29902 rpcz_store.cc:276] 0709 13:05:43.866487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.868775 (+  2288us) service_pool.cc:224] Handling call
0709 13:05:43.869228 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.074199 (+204971us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:44.074470 (+   271us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.115036 (+ 40566us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:44.115103 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:44.115149 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.038683 (+923534us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54005a4e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:45.038739 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.038952 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13226,"prepare.queue_time_us":529,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":39,"raft.queue_time_us":18384,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5899,"spinlock_wait_cycles":470842112,"thread_start_us":22226,"threads_started":1,"wal-append.queue_time_us":21026}
I20260709 13:05:45.158807 29222 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:45.159543 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2420) took 1590 ms. Trace:
I20260709 13:05:45.069936  1460 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.159829 29222 rpcz_store.cc:276] 0709 13:05:43.569181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.158305 (+1589124us) service_pool.cc:224] Handling call
0709 13:05:45.158638 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.159291 (+   653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.159488 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:45.143555  1414 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.854s	user 0.010s	sys 0.004s
I20260709 13:05:45.164535 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2422) took 1594 ms. Trace:
I20260709 13:05:45.166003 29220 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:45.168817 29222 rpcz_store.cc:276] 0709 13:05:43.569682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.163539 (+1593857us) service_pool.cc:224] Handling call
0709 13:05:45.163929 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.164147 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.164182 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.164238 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:45.164281 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.164349 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.164470 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.169569 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2421) took 1600 ms. Trace:
I20260709 13:05:45.170162 29220 rpcz_store.cc:276] 0709 13:05:43.569426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.165403 (+1595977us) service_pool.cc:224] Handling call
0709 13:05:45.165756 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.169250 (+  3494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.169500 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.170754 29222 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:45.171553 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2423) took 1601 ms. Trace:
I20260709 13:05:45.171661 29220 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:45.171990 29222 rpcz_store.cc:276] 0709 13:05:43.569962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.170159 (+1600197us) service_pool.cc:224] Handling call
0709 13:05:45.170514 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.171270 (+   756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.171489 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.172519 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2426) took 1590 ms. Trace:
I20260709 13:05:45.084504  1447 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:45.173620 29222 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:45.174407 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2427) took 1579 ms. Trace:
I20260709 13:05:45.173867  1447 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.174829 29222 rpcz_store.cc:276] 0709 13:05:43.594614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.173090 (+1578476us) service_pool.cc:224] Handling call
0709 13:05:45.173410 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.174122 (+   712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.174351 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.175346  1447 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:45.148124  1483 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 13:05:45.184258  1404 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 1.635s	user 0.001s	sys 0.009s
I20260709 13:05:45.185513 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2352) took 3208 ms. Trace:
I20260709 13:05:45.185956 29071 rpcz_store.cc:276] 0709 13:05:41.977314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.551158 (+1573844us) service_pool.cc:224] Handling call
0709 13:05:43.551975 (+   817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.703435 (+151460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.185232 (+1481797us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400146754. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:45.185444 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301670784}
I20260709 13:05:45.186673 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2252) took 2058 ms. Trace:
I20260709 13:05:45.187093 29142 rpcz_store.cc:276] 0709 13:05:43.128763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.128949 (+   186us) service_pool.cc:224] Handling call
0709 13:05:44.524894 (+1395945us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.748208 (+223314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.185945 (+437737us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400041854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:45.186464 (+   519us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400041850. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:45.186609 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3643152384}
I20260709 13:05:45.190054 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2477) took 2052 ms. Trace:
I20260709 13:05:45.190498 29144 rpcz_store.cc:276] 0709 13:05:43.137727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.359721 (+221994us) service_pool.cc:224] Handling call
0709 13:05:43.369453 (+  9732us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.392236 (+ 22783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.392267 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.043788 (+1651521us) spinlock_profiling.cc:243] Waited 8.08 ms on lock 0x7b2c00010da8. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:05:45.044343 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.083515 (+ 39172us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:45.083611 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.083685 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.189703 (+106018us) raft_consensus.cc:1685] finished
0709 13:05:45.189740 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.189987 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6659,"raft.queue_time_us":22917,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":25994,"spinlock_wait_cycles":18592512,"thread_start_us":29545,"threads_started":1,"wal-append.queue_time_us":143654}
I20260709 13:05:45.193271 29144 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:45.194499  1149 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.195283  1337 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:45.196966 29214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2447) took 2675 ms (client timeout 1549 ms). Trace:
W20260709 13:05:45.197484 29214 rpcz_store.cc:269] 0709 13:05:42.521360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.523684 (+  2324us) service_pool.cc:224] Handling call
0709 13:05:45.196679 (+2672995us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54002837d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:45.196872 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279587712}
I20260709 13:05:45.199129  1462 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:45.199277 29214 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:45.199724  1462 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:45.201943 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2430) took 1585 ms. Trace:
I20260709 13:05:45.195287  1149 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:45.151661 29073 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:45.204943  1273 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:45.205359  1273 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:45.206213  1273 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:45.084132  1461 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:45.207883  1461 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:45.209106 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:45.148618  1329 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.443s	user 0.008s	sys 0.036s
I20260709 13:05:45.148329  1171 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:45.202333 29214 rpcz_store.cc:276] 0709 13:05:43.616884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.198625 (+1581741us) service_pool.cc:224] Handling call
0709 13:05:45.199047 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.199870 (+   823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.201883 (+  2013us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.180397 29220 rpcz_store.cc:276] 0709 13:05:43.582653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.171091 (+1588438us) service_pool.cc:224] Handling call
0709 13:05:45.171456 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.172246 (+   790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.172458 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.217235  1049 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.180184  1463 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:45.223987 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:45.149232  1036 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:45.211745  1273 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:45.227514  1460 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:45.231132  1460 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:45.227380  1463 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.225348  1166 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:45.231633  1456 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:45.232111  1166 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:45.233407  1463 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:45.233798 29220 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:45.234473 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2470) took 1292 ms. Trace:
I20260709 13:05:45.234773 29220 rpcz_store.cc:276] 0709 13:05:43.941812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.218385 (+1276573us) service_pool.cc:224] Handling call
0709 13:05:45.233636 (+ 15251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.234205 (+   569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.234416 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.240236 29214 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:45.218881  1314 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.210356  1414 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.238121  1036 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:45.225662  1049 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.238782  1201 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.256480  1049 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:45.257405  1314 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.259146  1171 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:45.259640  1314 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:45.441778  1458 consensus_meta.cc:287] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.576s	user 0.004s	sys 0.007s
I20260709 13:05:45.442396  1458 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:45.489820  1462 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:45.538997  1462 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:45.594110 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.595559 29068 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:45.598968 29144 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:45.681725  1149 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:45.684024  1166 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:45.599581 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.706879 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2436) took 1708 ms. Trace:
I20260709 13:05:45.740017  1094 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:45.754290  1336 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:45.705220 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 481170us
I20260709 13:05:45.873900  1414 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:45.875731  1417 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.708s	user 0.010s	sys 0.000s
I20260709 13:05:46.095336 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 778) took 1389 ms. Trace:
I20260709 13:05:46.095647 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:46.077513  1171 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:46.081071  1450 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:46.082835  1489 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:45.894541  1448 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.002s	sys 0.007s
I20260709 13:05:46.099570 29221 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:46.102499  1336 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.102327 29907 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:46.068003 29220 rpcz_store.cc:276] 0709 13:05:43.998479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.235483 (+1237004us) service_pool.cc:224] Handling call
0709 13:05:45.235769 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.235988 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:45.236666 (+   678us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.705965 (+469299us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.706078 (+   113us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:45.706195 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.706313 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.706584 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":178,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":135295,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":298,"thread_start_us":384937,"threads_started":1,"wal-append.queue_time_us":871938}
W20260709 13:05:46.113503  1448 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.002s	sys 0.008s
I20260709 13:05:46.127151 29222 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:05:46.128455 29219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 760) took 2389 ms (client timeout 2229 ms). Trace:
I20260709 13:05:46.129328  1485 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:05:46.129076 29219 rpcz_store.cc:269] 0709 13:05:43.714193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.066825 (+352632us) service_pool.cc:224] Handling call
0709 13:05:45.893530 (+1826705us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54002f62d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.103950 (+210420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586485632}
I20260709 13:05:46.130308  1485 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:46.128140  1450 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.128566  1336 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:46.133839  1094 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.133967 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2439) took 2085 ms. Trace:
I20260709 13:05:46.135896  1489 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.132936 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2437) took 2114 ms. Trace:
I20260709 13:05:46.100651 29904 rpcz_store.cc:276] 0709 13:05:44.705558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.707209 (+  1651us) service_pool.cc:224] Handling call
0709 13:05:44.877838 (+170629us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.878524 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.094720 (+1216196us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54005bc550. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.095042 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391632128}
I20260709 13:05:46.139966 29219 rpcz_store.cc:276] 0709 13:05:44.047844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.130571 (+2082727us) service_pool.cc:224] Handling call
0709 13:05:46.131149 (+   578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.131654 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.131691 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.131749 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:46.131790 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.131864 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.132640 (+   776us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.141523 29221 rpcz_store.cc:276] 0709 13:05:43.998950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.097672 (+2098722us) service_pool.cc:224] Handling call
0709 13:05:46.098848 (+  1176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.112001 (+ 13153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.113709 (+  1708us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.134198  1490 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.182142  1490 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:46.183002  1112 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.015s	user 0.027s	sys 0.000s
I20260709 13:05:46.184047  1112 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.198004 29067 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:46.199726  1254 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:46.203891 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2239) took 3308 ms. Trace:
I20260709 13:05:46.204859 29141 rpcz_store.cc:276] 0709 13:05:42.814145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.966202 (+152057us) service_pool.cc:224] Handling call
0709 13:05:42.966668 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:43.009015 (+ 42347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.009045 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:43.009106 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:43.009154 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.112805 (+3103651us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b54002fab54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:46.112961 (+   156us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.122147 (+  9186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96892672}
W20260709 13:05:46.207782 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 1.20977e+06us
I20260709 13:05:46.211606 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:46.214358  1218 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:05:46.215767  1390 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:46.216614  1390 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:46.218515 29219 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:05:46.218370  1390 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.219372 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2440) took 2165 ms. Trace:
I20260709 13:05:46.219784 29219 rpcz_store.cc:276] 0709 13:05:44.053717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.158982 (+2105265us) service_pool.cc:224] Handling call
0709 13:05:46.194613 (+ 35631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.219061 (+ 24448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.219284 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.220675  1218 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:46.221231  1218 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:46.221683 29219 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:46.222836 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2442) took 2121 ms. Trace:
I20260709 13:05:46.223259 29219 rpcz_store.cc:276] 0709 13:05:44.101000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.220819 (+2119819us) service_pool.cc:224] Handling call
0709 13:05:46.221317 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.222410 (+  1093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.222732 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:46.224221   948 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.321s	user 0.002s	sys 0.029s
W20260709 13:05:46.227018 29218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2474) took 2092 ms (client timeout 2112 ms). Trace:
I20260709 13:05:46.232422 29037 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:46.156672  1485 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.233642  1450 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:46.234998  1217 consensus_meta.cc:287] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 1.388s	user 0.006s	sys 0.022s
I20260709 13:05:46.236711 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:46.126410 29220 raft_consensus.cc:2955] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 5 to 8, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:46.229175 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 736) took 1721 ms. Trace:
I20260709 13:05:46.240979 29073 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:46.222955  1218 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:46.238533 29218 rpcz_store.cc:269] 0709 13:05:44.134014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.223062 (+ 89048us) service_pool.cc:224] Handling call
0709 13:05:46.226173 (+2003111us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b54003557d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.226680 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58209152}
I20260709 13:05:46.240475 29140 rpcz_store.cc:276] 0709 13:05:44.507576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.181099 (+1673523us) service_pool.cc:224] Handling call
0709 13:05:46.217468 (+ 36369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.218036 (+   568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.218115 (+    79us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.218191 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:46.218220 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.218291 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.228925 (+ 10634us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:46.252628 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:46.257556 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 80497.6us
I20260709 13:05:46.263463 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:46.264947 29145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 728) took 1833 ms (client timeout 2229 ms). Trace:
W20260709 13:05:46.265453 29145 rpcz_store.cc:269] 0709 13:05:44.431887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.432379 (+   492us) service_pool.cc:224] Handling call
0709 13:05:46.223643 (+1791264us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b5800003098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:05:46.264819 (+ 41176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5323520}
I20260709 13:05:46.269387  1112 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:46.250914   948 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.273794 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 735) took 1782 ms. Trace:
I20260709 13:05:46.274363 29144 rpcz_store.cc:276] 0709 13:05:44.492500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.194010 (+701510us) service_pool.cc:224] Handling call
0709 13:05:45.194272 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.111136 (+916864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.273374 (+162238us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54003b38d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:46.273694 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929062272}
I20260709 13:05:46.277245 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2260) took 1678 ms. Trace:
I20260709 13:05:46.277817 29144 rpcz_store.cc:276] 0709 13:05:44.598952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.275700 (+1676748us) service_pool.cc:224] Handling call
0709 13:05:46.276246 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.276809 (+   563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.276843 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.276929 (+    86us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:46.276970 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.277030 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.277155 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.279418 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2351) took 4376 ms. Trace:
I20260709 13:05:46.279804 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2261) took 1670 ms. Trace:
I20260709 13:05:46.280406  1112 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:46.279871 29068 rpcz_store.cc:276] 0709 13:05:41.902908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.550039 (+1647131us) service_pool.cc:224] Handling call
0709 13:05:44.984446 (+1434407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.111235 (+1126789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.278596 (+167361us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540012a554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:46.279158 (+   562us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b540012a550. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.279313 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4701525120}
W20260709 13:05:46.282095 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 50251us
I20260709 13:05:46.284044  1112 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:46.284399  1467 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:46.285426  1467 consensus_queue.cc:1243] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:05:46.255149  1217 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.287932  1341 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:46.289865  1341 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:46.290443  1341 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:46.289316  1099 raft_consensus.cc:2955] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Committing config change with OpId 9.8: config changed from index 5 to 8, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:46.292266 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 766) took 2189 ms. Trace:
I20260709 13:05:46.287870  1461 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:46.297716  1249 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:46.297317 29219 rpcz_store.cc:276] 0709 13:05:44.103161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.250828 (+2147667us) service_pool.cc:224] Handling call
0709 13:05:46.291213 (+ 40385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.291755 (+   542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.291791 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.291861 (+    70us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:46.291903 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.291989 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.292165 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.300846 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 768) took 2197 ms. Trace:
W20260709 13:05:46.297680 29874 connection.cc:256] RPC call timeout handler was delayed by -1.1533556500007762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
I20260709 13:05:46.301357 29219 rpcz_store.cc:276] 0709 13:05:44.103665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.299435 (+2195770us) service_pool.cc:224] Handling call
0709 13:05:46.299971 (+   536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.300439 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.300484 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.300554 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:46.300600 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.300667 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.300789 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36992}
I20260709 13:05:46.302889 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:46.304306  1425 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:46.308995 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:46.310262 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 764) took 2369 ms. Trace:
I20260709 13:05:46.310593  1430 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:46.292961 29218 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:46.291864  1341 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.280514 29144 rpcz_store.cc:276] 0709 13:05:44.609118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.278837 (+1669719us) service_pool.cc:224] Handling call
0709 13:05:46.279243 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.279528 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.279542 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.279599 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:46.279619 (+    20us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.279657 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.279723 (+    66us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.295251 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:46.293085  1467 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:05:46.296752 29073 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:05:46.310796 29214 rpcz_store.cc:276] 0709 13:05:43.940639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.218008 (+1277369us) service_pool.cc:224] Handling call
0709 13:05:45.225223 (+  7215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.245613 (+ 20390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.309753 (+1064140us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b540033aed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:46.310052 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34001792}
I20260709 13:05:46.314168  1341 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:46.314930  1388 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 3.526s	user 0.015s	sys 0.027s
I20260709 13:05:46.316686 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 720) took 4266 ms. Trace:
I20260709 13:05:46.317260 29139 rpcz_store.cc:276] 0709 13:05:42.050499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.050617 (+   118us) service_pool.cc:224] Handling call
0709 13:05:42.050881 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.360790 (+1309909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.315880 (+2955090us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400290254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:46.316414 (+   534us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3010511232}
I20260709 13:05:46.319509  1467 raft_consensus.cc:2764] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:05:46.320060 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:46.320297 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 767) took 2216 ms. Trace:
I20260709 13:05:46.317847 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:46.320873 29218 rpcz_store.cc:276] 0709 13:05:44.103411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.292039 (+2188628us) service_pool.cc:224] Handling call
0709 13:05:46.292669 (+   630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.319995 (+ 27326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.320227 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.325330 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 771) took 1737 ms. Trace:
I20260709 13:05:46.325798 29214 rpcz_store.cc:276] 0709 13:05:44.588107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.324010 (+1735903us) service_pool.cc:224] Handling call
0709 13:05:46.324705 (+   695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.325031 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.325057 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.325098 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:46.325125 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.325177 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.325275 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.328667 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2488) took 1564 ms. Trace:
I20260709 13:05:46.330704 29214 rpcz_store.cc:276] 0709 13:05:44.764693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.326860 (+1562167us) service_pool.cc:224] Handling call
0709 13:05:46.327702 (+   842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.328300 (+   598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.328331 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.328378 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:46.328410 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.328473 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.328600 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.332612 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 772) took 1740 ms. Trace:
I20260709 13:05:46.332703 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 737) took 1767 ms. Trace:
I20260709 13:05:46.333527 29218 rpcz_store.cc:276] 0709 13:05:44.588395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.323674 (+1735279us) service_pool.cc:224] Handling call
0709 13:05:46.326184 (+  2510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.326504 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.326527 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.326565 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:46.328888 (+  2323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.328964 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.329067 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.334136 29145 rpcz_store.cc:276] 0709 13:05:44.562434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.266893 (+1704459us) service_pool.cc:224] Handling call
0709 13:05:46.267366 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.312432 (+ 45066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.312463 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.322598 (+ 10135us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:46.329046 (+  6448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.329150 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.329298 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.311204  1430 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:46.274734   948 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:46.353839 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2490) took 2411 ms. Trace:
I20260709 13:05:46.354350 29148 rpcz_store.cc:276] 0709 13:05:43.942805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.040297 (+ 97492us) service_pool.cc:224] Handling call
0709 13:05:44.048851 (+  8554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.049327 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:44.049676 (+   349us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.220753 (+2171077us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b18003df960. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a 00007f7836b17a2a
0709 13:05:46.339586 (+118833us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.353177 (+ 13591us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:46.353284 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.353423 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.353667 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":159,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":3092,"raft.queue_time_us":5010,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":19238,"spinlock_wait_cycles":7209344,"thread_start_us":90573,"threads_started":1}
I20260709 13:05:46.356102 29143 raft_consensus.cc:606] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:46.357440 29143 raft_consensus.cc:614] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:46.341934  1217 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:46.360378 29215 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:46.361805 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2253) took 1849 ms. Trace:
I20260709 13:05:46.366923 29141 rpcz_store.cc:276] 0709 13:05:44.512507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.237200 (+1724693us) service_pool.cc:224] Handling call
0709 13:05:46.256115 (+ 18915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.357972 (+101857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.358010 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.361410 (+  3400us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:46.361472 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.361559 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.361731 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.369968 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2489) took 1510 ms. Trace:
W20260709 13:05:46.364777 29215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2477) took 1837 ms (client timeout 2222 ms). Trace:
I20260709 13:05:46.371802 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 580) took 4742 ms. Trace:
I20260709 13:05:46.372515  1191 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:46.370615  1149 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:46.371979 29214 rpcz_store.cc:276] 0709 13:05:44.861223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.333868 (+1472645us) service_pool.cc:224] Handling call
0709 13:05:46.358054 (+ 24186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.369419 (+ 11365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.369466 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.369540 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:46.369590 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.369681 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.369881 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.384061 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 741) took 1725 ms. Trace:
I20260709 13:05:46.383785  1149 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:46.371451 29215 rpcz_store.cc:269] 0709 13:05:44.527089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.858635 (+331546us) service_pool.cc:224] Handling call
0709 13:05:46.319705 (+1461070us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:05:46.364696 (+ 44991us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4269824}
I20260709 13:05:46.384509 29144 rpcz_store.cc:276] 0709 13:05:44.658855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.330311 (+1671456us) service_pool.cc:224] Handling call
0709 13:05:46.357108 (+ 26797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.383567 (+ 26459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.383609 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.383710 (+   101us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:46.383756 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.383828 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.383999 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:46.385898  1350 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.022s	user 0.041s	sys 0.087s
W20260709 13:05:46.386473  1350 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.042s	user 0.041s	sys 0.087s
I20260709 13:05:46.395700 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:46.372294 29064 rpcz_store.cc:276] 0709 13:05:41.629343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.006308 (+376965us) service_pool.cc:224] Handling call
0709 13:05:42.548015 (+541707us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:42.548492 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:42.548545 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:42.549245 (+   700us) log.cc:844] Serialized 52 byte log entry
0709 13:05:42.825626 (+276381us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:43.035709 (+210083us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.585038 (+549329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.585132 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:43.585197 (+    65us) raft_consensus.cc:1685] finished
0709 13:05:43.585217 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.371414 (+2786197us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b54002cbad0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.371616 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":464401,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":547,"raft.queue_time_us":291472,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":189,"spinlock_wait_cycles":1244728192,"thread_start_us":973922,"threads_started":3,"wal-append.queue_time_us":451447,"wal-append.run_cpu_time_us":4091,"wal-append.run_wall_time_us":1539634}
W20260709 13:05:46.410552 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 92853us
I20260709 13:05:46.412075 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:46.414479 29185 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:46.416313 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 107560us
I20260709 13:05:46.409742  1219 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:46.420670  1191 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:46.426034 29139 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:05:46.427240 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2441) took 2327 ms. Trace:
I20260709 13:05:46.429298  1055 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:46.423270  1485 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:46.434121 29907 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 702) took 1637 ms (client timeout 2008 ms). Trace:
W20260709 13:05:46.434728 29907 rpcz_store.cc:269] 0709 13:05:44.796691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.797839 (+  1148us) service_pool.cc:224] Handling call
0709 13:05:46.433729 (+1635890us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54005e92d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.433984 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2948091264}
I20260709 13:05:46.435278  1425 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:46.438912 29147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2493) took 2436 ms (client timeout 2139 ms). Trace:
W20260709 13:05:46.439456 29147 rpcz_store.cc:269] 0709 13:05:44.002712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.038637 (+ 35925us) service_pool.cc:224] Handling call
0709 13:05:46.438433 (+2399796us) spinlock_profiling.cc:243] Waited 92.7 ms on lock 0x7b5400041354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.438663 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213171584}
I20260709 13:05:46.441876  1458 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:46.367583  1430 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.450681 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 732) took 2007 ms. Trace:
I20260709 13:05:46.454535 29142 rpcz_store.cc:276] 0709 13:05:44.442722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.188289 (+745567us) service_pool.cc:224] Handling call
0709 13:05:46.448097 (+1259808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.448582 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.448610 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.448698 (+    88us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:46.448738 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.448812 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.450308 (+  1496us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400041350. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.450493 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2896168320}
W20260709 13:05:46.468873 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 127675us
I20260709 13:05:46.470098 29037 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:46.472414 29874 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:46.473455  1219 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:46.474123  1219 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:46.474802  1219 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:46.481909  1421 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:46.484213  1408 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:46.484473 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:46.490978  1490 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:46.493858  1321 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:46.503542 29142 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:46.507272  1408 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:46.508720  1408 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.507774 29147 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:05:46.528244 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 99754.5us
I20260709 13:05:46.532428  1217 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:46.533108  1217 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:46.533609  1217 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:46.536319 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2479) took 2008 ms. Trace:
W20260709 13:05:46.538156 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 123903us
I20260709 13:05:46.537014 29219 rpcz_store.cc:276] 0709 13:05:44.527602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.302867 (+1775265us) service_pool.cc:224] Handling call
0709 13:05:46.303526 (+   659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:46.304694 (+  1168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.304721 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:46.305478 (+   757us) log.cc:844] Serialized 26 byte log entry
0709 13:05:46.397296 (+ 91818us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:46.417936 (+ 20640us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.423966 (+  6030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.424087 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.486429 (+ 62342us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b180035c7a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:46.535708 (+ 49279us) raft_consensus.cc:1685] finished
0709 13:05:46.535943 (+   235us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.536196 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5378,"prepare.queue_time_us":20432,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":1400,"raft.queue_time_us":5948,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":4092,"spinlock_wait_cycles":52636032,"thread_start_us":91248,"threads_started":1,"wal-append.queue_time_us":119211}
W20260709 13:05:46.547158 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.503108  1491 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:46.566951   884 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:46.567456   884 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:46.568179  1493 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.336s	user 0.002s	sys 0.000s
I20260709 13:05:46.568912   884 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:46.569962  1498 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.368s	user 0.001s	sys 0.001s
I20260709 13:05:46.571038   884 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:46.566250 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59184.7us
I20260709 13:05:46.556798  1094 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:46.560917 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:46.563064  1489 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:46.487792 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:46.539513 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:46.576463 29213 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:46.510321  1408 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:46.581673 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2268) took 1900 ms. Trace:
I20260709 13:05:46.582021  1273 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:46.582530  1291 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:46.582715  1273 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.582177 29139 rpcz_store.cc:276] 0709 13:05:44.681129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.318557 (+1637428us) service_pool.cc:224] Handling call
0709 13:05:46.319132 (+   575us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:46.426505 (+107373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.581278 (+154773us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54003a4dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:46.581539 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244773120}
I20260709 13:05:46.584168  1417 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:46.584766  1482 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:46.585228  1482 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:46.586514  1434 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:46.587060  1434 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:46.588706  1434 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.590277  1434 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:46.586429  1482 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.593019  1482 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:46.594544 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39218 (request call id 20) took 2152 ms. Trace:
I20260709 13:05:46.594986 29143 rpcz_store.cc:276] 0709 13:05:44.441777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.890676 (+448899us) service_pool.cc:224] Handling call
0709 13:05:46.594225 (+1703549us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540029f254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b07853 00007f783bad560a 00007f783505b7d4 00007f783505b762 00007f783505b691 00007f783505b60c 00007f783505a8c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.594388 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3368104320}
I20260709 13:05:46.585726  1273 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:46.606534 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2269) took 1584 ms (client timeout 2008 ms). Trace:
I20260709 13:05:46.606761 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2419) took 3037 ms. Trace:
W20260709 13:05:46.607079 29067 rpcz_store.cc:269] 0709 13:05:45.022115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.024679 (+  2564us) service_pool.cc:224] Handling call
0709 13:05:46.606246 (+1581567us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001d8654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.606446 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2590360960}
I20260709 13:05:46.607215 29217 rpcz_store.cc:276] 0709 13:05:43.568898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.094731 (+1525833us) service_pool.cc:224] Handling call
0709 13:05:46.226717 (+1131986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.227394 (+   677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.227437 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.227510 (+    73us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:46.227539 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.227616 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.606441 (+378825us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003557d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.606587 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2601933184}
I20260709 13:05:46.609125 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:46.610482  1413 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:46.611132  1413 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:46.611780 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 124279us
I20260709 13:05:46.611776  1336 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.613126 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:46.613422  1336 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.616925  1336 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:46.621344 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.632810  1433 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:46.635903  1445 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.636821  1445 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.625427  1112 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:46.625478 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:05:46.640964 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2428) took 3040 ms. Trace:
I20260709 13:05:46.641469 29222 rpcz_store.cc:276] 0709 13:05:43.600774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.175786 (+1575012us) service_pool.cc:224] Handling call
0709 13:05:46.094290 (+918504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.127647 (+ 33357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.639552 (+511905us) spinlock_profiling.cc:243] Waited 32.3 ms on lock 0x7b54002f62d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:46.640653 (+  1101us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b54002f62d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.640890 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1814639616}
I20260709 13:05:46.625102  1413 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:46.658722 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 85810us
I20260709 13:05:46.428341 29221 rpcz_store.cc:276] 0709 13:05:44.100434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.299546 (+2199112us) service_pool.cc:224] Handling call
0709 13:05:46.304679 (+  5133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.305361 (+   682us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:46.359198 (+ 53837us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.425751 (+ 66553us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.426276 (+   525us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.426744 (+   468us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.426796 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:46.426835 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.427004 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.427170 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":53949,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":131,"raft.queue_time_us":3441,"raft.run_cpu_time_us":422,"raft.run_wall_time_us":1737,"spinlock_wait_cycles":24264960,"thread_start_us":116412,"threads_started":2}
I20260709 13:05:46.669247  1482 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:46.670140  1482 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:46.671536  1482 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.673080 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:46.675052 29038 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:46.675631 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.675848  1397 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.676416  1291 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:46.677448  1496 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.474s	user 0.004s	sys 0.001s
I20260709 13:05:46.619848  1149 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:46.677834  1433 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.678623  1273 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:46.679379  1434 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:46.680755  1414 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:46.681056  1046 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:46.681412  1112 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:46.682863  1201 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:46.683200 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:46.681476 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 142191us
I20260709 13:05:46.683255 29909 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:46.718180 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:46.722741 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.732160  1433 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.733421 29070 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:05:46.733795 29070 pending_rounds.cc:85] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 10
I20260709 13:05:46.734052 29070 pending_rounds.cc:107] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:05:46.734328 29070 raft_consensus.cc:2889] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:46.734862  1433 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:46.705783  1273 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:46.712664  1414 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.731573 29911 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
W20260709 13:05:46.742468 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.711012  1291 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:46.740343  1390 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.011s	sys 0.007s
I20260709 13:05:46.751754 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.739907  1414 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:46.753708 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.759359 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.719386  1397 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.762108 29904 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:46.763473  1397 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:46.757828 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:46.765213 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:46.766889 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:46.767961  1451 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.768088 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.768689  1451 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.769135  1446 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:46.769625  1198 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:46.770077 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:46.771026  1451 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:46.771409  1291 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:46.756551  1390 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.011s	sys 0.007s
I20260709 13:05:46.772604 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.772959 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.773787 29907 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:46.758709  1219 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.775358  1237 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:46.760123 29903 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:46.756470 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:46.777441 29067 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:46.738109  1149 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:46.784162 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.785362  1149 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:46.786034 29213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2429) took 3177 ms (client timeout 1753 ms). Trace:
I20260709 13:05:46.787545 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:46.786957 29213 rpcz_store.cc:269] 0709 13:05:43.608584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.774724 (+166140us) service_pool.cc:224] Handling call
0709 13:05:46.785704 (+3010980us) spinlock_profiling.cc:243] Waited 2.68 s on lock 0x7b54000bd954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:46.785933 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6170282240}
I20260709 13:05:46.791587 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:46.794162 29904 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:46.794760 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.761243  1046 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:05:46.799345 29908 raft_consensus.cc:2410] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:05:46.781379  1217 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:46.813721 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 100188us
I20260709 13:05:46.788940  1397 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:05:46.809811 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:46.816629 29073 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:46.788039  1099 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:05:46.814819  1046 consensus_queue.cc:1243] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:46.830715 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:46.833611 29903 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:46.812757  1273 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.837882  1273 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:46.840679 29909 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:46.841931 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:46.842662 29909 raft_consensus.cc:2393] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:46.842888 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 200256us
W20260709 13:05:46.843629 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.829276  1423 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:05:46.843501 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:46.844609  1423 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:46.845101  1423 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:46.845497  1237 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:46.846318 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:46.846495  1237 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:46.846920  1237 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:46.846978 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:46.847580 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.848301  1237 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.849233 29146 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:46.848687 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:46.849709  1237 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:46.850152 29147 raft_consensus.cc:2393] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:46.850380 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:46.846083  1423 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.852651 29904 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:46.851807 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:46.854135 29142 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:05:46.848327 29902 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:46.855396 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:46.835397  1219 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:46.832870  1413 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:46.866626 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.869127 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:46.871416 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 243158us
I20260709 13:05:46.876263  1209 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:46.877694  1209 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:46.879209 29908 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:46.870972 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:46.855211  1201 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:46.861361 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:46.884285 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2255) took 2320 ms. Trace:
I20260709 13:05:46.884543 29905 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:46.883206 29910 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:46.879828  1209 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:46.888203 29139 raft_consensus.cc:2410] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:05:46.890195 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:46.887751  1209 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:46.896514 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:46.895661  1336 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:46.900353 29147 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:05:46.904362 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 79948.7us
I20260709 13:05:46.887517  1219 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:05:46.906481  1209 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.906759  1201 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.907219  1337 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:46.908124  1209 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:46.910531 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2438) took 2895 ms. Trace:
I20260709 13:05:46.910983 29220 rpcz_store.cc:276] 0709 13:05:44.008424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.124834 (+2116410us) service_pool.cc:224] Handling call
0709 13:05:46.125396 (+   562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.125934 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:46.404749 (+278815us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.794185 (+389436us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.878420 (+ 84235us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b180014f8c0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f783a5053c7 00007f7836b179a7 00007f7836b270bf 00007f7836b27038
0709 13:05:46.886826 (+  8406us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.892509 (+  5683us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.892559 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:46.892601 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.892756 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.903360 (+ 10604us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69830,"prepare.queue_time_us":103843,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":106,"raft.queue_time_us":23860,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":5366,"spinlock_wait_cycles":5194112,"thread_start_us":317861,"threads_started":1,"wal-append.queue_time_us":453583}
I20260709 13:05:46.917304  1445 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:46.917990  1445 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.917932  1046 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:46.918769  1445 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:46.920608  1336 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:46.921916  1336 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:46.922432  1046 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.923135  1336 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:46.901934  1413 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.923480  1462 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:46.924870  1517 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:05:46.920675  1235 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:46.926380  1218 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:46.923933  1094 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:46.923034  1046 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.926856  1423 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:46.929996  1445 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:46.898806  1451 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:47.015873  1414 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.884832 29140 rpcz_store.cc:276] 0709 13:05:44.560345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.255289 (+1694944us) service_pool.cc:224] Handling call
0709 13:05:46.323903 (+ 68614us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:46.340458 (+ 16555us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:46.403232 (+ 62774us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.689658 (+286426us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:46.721017 (+ 31359us) log.cc:844] Serialized 52 byte log entry
0709 13:05:46.721206 (+   189us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:46.738903 (+ 17697us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.740727 (+  1824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.740847 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.880597 (+139750us) raft_consensus.cc:1685] finished
0709 13:05:46.880630 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.880839 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38417,"prepare.queue_time_us":129712,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":12798,"raft.run_cpu_time_us":591,"raft.run_wall_time_us":24180,"thread_start_us":322510,"threads_started":2,"wal-append.queue_time_us":346281}
I20260709 13:05:46.950362  1235 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:46.956486 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.962417  1413 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:47.027158  1235 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.963488  1149 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:46.965221 29113 leader_election.cc:336] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.753s (SENT)
I20260709 13:05:46.966930 29112 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:46.967850   884 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:46.968214 29139 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:46.968542  1140 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:47.029312  1451 consensus_queue.cc:1059] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:05:47.033991  1140 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:47.034121  1451 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:47.031723 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:46.972525  1458 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:46.973726  1198 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:47.039542  1198 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.034515  1140 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:47.016867 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:47.041529 29148 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:47.042783  1198 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:46.976218 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.980352 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:46.979493  1491 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:05:47.045143 29903 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:47.047123  1205 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:47.047782  1205 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:47.048715  1205 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.049844  1408 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:46.974613  1094 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:47.051569  1094 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.028056 29186 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:47.053046  1094 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:47.054129 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:46.986987 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:47.034161 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:47.056586 29908 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:47.002651 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.057330  1423 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:47.058014 29213 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:47.005143 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:46.978715 29906 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:05:47.007460 29222 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:47.009011  1099 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:47.014882 29147 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:47.015785 29146 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:05:47.016336 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:47.016716 29141 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:05:47.017624  1195 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:47.018419 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.019374  1112 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:47.016446 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:47.021934  1046 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:47.023761  1217 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:47.024959 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 132869us
I20260709 13:05:47.034720  1451 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:47.036015  1458 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:47.035943 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 70922.6us
I20260709 13:05:47.032462   884 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:46.970887 29907 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:05:47.036912  1237 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:47.037338  1462 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:05:47.032081 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 65364.9us
I20260709 13:05:47.034535 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:47.031380  1235 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:47.041003  1140 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:47.050249  1205 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:46.983072 29142 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:47.005575 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.065429 29905 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:47.065655 29219 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:47.095431 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.114465 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47825.1us
I20260709 13:05:47.148741 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:47.073282 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:47.065873 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 161530us
W20260709 13:05:47.144084 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 65467.6us
I20260709 13:05:47.073652  1408 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:47.057837  1423 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:47.151381 29186 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:47.152056 29909 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:47.155246 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:47.152025 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.068037  1403 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:47.062882  1099 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:47.065215  1195 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:47.117964  1198 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:47.047173  1414 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:47.160706  1343 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:05:47.164001  1446 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:47.161623 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.162165  1120 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.147511  1336 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.140029  1237 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:47.073796  1452 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:47.070812  1413 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:47.157366  1423 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:47.175068 29073 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:47.179234  1337 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:47.179497 29222 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:05:47.196095 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.179837 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.180052 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:47.180179 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.180760  1120 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.181759  1336 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.174348 29144 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:47.177462 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:47.187882 29220 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:47.159873  1503 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:47.159888  1403 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:47.184994 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:47.198987 29217 raft_consensus.cc:2410] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:47.202391  1337 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:47.204737 29909 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:47.184284  1446 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.159039  1112 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.172791  1149 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:05:47.203157  1503 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.231199 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.228003 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.240518 29072 raft_consensus.cc:2368] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11.
I20260709 13:05:47.239552  1503 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:47.241140 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:47.241379 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:47.242151 29072 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:47.227627 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:47.237954 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.209393 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.217870  1337 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.246716 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.247906 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.249018 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.249383  1112 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:47.225265 29067 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:05:47.249661  1149 consensus_queue.cc:1243] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:47.225754 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.250084 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.250511 29069 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:47.210816  1451 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.250676 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.251183 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.251289 29067 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:47.251538  1451 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:47.212085 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.212410 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:47.259212 29070 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:47.212668  1364 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:47.264190  1364 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.264683  1364 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:47.194363  1413 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:47.208515 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:47.245517 29071 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:47.211230 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 152056us
I20260709 13:05:47.221007 29064 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:47.252127 29073 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:47.252853 29069 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:47.208179 29905 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:47.254568 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:47.255987  1046 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.257332 29143 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:47.256601   884 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.284283  1140 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.242466 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:47.276861 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 116320us
I20260709 13:05:47.285714 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:47.277732 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 225831us
I20260709 13:05:47.280359 29070 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:47.284346  1046 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.218736 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.286293 29220 raft_consensus.cc:2376] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11. Re-sending same reply.
W20260709 13:05:47.286836 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 78955.9us
I20260709 13:05:47.289880 29068 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:47.292801   884 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:47.293205 29069 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:47.294080 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:47.290721  1205 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:47.295233 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.295374  1205 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:47.296026  1195 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.296468  1205 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.297138  1195 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:47.213605  1446 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:47.301458 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.289480 29072 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:47.302961 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:47.304703 29073 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:05:47.317229 29070 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:47.330741 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:47.291644  1237 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.334440  1237 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:47.336066 29068 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:05:47.337049  1140 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:47.338189 29069 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:47.338737  1458 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.340413  1458 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:47.340899  1403 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.341933  1403 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:47.342609 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.343102 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.343274 29216 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:47.344254 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.344353 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.345177 29214 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:47.345664 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.345954 29216 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:05:47.346904 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.347641 29213 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:47.348587 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.350145  1099 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.350674  1046 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:47.351296  1099 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:47.352779 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.356222 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:47.356302 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:47.357061 29145 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:47.357857 29039 leader_election.cc:400] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:05:47.358224 29039 leader_election.cc:403] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:05:47.358103 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.360790 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.361682 29218 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:05:47.361971 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:47.364143 29067 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:47.364641 29146 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:05:47.326797  1423 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.367408 29071 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
W20260709 13:05:47.363827 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.368230  1413 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.369881  1413 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:47.370878 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:47.372233 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.372076  1491 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:05:47.373644  1491 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:47.374413  1140 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:47.375381 29186 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:47.375994 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.376161  1530 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:47.357722 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 57323.8us
I20260709 13:05:47.376693  1451 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:47.377278 29142 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:47.364938 28968 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.374487  1491 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:05:47.389196 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 103076us
I20260709 13:05:47.399617 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:47.406420 29218 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:47.408815 29065 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:47.416507  1536 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.417465  1536 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.419240 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:47.384994  1491 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:47.351684  1046 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:05:47.397398  1530 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:05:47.425675 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:47.428134 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.433220 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:47.405386  1217 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:05:47.438395 29222 pending_rounds.cc:85] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 5
I20260709 13:05:47.438787 29222 pending_rounds.cc:107] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 13.6
I20260709 13:05:47.439023 29222 raft_consensus.cc:2889] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:47.447999 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.449497 29064 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
W20260709 13:05:47.452904 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 124003us
I20260709 13:05:47.459674 29065 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:47.428241  1209 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:05:47.465870 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:47.469285 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 57423.6us
I20260709 13:05:47.470346 29037 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:47.454726  1343 consensus_queue.cc:1243] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:05:47.461966  1198 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:47.484699 29145 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:47.484258 29218 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:47.486449 29064 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:05:47.486963 29064 pending_rounds.cc:85] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 6
I20260709 13:05:47.505330 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:47.514197 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:47.515246 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 116443us
I20260709 13:05:47.516391 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:47.517649 29141 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:05:47.519547 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.523171 29146 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:47.456704  1503 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.483857  1209 consensus_queue.cc:1243] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:05:47.531080 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.514256 28968 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 7 to 8, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 8 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:47.509020  1112 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:47.540953 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:47.542009 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 78238.1us
I20260709 13:05:47.540123 29064 pending_rounds.cc:107] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 9.7
W20260709 13:05:47.542927 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.538702 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.529538  1217 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:47.546836 29064 raft_consensus.cc:2889] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:47.550120 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:47.550256 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 80024.6us
I20260709 13:05:47.556469 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:47.576654 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 108026us
I20260709 13:05:47.577801 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:47.588541 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:47.590183 28969 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 240) took 20074 ms (client timeout 15000 ms). Trace:
I20260709 13:05:47.594744 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:47.595894 29064 raft_consensus.cc:2368] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15.
I20260709 13:05:47.598003 29070 raft_consensus.cc:1217] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 7.4->[9.5-14.7]   Dedup: 9.5->[12.6-14.7]
I20260709 13:05:47.598774 29070 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: [9.5-14.7]
I20260709 13:05:47.536738  1198 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.585989  1534 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:47.613665 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.620721 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:47.617904  1336 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:47.623334 29070 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 13:05:47.569535  1491 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.619846 28968 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:05:47.633962 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 117676us
I20260709 13:05:47.605232  1452 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:47.609918  1537 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:47.640046 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.640213 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 88944us
I20260709 13:05:47.610736  1364 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.665431 29219 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:05:47.667128 29111 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.188s (SENT)
I20260709 13:05:47.662725 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:47.669377 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.640807  1112 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:47.683792 28968 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.641732  1536 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:47.668397 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:47.691852 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.692378 29215 raft_consensus.cc:2410] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:05:47.653179  1120 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:47.693804 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.694437 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.697402  1364 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:47.708243  1446 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.709043  1502 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:47.709517  1502 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:47.710933  1446 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:47.710469  1502 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.712419  1423 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:47.687186  1046 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:47.690233  1343 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:47.715149 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.708334 28968 catalog_manager.cc:5697] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 6 to 8, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 8 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:05:47.722476 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 54212.2us
I20260709 13:05:47.692579  1503 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:47.702843  1530 raft_consensus.cc:2764] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:05:47.717785 29065 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:47.702569  1531 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:47.725833 29188 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:47.729703 29070 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:05:47.719732  1491 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:47.735633 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 122333us
I20260709 13:05:47.723104  1343 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.747951 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:47.752625  1446 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:47.753399  1446 consensus_queue.cc:1243] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:05:47.752794  1422 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:47.744266  1535 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:05:47.757038  1530 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:47.732942  1195 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:05:47.758872 29070 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:05:47.759409 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:47.760637 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.764667 29215 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:47.744699  1209 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:47.744009 28968 catalog_manager.cc:5697] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.747200  1343 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:47.734923  1545 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:47.723809  1046 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.751645  1502 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:47.590615 28969 rpcz_store.cc:269] 0709 13:05:27.380461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.380990 (+   529us) service_pool.cc:224] Handling call
0709 13:05:28.650942 (+1269952us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b5400018954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a77090 00007f7836b15f3a 00007f783c8cc1cc 00007f783c9ca37e 00007f783ae8b2a4 00007f783ae8b232 00007f783ae8b161 00007f783ae8b0dc 00007f783ae8a392 00007f7834e4848c 00007f7834e478c6
0709 13:05:47.444336 (+18793394us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:39.692413 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:39.693174 (+   761us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:39.693227 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:05:39.693248 (+    21us) tablet.cc:662] Decoding operations
0709 13:05:39.704348 (+ 11100us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:39.704462 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:39.704489 (+    27us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:05:39.727066 (+ 22577us) tablet.cc:701] Row locks acquired
0709 13:05:39.784609 (+ 57543us) write_op.cc:260] PREPARE: finished
0709 13:05:39.809750 (+ 25141us) write_op.cc:270] Start()
0709 13:05:39.837645 (+ 27895us) write_op.cc:276] Timestamp: P: 1783602339804011 usec, L: 0
0709 13:05:39.858600 (+ 20955us) op_driver.cc:348] REPLICATION: starting
0709 13:05:39.867269 (+  8669us) log.cc:844] Serialized 10586 byte log entry
0709 13:05:40.900357 (+1033088us) op_driver.cc:464] REPLICATION: finished
0709 13:05:41.319813 (+419456us) write_op.cc:301] APPLY: starting
0709 13:05:41.319955 (+   142us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:42.509667 (+1189712us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:42.509789 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:44.453121 (+1943332us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:44.453786 (+   665us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=54,mrs_lookups=0
0709 13:05:44.453873 (+    87us) write_op.cc:312] APPLY: finished
0709 13:05:44.454682 (+   809us) log.cc:844] Serialized 236 byte log entry
0709 13:05:44.704674 (+249992us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:44.704943 (+   269us) write_op.cc:454] Released schema lock
0709 13:05:44.705042 (+    99us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:44.705511 (+   469us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:44.705538 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2915792,"reported_tablets":48,"spinlock_wait_cycles":971342080,"child_traces":[["op",{"apply.queue_time_us":344640,"apply.run_cpu_time_us":71598,"apply.run_wall_time_us":3428950,"cfile_cache_hit":110,"cfile_cache_hit_bytes":116987,"num_ops":27,"prepare.queue_time_us":259259,"prepare.run_cpu_time_us":20306,"prepare.run_wall_time_us":363627,"raft.queue_time_us":487223,"raft.run_cpu_time_us":23170,"raft.run_wall_time_us":403907,"replication_time_us":1020160,"thread_start_us":1043705,"threads_started":5,"wal-append.queue_time_us":464533,"wal-append.run_cpu_time_us":40913,"wal-append.run_wall_time_us":2965734}]]}
I20260709 13:05:47.760178  1422 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:47.782680 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 147720us
I20260709 13:05:47.783488 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:47.788329 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:47.788556 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:47.789008 29906 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:47.789511  1046 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:47.790248 29908 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:05:47.790809 29908 pending_rounds.cc:85] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 7
I20260709 13:05:47.791154 29908 pending_rounds.cc:107] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:05:47.791445 29908 raft_consensus.cc:2889] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:47.788359 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:47.790621  1422 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.792260  1451 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:47.793260  1422 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:47.796559  1403 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:47.797083  1403 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.797501  1403 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:47.775027  1545 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.786912 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.777014  1112 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:47.769332  1209 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:47.801688 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.803805 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 59759.1us
I20260709 13:05:47.803727 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:47.802071 28968 catalog_manager.cc:5697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 1 to 7, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 7 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:47.808748 29222 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:47.809046 29905 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:47.807140 29906 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
W20260709 13:05:47.812366 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 69404.6us
I20260709 13:05:47.814131 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:47.815528 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.817590 29070 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:47.819696 29908 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:47.808744  1209 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:47.800694  1545 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:47.821168 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 74769.1us
I20260709 13:05:47.828940 29215 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:47.702682  1536 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:05:47.828778 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.830377 29070 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:47.820024  1217 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:47.836053  1046 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.722517  1537 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:05:47.836716  1046 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.754066  1539 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:05:47.838922  1046 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:47.843580 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.844689  1538 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 13:05:47.846616  1543 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:05:47.847039  1403 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.848062 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.848686 29215 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
W20260709 13:05:47.852360  1544 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:05:47.852661  1209 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:47.854923 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.855106 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.856235 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:47.857940  1403 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:47.858060  1546 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.871129 29185 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:47.895665 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55281.5us
I20260709 13:05:47.896167 29905 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:47.783195  1452 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:47.865581 29037 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b9574de46fcd490e84208e41907200b0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:47.897610 29186 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:47.902192 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:47.897925 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 84507.9us
I20260709 13:05:47.907328 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.907742 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.909657 29218 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:47.909884 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:47.881325  1046 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.887478  1539 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:47.899186 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:47.910786 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.916579 29216 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:47.917861 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:47.913256 29213 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:47.919885 29213 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:47.921124 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.922232 29221 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:47.922613 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.923867 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:47.924693 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:47.925531 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:47.880452  1403 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:05:47.930965 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:47.927366 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:47.910594  1198 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:47.934232 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:47.934427 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:47.930397 29213 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.880961  1546 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.929911  1209 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:47.923673  1451 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:47.946673 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:47.952358 29142 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:47.948428 29213 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:47.950048 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:47.947992  1198 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:47.960194  1198 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:47.961297  1530 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:47.962009 29070 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:47.937415  1273 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:47.965310 29905 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:47.965941 29218 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:47.967301 29218 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:05:47.968664 29218 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:05:47.971959 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:47.972450 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.949327 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:47.957124 29139 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:05:47.934085   884 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:47.973806 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:47.997293 29146 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:47.994917 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:47.995016 29070 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:05:47.944471  1195 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:48.005657 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.006789 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:48.005208 29215 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:48.009328 29908 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:05:47.921638  1046 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:48.012672 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 78520.3us
I20260709 13:05:48.013439 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:48.019865 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.025018  1536 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:48.029178  1530 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:48.029650  1530 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:48.030031  1530 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:48.036646  1540 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.388s	user 0.003s	sys 0.003s
I20260709 13:05:47.944205  1542 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:05:48.016185  1423 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:48.053627 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.054497 29142 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:48.043733  1531 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:48.055321  1531 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:48.066721  1235 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:48.068252 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:48.069235 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:48.069671 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 181255us
I20260709 13:05:48.069977  1535 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:48.070359  1195 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:48.061307  1423 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:48.071422 29874 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:48.071709  1503 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:48.072366  1548 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:48.067698 29147 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:48.070436  1535 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:48.022586  1237 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.688s	user 0.015s	sys 0.000s
I20260709 13:05:48.073302 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.075841 29141 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.076143 29185 leader_election.cc:400] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15.
I20260709 13:05:48.021740  1451 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:48.005162  1537 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.068s
W20260709 13:05:48.083243 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 67726.6us
I20260709 13:05:48.005050  1336 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:05:48.088214 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:48.092320 29113 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:48.097043 29113 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:48.076802  1535 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:05:48.100144 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 94123.7us
I20260709 13:05:48.100673 29213 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:05:48.101823 29213 pending_rounds.cc:85] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 10
I20260709 13:05:48.102126 29213 pending_rounds.cc:107] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:05:48.102406 29213 raft_consensus.cc:2889] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:48.083886  1451 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.107025 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:48.109881 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 203048us
I20260709 13:05:48.105261  1120 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:48.113509  1120 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:48.113901  1120 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:48.101046  1099 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:05:48.121599 29146 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:48.098608  1237 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:48.141104 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 64912us
I20260709 13:05:48.137673  1554 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:48.144394  1554 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:48.144850  1554 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.117944  1551 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:05:48.149349  1551 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:48.150807  1551 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:48.105876  1553 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:05:48.072490  1423 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:48.155742 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.100044  1458 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:48.152513 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.159394 29144 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:48.159703 29146 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:48.164529 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 140884us
I20260709 13:05:48.167945 29216 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
W20260709 13:05:48.171531 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 255095us
I20260709 13:05:48.174399 29141 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:48.109432  1451 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:48.094077  1217 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
W20260709 13:05:48.117467  1445 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:05:48.176239 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 63662.4us
I20260709 13:05:48.192432 29146 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:05:48.194198 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.143028  1237 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.174520  1539 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.202463 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.203540 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.166457  1458 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:48.142032  1112 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:05:48.213981 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:48.213516 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 279133us
I20260709 13:05:48.194800  1209 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:48.203898  1237 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.181108  1195 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:05:48.092909  1531 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.230002 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:48.231200 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.234264 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.211172  1458 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:48.251376 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 130421us
I20260709 13:05:48.252800 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:48.253580 29070 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.261045 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.261996 29147 raft_consensus.cc:2410] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:05:48.206288  1536 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:05:48.270126 29141 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:48.276181 29906 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:48.280494 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.280985 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.223984  1539 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:48.282752 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.289017 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:48.293892 29072 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:48.297021 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.298238 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:48.260782  1413 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.890s	user 0.015s	sys 0.002s
I20260709 13:05:48.305202 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.305677 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.259339  1557 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:48.320222 29068 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:48.321847 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:48.322700 29064 raft_consensus.cc:2376] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15. Re-sending same reply.
I20260709 13:05:48.328991 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:48.330821 29066 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:48.331709 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.336354 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.337975 29070 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
W20260709 13:05:48.338798 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 138177us
I20260709 13:05:48.339563 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:48.347579 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 117689us
I20260709 13:05:48.356621 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.357033 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:48.359026 29141 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:48.360246 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.364514 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:48.305122  1551 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:48.317410  1403 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:48.365265 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:48.365650 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2456) took 1037 ms. Trace:
I20260709 13:05:48.370795 29220 rpcz_store.cc:276] 0709 13:05:47.316519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.329099 (+ 12580us) service_pool.cc:224] Handling call
0709 13:05:48.353413 (+1024314us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b540033aed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:48.353537 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2094314880}
I20260709 13:05:48.376787 29066 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:48.376878 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:48.377839 29072 raft_consensus.cc:2376] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13. Re-sending same reply.
I20260709 13:05:48.345196  1423 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.391502 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.394392 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:48.351310  1546 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:48.399082 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.400959 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.394731 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.405016 29144 raft_consensus.cc:2410] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:05:48.368054  1046 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:48.416213 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 76766.1us
I20260709 13:05:48.416855 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:48.409292 29188 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:48.368362  1575 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:48.350651  1530 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:48.397354  1502 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.004s
I20260709 13:05:48.421697 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.428222 29039 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:48.430733 29141 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:05:48.431792 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.433035 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.434223 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.435267 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.437032 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.438033 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.439069 29141 raft_consensus.cc:2410] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:05:48.439872 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.440577 29141 raft_consensus.cc:2368] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12.
I20260709 13:05:48.441670 29141 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:48.445288 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:48.487509 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.488324 29142 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:05:48.412721  1195 consensus_queue.cc:1243] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:05:48.493427  1561 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:05:48.428619  1403 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:48.424620  1575 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:05:48.495232  1559 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:05:48.496407  1565 log.cc:927] Time spent T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.238s	user 0.000s	sys 0.000s
I20260709 13:05:48.497074  1452 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:48.498394 29145 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:48.499666 29145 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
W20260709 13:05:48.489650 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 80478.5us
I20260709 13:05:48.490088 29066 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:05:48.497121  1502 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.503008 29147 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
W20260709 13:05:48.504271 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 87468.5us
I20260709 13:05:48.505589 29142 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
W20260709 13:05:48.492492  1560 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:05:48.501797 29145 pending_rounds.cc:85] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 6
I20260709 13:05:48.514446 29145 pending_rounds.cc:107] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:05:48.514832 29145 raft_consensus.cc:2889] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:48.508701 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:48.359781  1557 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.380596  1551 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.406812  1423 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.415474  1546 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 13:05:48.504925  1403 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:48.420274  1273 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 13:05:48.495189  1575 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.534219 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.534276 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:48.552059 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:48.552366 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.519738  1446 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:48.532275  1553 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:48.556383 29072 raft_consensus.cc:2376] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13. Re-sending same reply.
I20260709 13:05:48.562424 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.552402 29071 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:48.556833 29068 raft_consensus.cc:2376] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10. Re-sending same reply.
I20260709 13:05:48.571050 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.519691  1502 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:48.576915 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 154513us
W20260709 13:05:48.554052  1569 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:05:48.586834 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:48.554268  1555 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:48.593523 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:48.594764  1553 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:48.595398 29039 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:48.596549  1553 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:48.596714 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:48.558140 29069 raft_consensus.cc:2955] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 5 to 8, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:48.572213  1120 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:48.604384  1403 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.542303  1413 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:05:48.589064  1539 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:48.625039 29144 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: []
W20260709 13:05:48.609136  1571 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:05:48.606631  1548 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.624577  1235 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.597463  1451 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:48.603976  1237 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:48.712844 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 116229us
I20260709 13:05:48.635454  1195 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.646821  1446 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:48.720439 29185 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.690s (SENT) [suppressed 29 similar messages]
I20260709 13:05:48.745824 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:48.747206 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:48.747749 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:48.748338 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:48.718782  1530 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:48.755086 29146 raft_consensus.cc:2368] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12.
I20260709 13:05:48.607110  1578 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:48.628605  1570 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:48.609161  1581 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.637856  1583 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:48.777460  1583 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:48.779017  1343 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.994s	user 0.011s	sys 0.003s
I20260709 13:05:48.725355 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:48.724799 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:48.727432 29067 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:48.729568  1451 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:48.732229  1120 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:48.732439  1572 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:05:48.733098  1423 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:48.734216  1563 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:05:48.738737 29216 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:05:48.751552  1577 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:48.651655  1548 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:48.775640  1545 consensus_meta.cc:287] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.945s	user 0.013s	sys 0.000s
W20260709 13:05:48.715904  1566 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:05:48.782902 29911 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:48.792707  1446 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:48.787568  1343 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.724500  1235 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:48.804813 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 80115.5us
W20260709 13:05:48.795109  1458 consensus_meta.cc:287] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.527s	user 0.004s	sys 0.007s
W20260709 13:05:48.795755  1535 consensus_meta.cc:287] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.002s
I20260709 13:05:48.811839  1545 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.751677  1195 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:48.814726  1598 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:48.818060 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.821165 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:48.826262 29907 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:48.794209  1555 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:48.800923  1502 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:48.846954 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:48.816469  1593 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:05:48.825912  1554 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.681s	user 0.007s	sys 0.004s
I20260709 13:05:48.853161 29911 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:48.830508  1553 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:48.812446  1120 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:48.817886  1600 ts_tablet_manager.cc:933] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:05:48.820469  1548 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:48.827365  1545 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:48.847568  1555 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:48.842713  1530 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:48.911384 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:48.912395 29905 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:48.874212  1554 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:48.825212  1458 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:48.902215 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 65480.7us
I20260709 13:05:48.900092  1552 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:48.901187  1590 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:05:48.914646  1534 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.290s	user 0.015s	sys 0.002s
I20260709 13:05:48.874969  1553 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:48.878691  1120 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.911576 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:48.921780 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 201357us
I20260709 13:05:48.830466  1598 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:48.924315 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.825436  1535 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:48.925783 29911 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:48.906105  1570 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:48.944645 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:48.948341 29140 raft_consensus.cc:1217] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 10.9->[12.10-15.12]   Dedup: 12.10->[13.11-15.12]
I20260709 13:05:48.916571  1551 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:48.948552 29145 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:48.952092 29139 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:48.948879 29140 pending_rounds.cc:85] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 10
I20260709 13:05:48.944629 29141 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:48.952790 29140 pending_rounds.cc:107] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:05:48.953099 29140 raft_consensus.cc:2889] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:48.957054  1605 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.954833  1570 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.957679  1605 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.929647  1452 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:48.977779 29216 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:05:48.962342 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:48.959153  1570 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:48.965376  1535 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:48.965969  1553 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:48.982733  1553 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:48.983408  1583 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:48.940655  1451 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:48.945102  1237 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:48.985204  1120 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:48.978785 29143 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:48.987093  1598 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:48.988605  1554 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:48.989740 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 797) took 1202 ms. Trace:
I20260709 13:05:48.990130 29907 rpcz_store.cc:276] 0709 13:05:47.787449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.787583 (+   134us) service_pool.cc:224] Handling call
0709 13:05:48.989488 (+1201905us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54005bd1d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:48.989670 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2384663424}
I20260709 13:05:48.969316  1458 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:48.969717  1603 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:05:48.970093 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48114.6us
I20260709 13:05:48.995069 29145 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:48.995326 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:48.941154  1577 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:48.955408  1545 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:48.945325  1534 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:48.986714  1451 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:48.944852  1594 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:48.968670 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:48.992158 29909 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:05:48.993577 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.997844 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:48.999490 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:49.002486  1545 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:49.003612  1545 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.005391  1451 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:49.005985 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:49.006735 29145 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:49.016258 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:49.018929 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:49.021992 29219 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:05:49.021517 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:49.030856 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:49.030678 29911 raft_consensus.cc:2368] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13.
W20260709 13:05:49.044788 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47027.5us
I20260709 13:05:48.998940  1554 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:49.040629 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:49.014369  1273 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.055831 29875 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:49.018062  1581 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:49.021781  1551 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:48.996443  1555 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.021938  1577 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.070309  1577 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:49.071794 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:49.072700 29905 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 9: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:49.079459 29037 leader_election.cc:400] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13.
I20260709 13:05:49.080017 29037 leader_election.cc:403] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:05:49.063426 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 153532us
I20260709 13:05:49.079756  1403 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:49.085390  1403 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.090088  1403 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:49.078128  1273 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.092051  1273 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:49.093585 29905 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:49.060876  1458 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:49.104701 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:49.085659  1554 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:05:49.080703  1605 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:49.014717  1594 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:49.113720  1458 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:49.115810  1613 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:49.116999  1613 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:49.117089  1235 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:49.117424  1613 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:49.103107  1590 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:05:49.111884  1099 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:05:49.118443  1613 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.119105  1099 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:49.119495  1099 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:49.120378  1099 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.119803  1613 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:49.121635  1099 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:49.124459 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:49.132858  1235 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:49.134442  1592 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:49.135095  1592 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.135704  1551 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:49.136312  1551 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:49.146052 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:49.148319  1592 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:49.149643  1551 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.157392 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:49.159207 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:49.153342   884 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:49.162642   884 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:49.163050   884 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:49.163311  1235 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:49.163808  1235 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:49.164230  1235 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:49.168129  1458 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:49.144335  1600 tablet_copy_client.cc:323] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.129:41631
I20260709 13:05:49.171000  1403 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.174552  1403 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:49.152171  1545 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.175452  1545 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.176157 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 716) took 1389 ms. Trace:
I20260709 13:05:49.176757 29909 rpcz_store.cc:276] 0709 13:05:47.786619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.786795 (+   176us) service_pool.cc:224] Handling call
0709 13:05:48.991734 (+1204939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:48.999191 (+  7457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.175874 (+176683us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54005bd1d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:49.176080 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2744892672}
I20260709 13:05:49.178390  1545 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:49.178699 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2274) took 1748 ms (client timeout 1690 ms). Trace:
W20260709 13:05:49.179148 29067 rpcz_store.cc:269] 0709 13:05:47.430145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.430260 (+   115us) service_pool.cc:224] Handling call
0709 13:05:49.178503 (+1748243us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54002aba54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:49.178628 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2080725888}
I20260709 13:05:49.190140 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:49.194818 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:49.196568  1614 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:05:49.197861 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:49.199590  1491 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:49.200049  1491 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:49.200424  1491 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:49.201360 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:49.203251  1423 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:49.203547 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:49.204243  1423 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:49.205281  1120 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:49.205799  1120 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:49.206733  1120 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.196390 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:49.212185  1578 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:49.216427  1195 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:49.218035  1539 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:49.219434  1554 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13.
I20260709 13:05:49.199426  1337 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:49.191654 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:49.237228 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 45652.3us
I20260709 13:05:49.220994 29140 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:49.233304 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:49.239773 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 154559us
I20260709 13:05:49.245288 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:49.249122 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.251238 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.248189 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:49.243219 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:49.193147   884 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.239773  1458 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:49.254562 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:49.255110  1545 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:49.256064  1594 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:49.256799 29069 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:05:49.257302  1539 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:49.212514  1574 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:05:49.259092  1423 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:49.247646  1099 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:05:49.260558 29146 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:49.260744 29909 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:49.262633 29144 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:49.268962 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.272555 29064 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:49.264099  1545 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.275411 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:49.287863 29145 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:49.288331 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:49.290566 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:49.290855 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.293587 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:49.295147 29144 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:49.294896 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.296223 29072 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
W20260709 13:05:49.298879 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 65647.4us
I20260709 13:05:49.300666 29066 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:05:49.288036  1545 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:49.316509 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.316854 29072 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:49.318215 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.319279  1554 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:49.324304 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:49.325031 29073 raft_consensus.cc:2376] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14. Re-sending same reply.
I20260709 13:05:49.273996  1539 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:49.327107 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.328032 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:49.328131 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:49.284451  1120 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:49.254068  1491 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.329169 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.331573 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:49.357036 29069 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 10: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:49.361318 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:49.363106 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:49.367555 29875 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:49.353825  1209 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:49.319834  1554 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.369300  1591 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:49.367624  1503 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:49.391659  1235 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.392416  1503 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.392764  1422 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:49.385284  1547 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:05:49.393306  1422 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:49.358454  1364 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:49.393925  1547 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:49.371244  1599 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:49.371698 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2402) took 1099 ms. Trace:
I20260709 13:05:49.372438  1623 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:49.373312  1586 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:49.377679   884 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:49.382508  1555 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:49.394292  1422 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.394243  1235 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:49.371721  1549 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:05:49.392802  1503 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:49.408344  1551 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:49.389580 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 89982.9us
I20260709 13:05:49.390695  1629 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.387046  1209 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.401739 29071 rpcz_store.cc:276] 0709 13:05:48.271942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.272081 (+   139us) service_pool.cc:224] Handling call
0709 13:05:49.371536 (+1099455us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54002cbad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:49.371639 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546970880}
I20260709 13:05:49.395123  1364 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.402894  1599 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.394320  1547 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:49.415939  1574 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:49.414199  1549 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:49.403961  1623 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.330204  1594 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:05:49.456195 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:49.462850 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:49.485841 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 71522.2us
I20260709 13:05:49.486761 29068 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:49.487112 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:49.454401  1209 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:49.467703 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 103819us
I20260709 13:05:49.417290  1586 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:49.460605  1599 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:49.462248  1422 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:49.492408 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:49.455557  1552 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:49.495039 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.496536 29908 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:49.501953 29905 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:49.463697  1235 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:49.460767  1364 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.506419 29082 tablet_copy_service.cc:140] P a936878f59c840a6bd176a1d2d7347b1: Received BeginTabletCopySession request for tablet b9574de46fcd490e84208e41907200b0 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:40862)
I20260709 13:05:49.507153 29082 tablet_copy_service.cc:161] P a936878f59c840a6bd176a1d2d7347b1: Beginning new tablet copy session on tablet b9574de46fcd490e84208e41907200b0 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:40862: session id = 7e78ffbcc98242068a874a5d9e341477-b9574de46fcd490e84208e41907200b0
I20260709 13:05:49.507578  1364 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:49.509137  1639 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:49.509374 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:49.511206  1452 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:49.528931 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.529604 29070 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
W20260709 13:05:49.592106 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 105056us
I20260709 13:05:49.592939 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:49.457334  1629 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:49.596271 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 102104us
I20260709 13:05:49.598156  1633 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.136s
W20260709 13:05:49.606812  1642 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:05:49.519251  1586 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.609059  1503 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.610843  1458 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:49.611405  1503 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:49.454344  1539 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.612869  1539 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:49.616568 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.616509  1099 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:49.617794  1554 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:49.619637  1612 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.536s	user 0.000s	sys 0.004s
I20260709 13:05:49.594589  1452 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.489243  1549 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.595831  1623 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:49.625140  1343 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:49.624706  1547 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.625873  1491 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:49.487933  1574 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.634977 29068 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:49.637506 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:49.650235 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.655025 29908 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:05:49.674007 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:49.676712 29039 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.532s (SENT)
I20260709 13:05:49.687556 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:49.534181  1557 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:49.535033  1235 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:49.705591 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51462.5us
W20260709 13:05:49.690752 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 97884.7us
I20260709 13:05:49.707821 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:49.655043 29082 tablet_copy_source_session.cc:215] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:49.673710  1336 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:49.664958  1578 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.665824  1554 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:49.729182 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.533746  1552 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.705613  1343 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.664618  1209 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.708799  1623 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:49.718410  1618 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:05:49.747982  1641 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:05:49.686432  1530 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:05:49.757025 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2512) took 1019 ms. Trace:
W20260709 13:05:49.764933 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 88276.7us
W20260709 13:05:49.717651  1613 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.596s	user 0.007s	sys 0.010s
I20260709 13:05:49.682895  1545 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:49.767465 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59733.5us
I20260709 13:05:49.768301 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:49.764532 29220 rpcz_store.cc:276] 0709 13:05:48.737794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.737946 (+   152us) service_pool.cc:224] Handling call
0709 13:05:48.753153 (+ 15207us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:48.753581 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:48.753619 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:48.794908 (+ 41289us) log.cc:844] Serialized 506 byte log entry
0709 13:05:48.925341 (+130433us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:48.925394 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.925506 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:49.618855 (+693349us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:49.745033 (+126178us) raft_consensus.cc:1685] finished
0709 13:05:49.745063 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.756900 (+ 11837us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6039936,"thread_start_us":89334,"threads_started":1,"wal-append.queue_time_us":253840}
I20260709 13:05:49.754415  1639 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:49.740932  1554 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.777891 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:49.754009  1423 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:05:49.761807  1592 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:49.746111  1578 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.764570  1336 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:49.782636  1632 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:05:49.756150  1343 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:49.781354  1577 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:49.797049 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.798149 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:49.799791 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:49.800356 29066 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:49.790241  1613 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:49.802647 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 165243us
I20260709 13:05:49.803876 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.790009  1639 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:49.807289 29068 raft_consensus.cc:2410] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:49.809795 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:49.811127 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.822414  1336 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:49.824615  1605 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:49.825376  1605 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:49.828457  1638 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:05:49.831315  1545 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:49.832015  1337 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:49.832636  1337 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:49.823438  1549 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:49.897401 29068 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:05:49.897449 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 87391.9us
I20260709 13:05:49.900307 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:49.907698 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:49.909405 29068 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:49.910768 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:49.912930 29068 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: [6.6-10.8]
I20260709 13:05:49.917737 29068 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:49.919088 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:49.921569 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:49.923611 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:49.925545 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:49.928104 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:49.930779 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 242537us
I20260709 13:05:50.013168 29139 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:50.013804 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:50.015254 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.015420 29070 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:50.018170 29142 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:05:50.013635 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 205608us
I20260709 13:05:50.019608 29068 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:05:50.019547 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.020402 29146 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 10: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:50.020354 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:50.020648 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.022550 29140 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:50.023715 29144 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:50.024302 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:50.026579 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.024888 29141 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:50.028281 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.029680 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.030259 29147 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:50.033381 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.033707  1217 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:50.034548 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.035398 29068 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:50.035588 29143 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:05:50.036180 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.036935  1337 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:50.038028  1337 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:50.038133 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.038900 29147 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:50.034600 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.039880 29140 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:50.040812 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:50.035280  1217 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:50.043699 29139 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:50.047401 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.049222 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:49.930070  1452 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:50.059836 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.017889  1586 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.017622  1605 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:50.064018 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 145005us
I20260709 13:05:50.074671 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:50.075385 29067 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:50.060262 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:50.076400 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.076512 29070 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:50.029501  1613 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.068714 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.078081 29147 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:50.024853  1562 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:50.064028 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:50.076071 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:50.073208 29909 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:05:50.032848  1577 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:50.057255  1237 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.118916 29071 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:50.103775 29070 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:50.109629 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:50.102785 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:50.129897 29216 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 9.
I20260709 13:05:50.131906 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:50.132587 29219 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:50.075094  1574 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:50.090296  1613 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:50.076021  1209 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:50.131131 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:50.134006 29070 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:05:50.135432 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:50.136003 29217 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:50.144433 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:50.144969 29070 pending_rounds.cc:85] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 11
I20260709 13:05:50.145372 29070 pending_rounds.cc:107] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:05:50.145432 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:50.145695 29070 raft_consensus.cc:2889] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:50.135406 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:50.146956 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:50.085325  1452 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.118189  1562 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:50.137585 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:50.159406 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.159682 29215 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:50.137921 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:50.139789  1551 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.002s
I20260709 13:05:50.140020 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:50.124792  1631 log.cc:927] Time spent T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:05:50.125933 29141 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:50.160156 29070 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:50.161057 29067 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:50.160853 29213 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:50.161705 29218 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
W20260709 13:05:50.163980 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.140103  1651 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.208114 29064 raft_consensus.cc:2410] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:05:50.121415  1237 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:50.085309  1639 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:50.089689  1586 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:50.115803  1577 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:50.152344  1574 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.148631  1578 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:50.231046 29144 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:50.211135  1649 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.159351  1554 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:50.214412  1650 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.220993  1343 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:50.244292 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 168281us
I20260709 13:05:50.144534  1217 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:50.252923 29218 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:05:50.256232 29039 leader_election.cc:400] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12.
I20260709 13:05:50.256918 29039 leader_election.cc:403] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:50.257789  1551 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12.
I20260709 13:05:50.265547  1237 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:50.219481  1209 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
W20260709 13:05:50.270131  1422 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.756s	user 0.009s	sys 0.000s
I20260709 13:05:50.235188  1574 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:50.273885  1574 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:50.220464  1557 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:05:50.241223  1578 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.275830  1578 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:50.277006  1613 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:50.278038  1578 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:50.278049 29068 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:50.243697  1649 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:50.278517 29066 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:50.281323  1649 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:50.281598  1586 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:50.240949  1643 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:05:50.285193 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.294425 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.296653 29908 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:05:50.220271  1452 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:50.213601  1651 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:50.301736 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:50.341945 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:50.305269 29144 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:05:50.344100 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:50.345029 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:50.346150 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:50.369536 29214 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:50.369804 29908 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:50.369881 29216 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:50.347397 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:50.370816 29217 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 9.
I20260709 13:05:50.349078 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:50.371896 29220 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:50.247869  1650 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:50.351188  1577 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:50.359143  1599 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.856s	user 0.007s	sys 0.003s
I20260709 13:05:50.360169  1554 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:50.360633  1557 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:50.364673 29911 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:05:50.365196 29909 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 13:05:50.299417  1574 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:50.366871 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.387701 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:50.373410 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 192271us
I20260709 13:05:50.374752 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:50.240756  1654 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:50.252844  1655 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:50.376168 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 245130us
I20260709 13:05:50.383072 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:50.384301 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:50.300155 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.402930 29215 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:50.403970 29213 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:50.407893 29221 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:50.408968 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.392724 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:50.410704 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:50.413669 29214 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:50.416066 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:50.417778 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:50.419485 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:50.420627 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:50.423266 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:50.424885 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:50.426038 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:50.428547 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:50.392014  1654 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:50.423931 29911 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 9.
I20260709 13:05:50.430362 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:50.290242  1551 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.131s
I20260709 13:05:50.419271  1580 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:05:50.443188 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.437871 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2309) took 1024 ms. Trace:
I20260709 13:05:50.383888  1599 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:50.423584  1537 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:50.444466 29073 rpcz_store.cc:276] 0709 13:05:49.413715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.415215 (+  1500us) service_pool.cc:224] Handling call
0709 13:05:49.415620 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:49.416008 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:49.416044 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:49.416656 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 13:05:49.753351 (+336695us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:49.753485 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:49.754843 (+  1358us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:50.437575 (+682732us) raft_consensus.cc:1685] finished
0709 13:05:50.437605 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:50.437787 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2460,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":6337,"thread_start_us":336070,"threads_started":1,"wal-append.queue_time_us":385826}
I20260709 13:05:50.455127 29071 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:50.432263  1343 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:50.421881  1422 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:05:50.410092  1574 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:50.434851  1654 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:50.432331  1557 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:05:50.454478  1537 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:50.433948  1661 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:05:50.470115  1554 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:50.504777  1554 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.505403  1554 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:50.507536  1554 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:50.497152 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.500460  1577 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.501042  1650 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:50.468945  1209 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:50.487816  1574 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:50.516556 29071 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:50.520131  1600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9574de46fcd490e84208e41907200b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:50.519775 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:50.532989 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:50.522814  1551 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:50.523166  1649 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:50.524762  1586 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:50.525372 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.526726 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:50.526804 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:50.481869  1537 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:50.546443 29065 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:50.547279  1650 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:50.547821  1650 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:50.547884 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:50.548826  1650 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:50.482158  1452 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:05:50.552820 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.553715  1452 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:50.553790 29065 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:50.554435  1452 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:50.555023  1452 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:50.507545  1599 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:50.541605 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 104869us
I20260709 13:05:50.546558 29073 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:50.563731 29222 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:50.565856  1580 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:50.566061 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 109068us
I20260709 13:05:50.511109  1557 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:05:50.575803 29221 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
W20260709 13:05:50.541057  1530 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.005s	sys 0.005s
I20260709 13:05:50.531435  1209 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:50.576208 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:50.538718  1591 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:50.572990  1629 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:50.570031  1592 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.786s	user 0.009s	sys 0.004s
I20260709 13:05:50.536657  1605 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:50.580528 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:50.583647 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.588454 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.594362 29142 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:05:50.596040 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:50.539065  1551 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:50.543630  1656 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:50.598796 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:50.605059 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.626869 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:50.627641 29142 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:50.606925 29073 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:50.628813 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.531894  1577 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:50.580432  1530 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.005s	sys 0.005s
I20260709 13:05:50.629266 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.647505 29143 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:05:50.648744 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:50.629490 29141 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 8.
I20260709 13:05:50.651335 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.652403 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.653296 29141 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:50.654409 29141 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:50.655414 29141 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:50.628829  1629 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:50.656180  1629 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:50.656571  1629 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:50.630463 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:50.607818  1235 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:50.580294  1591 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:50.580134  1209 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:50.543272  1422 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:05:50.662513 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:50.664340 29143 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:50.665231 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:50.601737  1592 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:50.670142 29148 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:50.628046  1648 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:50.598443  1679 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:50.636039  1605 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:50.684331 29142 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:50.591812  1674 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:05:50.602262  1675 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:05:50.664945  1591 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:50.686450  1648 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:50.699550  1580 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:50.682489  1551 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:50.714231  1552 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:50.715317  1577 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:50.665849  1651 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:50.674877  1536 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:50.723724 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:50.670802  1682 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:50.674551  1235 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:50.722278  1549 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:50.669751  1681 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:05:50.708544  1648 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:50.786263  1672 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
I20260709 13:05:50.790270 29143 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:05:50.733846  1235 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:50.730712  1590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.000s	sys 0.009s
I20260709 13:05:50.954767  1120 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.778430  1549 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:50.686797  1680 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.206s
I20260709 13:05:50.781759  1536 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:50.882001  1530 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:50.956056  1590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.126s	user 0.001s	sys 0.009s
I20260709 13:05:50.947955  1675 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:50.735366  1682 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:50.976001  1195 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.060s	sys 0.060s
I20260709 13:05:50.955979  1674 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.021818  1671 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.454s	user 0.000s	sys 0.001s
I20260709 13:05:50.945688  1616 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:50.948357  1577 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:51.021616  1549 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:50.976433  1639 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:51.063414 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:51.108767 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 45481.2us
I20260709 13:05:51.109676 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.110533 29222 raft_consensus.cc:2410] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:05:51.110399 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:51.111248 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.111598 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:51.027140  1542 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:51.055879  1643 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:05:51.023252  1120 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:50.948153  1605 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.073324  1536 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.067099  1235 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:50.950850  1654 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.067576  1675 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.082841 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2315) took 1057 ms. Trace:
I20260709 13:05:51.129444 29144 rpcz_store.cc:276] 0709 13:05:50.025488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.045653 (+ 20165us) service_pool.cc:224] Handling call
0709 13:05:51.082611 (+1036958us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54003cc3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:51.082761 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334960896}
I20260709 13:05:51.088798 29143 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:51.093936 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 431480us
W20260709 13:05:50.991154 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 360763us
I20260709 13:05:51.067651  1599 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:05:51.068747  1574 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.054s
W20260709 13:05:51.116489 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.116551 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.079457  1591 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.050448  1530 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.105201  1652 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:51.104903  1557 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.025146  1594 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.191s	user 0.010s	sys 0.017s
I20260709 13:05:51.116293  1577 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.126557  1590 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.105345  1666 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:51.135118 29188 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:51.069165  1195 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.068s	sys 0.072s
I20260709 13:05:51.139940 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:51.106226  1616 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.147862 29222 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:05:51.141636  1594 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.331s	user 0.010s	sys 0.017s
I20260709 13:05:51.136943  1696 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:51.138572  1235 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:51.121227  1639 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.098088  1674 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.187163 29218 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:51.151423  1676 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.426s	user 0.000s	sys 0.002s
I20260709 13:05:50.985499  1343 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:51.195199 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 60173.3us
I20260709 13:05:51.213738 29144 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:51.149623  1590 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.143325  1557 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.199307  1530 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:51.201282  1195 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:51.201982  1675 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:51.203377 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.125025  1542 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.210697  1591 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:51.211658  1629 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.000s
I20260709 13:05:51.146507  1599 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
W20260709 13:05:51.216509 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.228148 29143 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:05:51.232414 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:51.196267  1422 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:51.239974  1675 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.244740  1343 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:51.245654  1675 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.247165  1557 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:51.203418  1654 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.152092  1666 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.249260  1675 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:51.251040 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.251968 29143 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:51.251071  1666 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:51.242056 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.143644  1652 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.212405  1680 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:51.259858 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.290982 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2520) took 1076 ms. Trace:
I20260709 13:05:51.326574 29213 rpcz_store.cc:276] 0709 13:05:50.215135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.229568 (+ 14433us) service_pool.cc:224] Handling call
0709 13:05:50.403689 (+174121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.404427 (+   738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.290758 (+886331us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540009fbd0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:51.290908 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395477248}
I20260709 13:05:51.268438  1616 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:51.236930  1685 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
W20260709 13:05:51.325587 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 93263.6us
W20260709 13:05:51.255838  1209 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.581s	user 0.007s	sys 0.000s
I20260709 13:05:51.326303 29143 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
W20260709 13:05:51.233810  1686 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 13:05:51.257611  1537 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.711s	user 0.009s	sys 0.001s
I20260709 13:05:51.244369  1629 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.335388 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:51.275592  1452 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.720s	user 0.009s	sys 0.000s
I20260709 13:05:51.350052 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.363524 29213 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:05:51.278869  1120 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:51.277468  1639 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:51.333042  1697 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.120s	user 0.000s	sys 0.003s
I20260709 13:05:51.256778  1577 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:51.266427  1682 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:51.333338  1688 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.210s	user 0.002s	sys 0.002s
I20260709 13:05:51.334376 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.346920  1666 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:51.375368 29222 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:51.376878 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.265872  1542 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:51.334654  1687 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.148s	user 0.001s	sys 0.002s
I20260709 13:05:51.227689  1696 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.119s
W20260709 13:05:51.345498  1644 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 1.522s	user 0.004s	sys 0.006s
I20260709 13:05:51.335650  1195 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:51.336028  1599 consensus_queue.cc:1243] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:05:51.343365  1551 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:51.334910  1693 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:05:51.373485 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 799) took 1070 ms. Trace:
I20260709 13:05:51.352829  1209 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.390827 29215 rpcz_store.cc:276] 0709 13:05:50.303257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.433260 (+130003us) service_pool.cc:224] Handling call
0709 13:05:50.433588 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.545081 (+111493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.545114 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.545617 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:05:50.649864 (+104247us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:50.649917 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:50.650011 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.373242 (+723231us) raft_consensus.cc:1685] finished
0709 13:05:51.373267 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.373412 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35418,"raft.queue_time_us":49547,"spinlock_wait_cycles":38332672,"thread_start_us":155692,"threads_started":2,"wal-append.queue_time_us":163611}
I20260709 13:05:51.376919 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.316146 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:51.366169  1537 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.368276  1452 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.385061 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:51.396459 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 80415.5us
I20260709 13:05:51.426909 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:51.427814  1675 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:51.428005  1639 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:51.428697  1639 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.429459  1674 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:51.345686  1652 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.385933  1195 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.432602 29217 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:51.442139 29215 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:51.443526 29215 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:51.380479  1666 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.449306 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:51.431098  1639 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:51.456638 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 345085us
I20260709 13:05:51.457633 29186 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:51.458128 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:51.462997 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:51.423846  1701 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:51.459621 29215 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.448403  1195 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:51.469077 29215 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:51.473999  1574 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:51.474221 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:51.474988 29907 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:51.476302  1574 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.476876  1574 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.480628  1629 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:51.467207  1530 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:51.481870  1530 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:51.482321  1530 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:51.454223  1652 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.484162 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:51.459353  1674 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:51.569092 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.520373  1422 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:51.578478  1422 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.461494  1557 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.112s
I20260709 13:05:51.577288  1701 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:51.588837  1701 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.468605  1554 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:51.598057 29215 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:05:51.598744 29215 pending_rounds.cc:85] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 9
I20260709 13:05:51.599109 29215 pending_rounds.cc:107] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:05:51.599486 29215 raft_consensus.cc:2889] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:51.456270  1675 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:05:51.471957  1600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:54902 (request call id 771) took 2882 ms. Trace:
I20260709 13:05:51.605230 29908 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 13:05:51.520200  1694 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.516134  1648 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.623849 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2535) took 1040 ms. Trace:
W20260709 13:05:51.624280 29071 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.011s	sys 0.021s
I20260709 13:05:51.484187  1652 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:51.624651 29071 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.011s	sys 0.021s
I20260709 13:05:51.603667  1554 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:51.631001 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2327) took 1018 ms. Trace:
I20260709 13:05:51.624423 29140 rpcz_store.cc:276] 0709 13:05:50.583991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.584174 (+   183us) service_pool.cc:224] Handling call
0709 13:05:50.624963 (+ 40789us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.625538 (+   575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.625576 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.635213 (+  9637us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.121162 (+485949us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:51.623130 (+501968us) log.cc:844] Serialized 14 byte log entry
0709 13:05:51.623434 (+   304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.623530 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.623585 (+    55us) raft_consensus.cc:1685] finished
0709 13:05:51.623602 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.623773 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19916,"prepare.queue_time_us":252695,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":13846,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":15776,"thread_start_us":938648,"threads_started":2,"wal-append.queue_time_us":704636}
I20260709 13:05:51.637297 29071 rpcz_store.cc:276] 0709 13:05:50.612730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.612888 (+   158us) service_pool.cc:224] Handling call
0709 13:05:50.613233 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.613436 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:51.120282 (+506846us) log.cc:844] Serialized 14 byte log entry
0709 13:05:51.630499 (+510217us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:51.630583 (+    84us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:51.630650 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.630749 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.630907 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14877,"prepare.queue_time_us":111291,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":42455,"raft.queue_time_us":841,"thread_start_us":965871,"threads_started":2}
I20260709 13:05:51.614789  1590 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:51.639261  1618 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.807s	user 0.024s	sys 0.050s
I20260709 13:05:51.634460  1554 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:51.647342 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 355315us
I20260709 13:05:51.639693  1452 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.665198 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:51.666138 29145 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:51.668081 29146 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:51.625921  1648 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.621938  1694 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.672911 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:51.677628 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:51.658318  1618 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.828s	user 0.024s	sys 0.050s
I20260709 13:05:51.673036  1675 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.692715  1675 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.698201  1652 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.673585  1703 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:51.670311  1594 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:51.684710 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.685842 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.703586 29140 raft_consensus.cc:2410] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:05:51.609668  1600 rpcz_store.cc:276] 0709 13:05:48.589537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.593191 (+  3654us) service_pool.cc:224] Handling call
0709 13:05:48.878233 (+285042us) ts_tablet_manager.cc:934] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
0709 13:05:50.518257 (+1640024us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:05:51.205208 (+686951us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:51.205350 (+   142us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:05:51.348654 (+143304us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:05:51.387348 (+ 38694us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:51.455985 (+ 68637us) ts_tablet_manager.cc:989] Replica registered.
0709 13:05:51.465206 (+  9221us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":35602,"tablet-copy.queue_time_us":215782,"thread_start_us":240416,"threads_started":2}
I20260709 13:05:51.690780  1648 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.645078  1557 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:51.718614 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.722329 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:51.724247 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 258960us
I20260709 13:05:51.725531 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:51.726538 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.700546  1209 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.699350  1343 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:51.731549 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.735137 29215 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:51.736781 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.741360 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:51.741437 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.742108 29069 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:51.742384 29213 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:51.742903 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:51.721367  1537 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.744798 29214 raft_consensus.cc:2410] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:05:51.750183 29145 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:05:51.750455 29066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "082c896dd04e451198eba8392e70fba7"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:39110
I20260709 13:05:51.751194 29066 raft_consensus.cc:606] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:51.750867 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:51.751830 29068 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
W20260709 13:05:51.752785  1549 consensus_meta.cc:287] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.636s	user 0.011s	sys 0.000s
I20260709 13:05:51.753129  1549 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.754115 29222 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:51.759142  1530 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.733176  1590 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:51.745026 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:51.766665  1554 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.767727 29066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39110
I20260709 13:05:51.768601 29066 raft_consensus.cc:606] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:51.769028 29066 raft_consensus.cc:614] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:51.770484  1549 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:51.772275  1600 tablet_copy_client.cc:806] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:51.773346  1600 tablet_copy_client.cc:670] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:51.774822 29069 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:51.776597 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:51.777556 29070 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:51.726962  1557 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.780964 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2500) took 1654 ms. Trace:
I20260709 13:05:51.783069  1557 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:51.785153  1666 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.787873  1549 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:51.740077  1343 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.788704  1549 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:51.789636  1676 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:05:51.740383  1618 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:51.791203  1618 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:51.799268  1618 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:51.799777  1640 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 1.970s	user 0.000s	sys 0.014s
I20260709 13:05:51.801124  1694 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:51.802260  1705 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:05:51.804821 29213 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:51.712783  1703 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.808081 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:51.808609 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.809022  1703 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.809795 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:51.811116 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2316) took 1785 ms. Trace:
I20260709 13:05:51.811270 29215 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:05:51.811523 29139 rpcz_store.cc:276] 0709 13:05:50.025828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.025942 (+   114us) service_pool.cc:224] Handling call
0709 13:05:50.043533 (+ 17591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:50.052664 (+  9131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.810949 (+1758285us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b54002fab50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:51.811064 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13927680}
I20260709 13:05:51.719944  1594 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.815264  1673 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:51.815709  1673 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:51.816795  1673 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.817852  1673 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:51.820621 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.821873 29215 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:51.781575 29219 rpcz_store.cc:276] 0709 13:05:50.126592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.141073 (+ 14481us) service_pool.cc:224] Handling call
0709 13:05:50.141375 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.152463 (+ 11088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.152499 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.159068 (+  6569us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.367551 (+1208483us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b180017e420. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:05:51.645900 (+278349us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:51.697110 (+ 51210us) log.cc:844] Serialized 14 byte log entry
0709 13:05:51.697385 (+   275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.697502 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.780667 (+ 83165us) raft_consensus.cc:1685] finished
0709 13:05:51.780766 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:51.780898 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11443,"prepare.queue_time_us":50922,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":82894,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":16229,"spinlock_wait_cycles":2479360,"thread_start_us":278104,"threads_started":1,"wal-append.queue_time_us":1521609}
I20260709 13:05:51.822937 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.823858 29215 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:51.824301 29066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:39110
I20260709 13:05:51.825026 29066 raft_consensus.cc:606] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:51.825428 29066 raft_consensus.cc:614] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:51.834528 29215 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:51.835222 29113 connection.cc:256] RPC call timeout handler was delayed by -1.079641545999948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 13:05:51.835613 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.836632 29215 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 13: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:05:51.835652 29113 connection.cc:256] RPC call timeout handler was delayed by -1.0760415459999422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1507s for a response
I20260709 13:05:51.847550 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.837662 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.848582 29139 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:51.841382 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:51.850597 29214 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:51.850529 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.848582 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:51.856663 29066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39110
I20260709 13:05:51.861758 29066 raft_consensus.cc:606] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:51.862237 29066 raft_consensus.cc:614] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:51.859388 29215 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:51.828939  1549 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.840081  1336 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.866545 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.832069  1557 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:51.867206 29139 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:51.831127  1551 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:51.869982 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.865918  1364 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:51.867063 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:51.850085  1694 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:51.878813 29215 raft_consensus.cc:2410] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:05:51.878690 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:51.879563  1557 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:51.886125 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:51.883296  1694 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:51.894546 29221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2490) took 1794 ms (client timeout 1737 ms). Trace:
I20260709 13:05:51.894658 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:51.895041 29221 rpcz_store.cc:269] 0709 13:05:50.099974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.100140 (+   166us) service_pool.cc:224] Handling call
0709 13:05:51.894364 (+1794224us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54000799d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:51.894478 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606295424}
I20260709 13:05:51.896008 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.898429 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:51.899339 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.899955 29143 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:51.900733 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:05:51.904807 29111 leader_election.cc:336] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.021s (SENT)
I20260709 13:05:51.906126 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:51.895641  1694 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.908377 29143 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:51.909409 29146 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:51.908787 29214 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:51.908033 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:51.910769 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.880913  1364 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:51.884132  1099 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:51.916285 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:51.914979 29113 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:51.919111 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.922595 29141 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:51.924435  1364 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:51.939374  1699 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.515s	user 0.001s	sys 0.002s
I20260709 13:05:51.939500  1594 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:51.878280  1195 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:51.967576  1721 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:05:51.940524  1718 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:05:52.058331 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.038980 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2538) took 1337 ms. Trace:
W20260709 13:05:52.048832 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 142773us
I20260709 13:05:51.972227 29066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:39110
I20260709 13:05:52.061657 29066 raft_consensus.cc:606] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:52.062281 29066 raft_consensus.cc:614] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:52.027192 29140 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:52.020380  1674 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:52.060709 29148 rpcz_store.cc:276] 0709 13:05:50.701499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.701696 (+   197us) service_pool.cc:224] Handling call
0709 13:05:50.715101 (+ 13405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:50.715455 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:50.715483 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:50.715988 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:05:51.133177 (+417189us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:51.133229 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:51.133304 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:51.938722 (+805418us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:05:52.038509 (+ 99787us) raft_consensus.cc:1685] finished
0709 13:05:52.038548 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:52.038893 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":559,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":99,"spinlock_wait_cycles":4033280,"thread_start_us":416783,"threads_started":1,"wal-append.queue_time_us":649570}
I20260709 13:05:52.086572 29071 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:51.981495 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.929567  1694 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:52.096047  1694 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:52.097775 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:52.098373  1679 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:05:52.097004  1694 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.099050  1682 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:52.087338  1343 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:51.920948  1577 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:52.101513  1577 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:52.101933  1577 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:51.917536  1235 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:51.876200  1336 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.109760  1661 raft_consensus.cc:3055] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:52.110193  1661 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:51.885151  1551 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.025698 29139 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:52.048913 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:51.966678  1709 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:05:52.058398 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 50021.3us
W20260709 13:05:52.113662 29064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2299) took 2820 ms (client timeout 1995 ms). Trace:
I20260709 13:05:52.113787 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:52.058961 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:51.928584  1639 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:52.063747 29213 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:52.112458  1661 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:52.064044 29140 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:52.064350 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:52.115902  1661 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:52.113943 29064 rpcz_store.cc:269] 0709 13:05:49.286995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.288014 (+  1019us) service_pool.cc:224] Handling call
0709 13:05:52.107346 (+2819332us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b54002085d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:52.107490 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2112332032}
I20260709 13:05:52.063599 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:52.061093 29143 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:05:52.119223  1590 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:52.119767  1590 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.065109  1547 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:52.067808  1530 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.122818 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:52.123741 29216 raft_consensus.cc:2376] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12. Re-sending same reply.
I20260709 13:05:52.041623  1673 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:52.124748  1673 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.073388 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:52.126785  1577 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.128351 29219 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:52.128919 29141 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:52.007368  1666 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:52.129671  1666 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.130168  1577 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.053086  1557 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:52.053401  1554 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.033984  1663 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:52.081550 29871 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:52.063555  1491 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 13:05:52.133258 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 1.57777e+06us
I20260709 13:05:52.089983 29148 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:52.094775 29220 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:52.134275 29148 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:52.134688 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 800) took 1831 ms. Trace:
I20260709 13:05:52.099453  1682 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:52.135015  1666 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.100337  1552 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:52.135097 29148 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:52.092818  1645 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 2.259s	user 0.006s	sys 0.009s
I20260709 13:05:52.135030 29220 rpcz_store.cc:276] 0709 13:05:50.303535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.094272 (+1790737us) service_pool.cc:224] Handling call
0709 13:05:52.094602 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:52.134436 (+ 39834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:52.134634 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:52.135850  1552 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:52.136353 29148 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:52.127580  1673 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.137157  1552 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:52.137693 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 74167.6us
I20260709 13:05:52.138240  1552 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:51.924322  1099 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.106007 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:52.139631  1099 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:52.112979  1336 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.140379  1336 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:52.135681  1682 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.141481  1336 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:52.141777  1682 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:52.142949  1336 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:52.143327  1336 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:52.015174  1364 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.145642  1364 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.148090 29145 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:52.148542 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.149844 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.132977  1663 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.100572  1623 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:52.150589 29140 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:52.150676  1623 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:52.146652  1336 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.136129  1575 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:52.106125  1674 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:52.106149  1629 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.152297  1663 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.151994  1623 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.152778  1629 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.153700  1623 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:52.154933  1629 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:52.105978 29113 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:52.155735 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 49849.7us
I20260709 13:05:52.157495  1491 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:52.157907  1491 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:52.152155  1575 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.158339  1491 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:52.158643  1575 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:52.157910 29113 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:52.160459  1674 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:52.160880  1674 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:52.166723 29145 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:52.169147  1364 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:52.130275  1730 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:52.169881  1674 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.177515  1575 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.179569 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:52.169935  1364 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:05:52.180251 29220 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:52.180733  1364 consensus_queue.cc:1243] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:52.181223  1730 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:52.181833  1730 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:52.183073  1730 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.184891 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:52.185806 29219 raft_consensus.cc:2368] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
I20260709 13:05:52.186295 29148 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:05:52.188969 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.189570 29148 pending_rounds.cc:85] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 10
I20260709 13:05:52.189659 29146 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:05:52.189980 29148 pending_rounds.cc:107] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:05:52.190251 29148 raft_consensus.cc:2889] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:52.187439 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.193277 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.194305 29139 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:05:52.194365 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.194806  1336 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:52.186730  1099 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.191779 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.195586 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.196853 29143 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:52.197593 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.191653  1491 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.193059 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.202033  1099 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:52.207115  1491 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:52.208801  1422 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:52.209826  1590 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.217232  1195 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.220738  1551 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.121810  1235 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:52.221990  1235 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:52.224048 29874 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.27.71.190:37537, user_credentials={real_user=slave}} blocked reactor thread for 100294us
I20260709 13:05:52.226635  1557 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.230319 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 124364us
I20260709 13:05:52.231141 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:52.237210 29936 heartbeater.cc:646] Failed to heartbeat to 127.27.71.190:37537 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.27.71.190:37537 timed out after 15.000s (SENT)
I20260709 13:05:52.238838  1554 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:52.239280  1554 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.239660  1554 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:52.223181  1235 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.244423 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:52.264624 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.267652 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.268932 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.271366 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:52.287511 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:52.288406 29144 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:52.293264 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:52.296861 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.203586 29220 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:52.308185 29905 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:52.204909  1655 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.310308 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 78114.3us
I20260709 13:05:52.319887 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.312857 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:52.323606 29905 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:52.332962 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.333645 29070 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:52.334157 29907 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
W20260709 13:05:52.334472 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.335207 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.336273 29070 raft_consensus.cc:2410] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:05:52.336387 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:52.240123  1661 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:52.339255 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.340806 29070 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:52.341647 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:52.342142 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 115779us
I20260709 13:05:52.345893  1674 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.347841 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:52.349331 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.350234 29065 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:52.331610 29144 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:52.354135 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:52.268440  1195 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:52.359642 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.361745 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.364917 29066 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:52.364836 29903 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:52.362242 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.368254 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:52.368958 29070 raft_consensus.cc:2376] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12. Re-sending same reply.
W20260709 13:05:52.369625 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.370129 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 208931us
W20260709 13:05:52.370625 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 47951.7us
I20260709 13:05:52.371001 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:52.371651 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:52.372797 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:52.377105 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:52.377142 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:52.378574 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:52.370831 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:52.379462 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:52.379961 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:52.380303 29905 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:52.380570 29219 raft_consensus.cc:2410] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:05:52.340549  1661 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.338984  1639 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:52.383211 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.241544  1557 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.384763 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.394315 29070 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:52.395352 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:52.386345 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:52.396435 29072 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:52.392513 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:52.400852 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:52.401474 29222 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:52.401595 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:52.402511 29215 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:52.403085 29068 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:05:52.401248 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:52.334394  1629 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:05:52.387713 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.412891 29908 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:52.338316  1596 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:52.389474 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:05:52.415366 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.415280 29073 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:05:52.381304 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:52.372504  1551 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:52.408311 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:52.408455 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 931829us
I20260709 13:05:52.386423 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:52.410980 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:52.418007 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:52.357056  1679 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:05:52.387588  1653 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:52.387717  1099 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:52.435990 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.387813  1682 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:52.392030  1661 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:52.424638 29064 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:52.421784 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:52.389772  1674 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:52.426007 29216 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:52.427767 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 306463us
I20260709 13:05:52.383332  1655 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.400453  1590 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:52.429342  1551 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.392702  1717 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:52.389382  1623 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:05:52.382069  1594 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:05:52.376791  1195 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.444264 29219 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:52.439105  1673 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:52.428262  1403 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:52.430647  1663 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:52.466326 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:52.504329 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:52.466586 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:52.467092  1578 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:52.474853 29186 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:52.480441 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:52.505453 29219 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:52.487903  1639 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.436734  1653 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.489178 29905 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:52.489552  1674 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:52.491493 29070 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:52.427029  1596 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:52.496977 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.497357 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.418588 29072 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:52.439760  1682 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.510468 29903 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:52.492398  1551 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:52.493450  1235 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:52.592186 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.599936 29906 raft_consensus.cc:2410] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:05:52.592561 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.511010  1491 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:52.504184  1663 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.515182  1653 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:52.633533 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 272802us
I20260709 13:05:52.635934 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:52.636819 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.586328  1673 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:52.595510  1682 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:52.653877 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 252671us
I20260709 13:05:52.507277  1736 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.138s
I20260709 13:05:52.663252 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.509227  1403 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.613842  1491 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:52.652976 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.638496  1663 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:52.681092 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 276066us
I20260709 13:05:52.681903 29113 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:52.651770  1594 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:52.673086  1629 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:52.647822 29216 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
W20260709 13:05:52.687027 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 271093us
I20260709 13:05:52.666517  1235 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:52.681349  1703 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:52.682456  1491 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:52.692400 29112 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:52.708518 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:52.717751 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 62915.6us
I20260709 13:05:52.736270 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:52.691488  1594 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:52.690814  1235 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:52.706322  1557 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.748080 29065 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:52.636510  1733 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:05:52.762329 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.763777 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.719720  1717 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.713111  1703 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:52.767872 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.764060 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 289261us
I20260709 13:05:52.768059 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.696648  1629 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.752875  1235 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:52.771529 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:52.772341 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:52.690227  1674 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.797950 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.799700 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.800997 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.801859 29065 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:05:52.788828 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 152991us
W20260709 13:05:52.776569 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.809604 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 343072us
I20260709 13:05:52.809607 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:52.761662  1578 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:05:52.822261 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 135453us
I20260709 13:05:52.770018  1732 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:52.740607  1673 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.840577 29219 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:05:52.795169  1655 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.790519  1746 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:52.864660  1594 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.876088  1748 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:52.875837  1746 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.878193  1746 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:52.843187 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.826323  1195 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.878728  1746 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:52.839886 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:52.853006 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 142469us
I20260709 13:05:52.794157  1731 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.856926  1738 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:05:52.823273  1557 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:52.860667 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:52.862731  1554 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.623s	user 0.008s	sys 0.001s
I20260709 13:05:52.864622  1364 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:52.869263  1629 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:52.869962 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:52.870817  1539 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:52.814489  1741 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:52.871635  1491 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.872334  1751 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.872587 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.953088 29070 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:52.875203  1680 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.813376 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:52.848331 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 113962us
I20260709 13:05:52.925134  1403 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:52.897086 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:52.867750  1730 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:52.857672  1737 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 13:05:52.964128 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 103603us
I20260709 13:05:52.965431 29113 leader_election.cc:400] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
I20260709 13:05:52.965905 29113 leader_election.cc:403] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:05:52.775949  1717 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.715761  1596 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.857672  1099 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:52.945919  1539 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:52.932619 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.936609 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:52.935609 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 95786.1us
W20260709 13:05:52.940927 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.897818  1693 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.900920  1752 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:52.954473 29065 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:52.893667 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.955184 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:52.967558 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 492249us
I20260709 13:05:52.980746 29072 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:52.961452 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.966856 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:52.902209  1746 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.948698  1756 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:52.970386 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 73365.2us
I20260709 13:05:52.935062  1364 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:52.987212 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:52.988189 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:52.985599 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:52.983323 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:52.989454 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:52.993746 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:52.995807 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 223528us
I20260709 13:05:52.996307 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:52.996914 29219 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:05:52.934662  1554 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.998567 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:52.999209 29219 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:05:53.000916 29219 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:05:52.999145 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.934724  1557 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.929569  1731 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:53.006023 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:52.981338  1741 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
I20260709 13:05:53.007575 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:53.008509 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:52.906406  1703 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:52.975111  1403 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:52.982679  1752 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:52.975035  1734 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:53.015004 29068 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:52.984539  1740 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:53.016394 29220 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:53.016603 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:53.003890 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:53.028558 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:53.034403  1767 raft_consensus.cc:2764] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:05:52.940281  1578 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.039618 29065 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:52.960028  1680 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.042856 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2331) took 1300 ms. Trace:
I20260709 13:05:52.998487  1629 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:53.060395 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:53.062500 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 81233.6us
W20260709 13:05:53.053627  1336 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.858s	user 0.011s	sys 0.003s
I20260709 13:05:53.028733  1742 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:53.072429 29071 rpcz_store.cc:276] 0709 13:05:51.738157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.738381 (+   224us) service_pool.cc:224] Handling call
0709 13:05:51.738785 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:52.087075 (+348290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.037928 (+950853us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b54000695d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:53.038125 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799390592}
I20260709 13:05:53.009059  1749 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:53.097839 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 111224us
I20260709 13:05:52.956072  1751 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.149109 29140 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:53.040392  1752 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:53.064050  1590 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.999789  1756 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:53.069741  1539 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.059855  1739 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:53.076851 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:53.028442  1530 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:53.173135 29216 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:53.136648 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 132897us
I20260709 13:05:53.076150  1629 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:53.151160 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:53.056170  1600 tablet_copy_client.cc:538] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:53.156746 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:53.070142  1364 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:52.986673  1693 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:05:53.171667 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 94914.8us
I20260709 13:05:53.140046  1403 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:05:53.151235  1336 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.137343  1750 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.190832 29071 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:53.192080 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:53.143217  1639 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:53.197094 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:53.197788  1629 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:53.205559 29214 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:53.194235  1661 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.729s	user 0.013s	sys 0.000s
I20260709 13:05:53.209950  1661 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.211738 29072 raft_consensus.cc:2368] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:53.143162  1580 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:53.180802  1663 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:53.204510 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 249363us
I20260709 13:05:53.256386 29220 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:53.167254  1554 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.269100 29216 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:53.256254 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:53.156795  1742 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:53.257671 29071 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:53.271311 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:53.272121 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:53.277678 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 80763.8us
W20260709 13:05:53.273465 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 81549.9us
I20260709 13:05:53.283571 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:53.167088  1680 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:53.166290  1763 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:05:53.289698 29071 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:53.250483  1539 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:53.250392  1580 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:53.258191  1674 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.305032 29218 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:05:53.305296  1701 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.305681  1693 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:53.307415 29218 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:53.308442  1735 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.134s
I20260709 13:05:53.309527  1760 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:53.309749  1776 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:53.310891  1760 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.311728  1580 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:53.313019  1758 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 13:05:53.314178  1754 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:53.315153 29066 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:53.314955  1754 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.316931 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2356) took 1123 ms. Trace:
I20260709 13:05:53.317416 29140 rpcz_store.cc:276] 0709 13:05:52.193716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.194121 (+   405us) service_pool.cc:224] Handling call
0709 13:05:53.316680 (+1122559us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b540029ccd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:53.316803 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2193547008}
I20260709 13:05:53.323012 29037 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:53.192726  1530 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.280514  1594 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.316639 29066 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:53.285774 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:53.297870 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:53.194156  1590 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:53.412958 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.333498  1734 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:53.431722 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 146032us
I20260709 13:05:53.311429  1760 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:53.321142  1680 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:53.354262 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:53.251506  1750 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.328213  1674 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:53.441035 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 86915.3us
I20260709 13:05:53.374696  1786 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:53.425912 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 128130us
I20260709 13:05:53.251295  1336 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.422298  1746 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.301918  1491 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:53.504521  1757 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:53.505178  1757 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.505612  1757 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:53.509217  1747 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:53.509502  1754 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.433521  1742 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:53.447360 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:53.512328  1590 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:53.512914 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 65654us
I20260709 13:05:53.513064  1590 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:53.444581  1661 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.514281  1751 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:53.515089 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:53.516610 29037 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:53.324626  1639 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.447757 29217 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:53.449096 29070 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:53.450464 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 170813us
I20260709 13:05:53.522846 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:53.361217  1701 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:53.481338  1784 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:05:53.446843  1554 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:53.481797  1779 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:53.482892  1772 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:05:53.483820 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 780) took 1576 ms. Trace:
I20260709 13:05:53.484532  1717 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:53.484643 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2447) took 1188 ms. Trace:
I20260709 13:05:53.486747  1760 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:53.477737  1680 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:53.488945  1336 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:53.491710  1235 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.002s
I20260709 13:05:53.500310  1655 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:53.500789  1770 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 13:05:53.445813  1675 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.432991  1557 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.479938 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:53.433941 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:05:53.524943 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.528760  1786 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.541817 29141 rpcz_store.cc:276] 0709 13:05:51.907287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.925037 (+ 17750us) service_pool.cc:224] Handling call
0709 13:05:52.006583 (+ 81546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:52.129326 (+122743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.483581 (+1354255us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400191754. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:53.483754 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280882432}
I20260709 13:05:53.541983 29064 rpcz_store.cc:276] 0709 13:05:52.296413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.296593 (+   180us) service_pool.cc:224] Handling call
0709 13:05:53.484457 (+1187864us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54002ad354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:53.484588 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291417344}
I20260709 13:05:53.516208  1746 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:53.536861  1551 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.921s	user 0.013s	sys 0.000s
I20260709 13:05:53.560031 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:53.500844  1618 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:53.614514 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 180354us
I20260709 13:05:53.560164  1731 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:53.540342  1735 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 13:05:53.530176  1661 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:53.630533 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 150741us
I20260709 13:05:53.561005  1701 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:53.636569 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 202665us
I20260709 13:05:53.656888 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:53.536145  1639 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:53.634294  1605 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:53.619817 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:53.658999  1746 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.663743 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:53.700595 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59735.8us
I20260709 13:05:53.700104  1557 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.716391  1701 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:53.737516 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:53.701638  1735 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:53.715988  1680 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.645782  1551 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:53.745059 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 81413.4us
I20260709 13:05:53.746515 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:53.713382  1674 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.720980 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 780) took 1224 ms. Trace:
I20260709 13:05:53.748714 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:53.749322 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:53.750890 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:53.725355  1530 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.700248  1783 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:05:53.728787  1785 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:05:53.733170  1750 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
W20260709 13:05:53.761583 29907 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 818) took 1494 ms (client timeout 1943 ms). Trace:
W20260709 13:05:53.762135 29907 rpcz_store.cc:269] 0709 13:05:52.266777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.268629 (+  1852us) service_pool.cc:224] Handling call
0709 13:05:53.761420 (+1492791us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b54007d0354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:53.761521 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148115328}
I20260709 13:05:53.763154  1574 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:53.763461  1751 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:53.764015  1751 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:53.764492  1751 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:53.766979  1682 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.095s	user 0.012s	sys 0.000s
I20260709 13:05:53.767616  1756 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:53.747757 29911 rpcz_store.cc:276] 0709 13:05:52.496260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.496426 (+   166us) service_pool.cc:224] Handling call
0709 13:05:52.501314 (+  4888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:52.894121 (+392807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:52.894152 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:52.899583 (+  5431us) log.cc:844] Serialized 26 byte log entry
0709 13:05:53.051061 (+151478us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:53.051115 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.051218 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:53.720685 (+669467us) raft_consensus.cc:1685] finished
0709 13:05:53.720714 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.720908 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69198,"raft.queue_time_us":230299,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":5451,"thread_start_us":410391,"threads_started":2,"wal-append.queue_time_us":336993}
W20260709 13:05:53.734951 29145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2350) took 1841 ms (client timeout 1866 ms). Trace:
I20260709 13:05:53.621203  1235 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:53.769373 29145 rpcz_store.cc:269] 0709 13:05:51.893830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.898198 (+  4368us) service_pool.cc:224] Handling call
0709 13:05:53.734734 (+1836536us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54001bc6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:53.734874 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":572872576}
I20260709 13:05:53.623548  1655 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.661144 29070 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
W20260709 13:05:53.644789  1730 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.673s	user 0.009s	sys 0.004s
I20260709 13:05:53.753037 29905 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:05:53.741488  1653 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.088s	user 0.014s	sys 0.001s
W20260709 13:05:53.739627  1752 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260709 13:05:53.747637  1775 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:53.767481  1682 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.753247 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:53.754124 29217 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 10: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:53.660130  1663 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:53.742708  1735 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:53.754709 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:53.775046  1655 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.752130  1403 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:05:53.744513  1746 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.318656 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:53.733947 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 662) took 1064 ms. Trace:
I20260709 13:05:53.801527 29068 rpcz_store.cc:276] 0709 13:05:52.669444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.670288 (+   844us) service_pool.cc:224] Handling call
0709 13:05:52.670599 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.057414 (+386815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.733572 (+676158us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400187c54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:53.733857 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791396224}
I20260709 13:05:53.663084  1774 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.257s
I20260709 13:05:53.803064 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:53.769754 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:53.649524  1618 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.772245 29073 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:05:53.770874 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:53.806849 29064 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:53.807842  1618 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.805459 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:53.774989  1773 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:53.776798 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:53.649178  1794 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:53.754247  1605 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:53.781076 29071 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:53.580770  1779 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.793177 29185 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:53.745776  1763 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:05:53.756608  1701 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.745023  1557 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.782335  1653 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.798597 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:53.803727 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 280948us
I20260709 13:05:53.804899 29213 raft_consensus.cc:2368] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:05:53.636915  1765 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.161s
I20260709 13:05:53.795377  1539 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:53.811106  1731 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.809465 29068 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:53.812834 29145 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:53.813197 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:53.869439 29144 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:53.795197  1775 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:53.770961  1578 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:53.832371 29140 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:05:53.879583 29140 pending_rounds.cc:85] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 7
I20260709 13:05:53.879908 29140 pending_rounds.cc:107] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:05:53.880174 29140 raft_consensus.cc:2889] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:53.825795 29142 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:53.828380 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:53.844549 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:53.742954  1580 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.845955  1552 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:53.834899  1663 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:53.904774 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 76465.9us
I20260709 13:05:53.825531  1787 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:53.849479 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2550) took 1703 ms. Trace:
I20260709 13:05:53.911964 29143 rpcz_store.cc:276] 0709 13:05:52.146216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.148282 (+  2066us) service_pool.cc:224] Handling call
0709 13:05:53.849208 (+1700926us) spinlock_profiling.cc:243] Waited 47.4 ms on lock 0x7b540001c554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:53.849398 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108936064}
I20260709 13:05:53.913461 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:53.914208 29143 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:53.782612  1752 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:53.813938  1745 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:53.894809 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.901927  1552 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:53.897863 29113 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 2.087s (SENT) [suppressed 14 similar messages]
I20260709 13:05:53.638106  1734 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.905975 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:53.954025 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48123.9us
I20260709 13:05:53.814110  1730 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.906673 29220 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 13: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:53.842792  1557 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.917688 29217 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:53.837787  1790 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:53.935559 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:53.985643 29905 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:53.936199 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:53.937268 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:53.879338  1578 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:53.810869  1773 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:53.982748 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:53.909649  1548 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:53.910939  1539 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.996351 29907 raft_consensus.cc:2368] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
I20260709 13:05:53.996404 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:53.981395 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 163114us
W20260709 13:05:53.953583  1629 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.748s	user 0.010s	sys 0.005s
I20260709 13:05:53.910694  1530 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:54.001897 29215 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:53.889103  1731 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.011336 29222 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:53.984726  1680 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:54.015406 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2341) took 1305 ms. Trace:
I20260709 13:05:54.002228  1786 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:53.999907  1578 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:54.049633 29148 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:54.045790 29072 rpcz_store.cc:276] 0709 13:05:52.709546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.987953 (+278407us) service_pool.cc:224] Handling call
0709 13:05:54.015099 (+1027146us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54000681d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:54.015279 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512236288}
I20260709 13:05:53.952889  1774 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:05:53.954250  1663 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 13:05:54.062484 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:54.001922  1790 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:53.933866  1787 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:53.905478  1735 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:53.953598  1745 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.044297  1551 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:54.060870  1793 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:05:53.995636  1730 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.072525 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:54.071318 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:54.083444 29903 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:54.050084  1639 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.085368 29146 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:54.001735  1802 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.121s
I20260709 13:05:54.067781  1235 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.057096  1552 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.097702  1552 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:54.023945  1629 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.097922  1790 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.045301  1530 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.076704  1736 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:54.115640  1736 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.044018  1775 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.118897  1736 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:54.120494  1803 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:05:54.121582 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.121838  1629 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.099236  1682 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:54.122684 29067 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 12. Current term is 13. Ops: []
W20260709 13:05:54.081179  1763 consensus_queue.cc:1175] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" responder_term: 16 status { last_received { term: 15 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:54.095490  1773 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.125638  1773 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:54.122040  1757 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.002s
I20260709 13:05:54.129429  1757 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.133898  1530 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.095100  1787 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.136065  1757 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.136210  1653 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:54.137291  1580 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:54.139612  1745 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.085057  1780 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.151516 29143 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
W20260709 13:05:54.154299 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 256462us
I20260709 13:05:54.158990  1605 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:54.079442  1788 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:05:54.165356  1542 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:54.165884  1542 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:05:54.160427 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 153834us
I20260709 13:05:54.169174 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:54.091899  1751 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.173143 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.174089 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.175488 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:54.176311 29064 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.179169 29903 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:54.179735 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.182821 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:54.183568 29069 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:54.145542  1746 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.184835 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.175091  1745 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:54.186908 29072 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:54.188800 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.192195  1751 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:54.173436  1780 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.194828  1780 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.195623  1731 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.196069  1752 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.196108  1745 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.197172  1745 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:54.197477  1730 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.197582 29069 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:54.167016  1542 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.201324 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:54.203812 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.204845 29143 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:05:54.207619 29143 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:54.215677 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.216856  1542 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:54.218775 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.225314 29069 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:54.195668  1779 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.188364  1600 tablet_bootstrap.cc:492] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:05:54.230592 29072 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:54.208717  1757 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.240370 29064 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:05:54.187332  1810 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:05:54.241407 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.248473 29069 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:54.247651 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 826) took 1091 ms. Trace:
I20260709 13:05:54.230556  1578 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.243115  1731 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:54.249753 29218 rpcz_store.cc:276] 0709 13:05:53.155872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.822799 (+666927us) service_pool.cc:224] Handling call
0709 13:05:53.847546 (+ 24747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.848061 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:53.904457 (+ 56396us) log.cc:844] Serialized 14 byte log entry
0709 13:05:54.247270 (+342813us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:54.247336 (+    66us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:54.247393 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.247470 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:54.247590 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":55934,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":5100,"thread_start_us":339535,"threads_started":1}
I20260709 13:05:54.260552  1779 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:54.268632  1779 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:54.260977  1639 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:05:54.270406  1639 consensus_queue.cc:1243] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:05:54.271231  1731 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:54.244393  1790 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.242600  1757 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.315096  1776 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:54.316736  1776 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:05:54.287385  1769 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.104s	user 0.004s	sys 0.014s
I20260709 13:05:54.287976  1694 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.292999  1790 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.281244  1578 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.261133  1745 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.250406  1730 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:05:54.300666 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2559) took 1116 ms. Trace:
W20260709 13:05:54.303357  1796 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.693s	user 0.001s	sys 0.006s
I20260709 13:05:54.306330  1740 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:54.283762 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 672) took 1225 ms. Trace:
W20260709 13:05:54.282630 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.326653 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.312795 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:54.314417 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:54.315289 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2516) took 1321 ms. Trace:
I20260709 13:05:54.315775 29186 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:54.318712 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.319610 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:54.315781  1777 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.799s	user 0.002s	sys 0.008s
I20260709 13:05:54.323784 29217 rpcz_store.cc:276] 0709 13:05:53.184365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.849360 (+664995us) service_pool.cc:224] Handling call
0709 13:05:53.849906 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.958833 (+108927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.300401 (+341568us) spinlock_profiling.cc:243] Waited 67.5 ms on lock 0x7b5400283cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:54.300597 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155283840}
I20260709 13:05:54.329442 29066 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:54.326231 29070 rpcz_store.cc:276] 0709 13:05:53.058338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.060132 (+  1794us) service_pool.cc:224] Handling call
0709 13:05:54.283588 (+1223456us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b5400186d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:54.283705 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848395008}
I20260709 13:05:54.330098 29218 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:54.328404 29216 rpcz_store.cc:276] 0709 13:05:52.993990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.005754 (+ 11764us) service_pool.cc:224] Handling call
0709 13:05:54.315120 (+1309366us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54002f58d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:54.315231 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355632384}
I20260709 13:05:54.334252 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.335294 29217 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:54.335698 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.336194 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.339144 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:54.342044 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:54.344661  1775 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:54.345110  1775 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:54.345502  1775 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:54.345682  1790 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.317680  1776 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.346802  1776 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:54.347391 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.348284 29070 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:54.328591  1740 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:54.334553  1578 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:54.343395  1757 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.350736 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:54.351089 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.352947 29903 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:54.353760 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.354903 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.355744 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.351817 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.350112  1730 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:54.358067  1730 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:54.359766  1730 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:54.364223  1682 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:54.357030  1578 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:54.372846  1578 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.357327  1790 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:54.374786 29188 connection.cc:256] RPC call timeout handler was delayed by -1.0215843350006253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
I20260709 13:05:54.376355 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.377055 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.378526 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2343) took 1611 ms. Trace:
I20260709 13:05:54.389271 29143 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:54.379668 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.383193 29185 leader_election.cc:400] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 10 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11.
I20260709 13:05:54.383858 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 822) took 1239 ms. Trace:
I20260709 13:05:54.390581 29219 rpcz_store.cc:276] 0709 13:05:53.144730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.340053 (+195323us) service_pool.cc:224] Handling call
0709 13:05:53.340541 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.352315 (+ 11774us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:53.411448 (+ 59133us) log.cc:844] Serialized 14 byte log entry
0709 13:05:53.610762 (+199314us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:53.611534 (+   772us) log.cc:844] Serialized 26 byte log entry
0709 13:05:53.611728 (+   194us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:53.611778 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.611885 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:54.302955 (+691070us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b24001fa498. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78338a69c4 00007f78338a4acb 00007f7833a5fdc9 00007f7833a5fa55 00007f7833a5f09d 00007f783669110e 00007f78366a888c 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221
0709 13:05:54.383524 (+ 80569us) raft_consensus.cc:1685] finished
0709 13:05:54.383645 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:54.383795 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50862,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":4296,"raft.queue_time_us":744,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":8538,"spinlock_wait_cycles":33698048,"thread_start_us":128445,"threads_started":1,"wal-append.queue_time_us":169137}
I20260709 13:05:54.376258 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.385783 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:54.388813 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:54.358839  1745 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.390189 29148 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:54.392613 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.393016 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:54.398813 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.399397 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:54.400552 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:54.405580 29143 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:54.393092 29038 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:54.409543 29871 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:54.410429  1794 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:54.410694  1530 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:54.411019  1794 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:54.411188  1530 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.411551  1542 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.411763  1530 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.412117  1542 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.412132  1794 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.414358  1542 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:54.418394  1735 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.419152  1747 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.420318  1746 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:54.419656  1747 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.420653  1099 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:54.420847  1746 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:54.421262  1099 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:54.421924  1746 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.422678  1747 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.389321 29065 rpcz_store.cc:276] 0709 13:05:52.767147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.008219 (+241072us) service_pool.cc:224] Handling call
0709 13:05:54.378325 (+1370106us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b540012a054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:54.378467 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17038720}
I20260709 13:05:54.388401  1681 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:54.425357  1681 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.422597  1549 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:54.445317  1549 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:54.445783  1549 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:54.395570 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.447472 29141 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:54.429739 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:54.432770 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:54.436350 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:54.449842 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:54.439126  1666 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:54.453581 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.454520 29143 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
W20260709 13:05:54.454078 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 53149.8us
I20260709 13:05:54.440186 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.456559 29218 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:54.440593  1539 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:54.440872  1779 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.441424  1681 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.442502 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.411239  1364 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.444132  1551 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:54.443681 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.469182 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.468657 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:54.449843 29219 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:54.452106 29112 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:54.449321 29903 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:54.474609 29112 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:54.457871 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:54.459151 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:54.459867 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.396068  1673 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:54.396225  1552 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:54.478227 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.481142 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:54.481528 29218 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:54.481987 29910 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:54.447031  1757 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.494877  1730 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.490715  1673 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:54.511620  1673 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:54.512740  1673 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.515847  1653 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:54.516947 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.516686  1653 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.496069 29908 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:54.519134 29219 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:54.518573  1099 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.521174  1099 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:54.523022 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.498479  1575 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:54.524842 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 660) took 1857 ms. Trace:
I20260709 13:05:54.525285 29073 rpcz_store.cc:276] 0709 13:05:52.668657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.668812 (+   155us) service_pool.cc:224] Handling call
0709 13:05:52.669151 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.806602 (+1137451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.524571 (+717969us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400383954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:54.524780 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2536279808}
I20260709 13:05:54.524871 29112 leader_election.cc:400] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 9 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 9. Current term is 10.
I20260709 13:05:54.528576 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:54.491297  1748 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:54.457518  1805 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.530025  1805 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.500535 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2558) took 1328 ms. Trace:
I20260709 13:05:54.453274  1666 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:54.466598  1747 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:05:54.532781  1666 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.533136  1805 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.500177  1740 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.533875  1666 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:54.534705  1623 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:54.535140  1623 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:54.458214  1745 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.536104  1623 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.536182  1745 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.537313  1623 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:54.538669  1745 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.539985  1491 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:54.540445  1491 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:54.540818  1491 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:54.541304  1730 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.523751 29146 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:54.543149 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.544389 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.545140 29146 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.546016 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.546669 29143 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:54.546916  1775 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.526710 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.548120  1805 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:54.457738  1629 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:54.553207 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:54.528826 29213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2517) took 1534 ms (client timeout 2021 ms). Trace:
W20260709 13:05:54.554502 29213 rpcz_store.cc:269] 0709 13:05:52.994327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.995096 (+   769us) service_pool.cc:224] Handling call
0709 13:05:54.528620 (+1533524us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b54001329d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:54.528764 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716978560}
I20260709 13:05:54.532090 29220 rpcz_store.cc:276] 0709 13:05:53.171954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.271030 (+ 99076us) service_pool.cc:224] Handling call
0709 13:05:54.500264 (+1229234us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b54001333d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:54.500457 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1446733056}
I20260709 13:05:54.508442  1757 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.453326  1764 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.556811 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.557539 29141 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:54.557094 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:54.479810 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:54.549314 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.457240  1680 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:54.508872 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.565213 29144 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.509260  1530 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.467362  1539 consensus_queue.cc:1243] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:54.564000 29143 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:05:54.564250 29146 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:05:54.566702 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:54.479391  1551 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:54.568889 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 89216.1us
I20260709 13:05:54.569859 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:54.576637 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.576848 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:54.576951 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.578683 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:54.578653 29146 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:05:54.584028 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.586798 29144 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:05:54.586584 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:54.588047  1557 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:54.588543 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.589157 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.590106 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.590977 29144 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:54.591024 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.592020 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.594704 29144 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:54.595336 29146 raft_consensus.cc:2410] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 15 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:05:54.596494 29141 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:54.601097 29141 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:54.602065  1530 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:54.597276  1818 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:05:54.602851 29141 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:05:54.566118  1764 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.565234  1343 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:54.599150  1775 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.605157  1764 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.598466  1757 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:54.605201 29148 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:05:54.567788  1680 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:54.607642  1680 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:54.564844  1745 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:54.607833 29910 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:54.578351  1551 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:54.613169  1552 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:54.613773  1552 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:54.614269  1552 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.567664  1577 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:05:54.619268  1680 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.604291  1343 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.620641  1343 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:54.622026  1680 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.578145  1762 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:54.623842  1762 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:54.624264  1762 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:54.578142  1580 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.628454  1580 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.629283 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 819) took 2556 ms. Trace:
I20260709 13:05:54.629809 29214 rpcz_store.cc:276] 0709 13:05:52.072703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.073774 (+  1071us) service_pool.cc:224] Handling call
0709 13:05:52.649307 (+575533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.206050 (+556743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.627823 (+1421773us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b540026b6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:54.628997 (+  1174us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b540026b6d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:54.629175 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2601482880}
I20260709 13:05:54.630532  1580 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:54.568416  1629 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:54.632141  1629 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.638056  1762 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:54.640313 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44146.5us
I20260709 13:05:54.641516 29146 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:54.642143 29071 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:54.643229 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:54.644608 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.642932 29146 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: [12.11-15.12]
I20260709 13:05:54.647111 29065 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:05:54.645190 29073 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:54.649878 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.650470 29073 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:54.651665 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.652456 29073 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:54.654107 29073 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:54.654250 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.655090 29070 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:54.655748 29146 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:54.656987 29146 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:54.657136 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:54.677260 29071 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:05:54.677599 29071 pending_rounds.cc:85] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 10
I20260709 13:05:54.677814 29071 pending_rounds.cc:107] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 18.11
W20260709 13:05:54.657742 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 201899us
I20260709 13:05:54.655619 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:54.659588 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.665052 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:54.678627 29073 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.679030 29069 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:54.666788 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:54.678126 29071 raft_consensus.cc:2889] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:54.680387 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.677727 29146 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:54.682240 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.685220 29072 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:05:54.686714 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.687044 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:54.687377 29146 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:54.703949 29065 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:05:54.705576 29073 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:05:54.678436  1099 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.712049 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.638751  1551 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.701426  1762 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.725955  1739 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:54.728730 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.729580 29065 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:54.731470  1739 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:54.733381 29146 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:54.735932 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.717193  1099 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.741006 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.746553 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:54.748039 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:54.748077 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.750651 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.751194 29142 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:54.752449 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:54.753731 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:54.754706  1653 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:54.755195 29903 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:54.755582 29112 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:54.756188 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:54.756752 29909 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.758211 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.760106 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.761251 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:54.761540  1797 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.141s	user 0.003s	sys 0.011s
I20260709 13:05:54.761893 29213 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:54.762076 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:54.764045 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:54.764149 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.764608 29910 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:05:54.767414 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:54.767637 29906 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:54.768049 29906 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:54.769258  1539 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:54.768894 29906 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.769688  1539 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.770056 29906 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:54.771210 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.771855 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.773108 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.773689 29219 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:54.770120  1539 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:54.775578  1762 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:54.778965 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:54.780435 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.780998 29906 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:05:54.781453 29906 pending_rounds.cc:85] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 9
I20260709 13:05:54.781740 29906 pending_rounds.cc:107] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:05:54.782032 29906 raft_consensus.cc:2889] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:54.783586 29068 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:54.784437  1679 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:54.786593 29068 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:54.792604 29065 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:54.798934 29141 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:05:54.800192 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 663) took 2130 ms. Trace:
I20260709 13:05:54.800725 29067 rpcz_store.cc:276] 0709 13:05:52.669739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.165150 (+495411us) service_pool.cc:224] Handling call
0709 13:05:53.165540 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:54.123858 (+958318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.794519 (+670661us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b5400129654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:54.800137 (+  5618us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2200640256}
W20260709 13:05:54.801998 29215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 825) took 1646 ms (client timeout 2188 ms). Trace:
W20260709 13:05:54.802378 29215 rpcz_store.cc:269] 0709 13:05:53.155516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.271770 (+116254us) service_pool.cc:224] Handling call
0709 13:05:54.793135 (+1521365us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b5400355cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:54.801940 (+  8805us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1622687488}
I20260709 13:05:54.803237 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:54.803453 29221 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:54.804066 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.804936 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 49438.4us
I20260709 13:05:54.806102 29146 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:05:54.818143 29222 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:54.829970 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2367) took 1015 ms. Trace:
I20260709 13:05:54.830892 29147 rpcz_store.cc:276] 0709 13:05:53.814531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.814670 (+   139us) service_pool.cc:224] Handling call
0709 13:05:53.836750 (+ 22080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:54.107088 (+270338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.107111 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:54.107140 (+    29us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:54.107159 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.829653 (+722494us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400145854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:54.829716 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:54.829878 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621201280}
I20260709 13:05:54.844420 29216 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:05:54.844929 29140 pending_rounds.cc:85] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 6
I20260709 13:05:54.845256 29140 pending_rounds.cc:107] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:05:54.845549 29140 raft_consensus.cc:2889] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:54.849566 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:54.851438 29143 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:05:54.856057 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.857273 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:54.869372 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.876582 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.821683  1577 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:54.881860 29143 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:54.818928  1740 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:54.885990 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 61844.5us
I20260709 13:05:54.893029 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:54.893312 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.876152  1788 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:05:54.896828  1788 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:54.897281  1788 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:54.898260  1788 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.898939  1491 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:54.871512  1799 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:05:54.876919  1763 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:05:54.891397 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:54.872550 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 157726us
I20260709 13:05:54.889940 29070 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:54.899976 29145 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
W20260709 13:05:54.902386 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.886957 29143 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:54.905575 29143 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:54.907253 29143 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:54.910255 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:54.875618  1730 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.899703  1788 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.826407  1549 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.924690 29909 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:05:54.925153 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.926096 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.927558 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:54.952729 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59783.8us
I20260709 13:05:54.929136 29215 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:54.921335  1653 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.954787 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:54.960317 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:54.961304 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:54.962591 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.965817 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:54.873342  1753 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:05:54.970819 29874 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:54.914037  1794 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:54.909739  1731 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.909425  1735 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:54.925215  1740 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:54.971814 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 80492.7us
I20260709 13:05:54.985220 29068 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:05:54.970885 29219 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:54.974982 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:54.975773 29213 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:54.976382 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:54.971668 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:54.950265  1575 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:54.980049 29218 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:54.987322 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:55.007640 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:55.007792 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:55.004287 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:55.009647 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:54.952184  1577 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:54.975829  1801 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.357s	user 0.004s	sys 0.010s
I20260709 13:05:55.012921 29220 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:55.014883 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:54.976480  1741 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:54.976568  1746 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:55.019413 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2519) took 1599 ms. Trace:
I20260709 13:05:55.024847 29221 rpcz_store.cc:276] 0709 13:05:53.419636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.008651 (+589015us) service_pool.cc:224] Handling call
0709 13:05:54.803288 (+794637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:54.803800 (+   512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.019237 (+215437us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b5400355cd0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:55.019358 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1826609408}
I20260709 13:05:54.978924  1745 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:55.024183 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:54.979070  1775 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:54.925158  1730 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:54.979561  1794 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:55.025525 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:55.027537 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:54.982718  1735 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:54.976122  1680 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:05:55.011582  1551 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:55.021384  1694 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:55.012132  1727 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:05:55.021544  1757 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:54.978744  1751 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:05:55.028285  1574 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:05:55.023070  1605 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:55.025637  1752 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:55.021229  1741 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.978219  1731 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.016027  1824 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:55.016230  1789 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:54.976151  1653 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.053920  1673 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:55.028774  1745 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.030093  1775 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:55.011736  1805 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:55.012220  1740 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.031368  1530 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:55.026559  1364 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:05:55.074729 29141 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:55.076939  1757 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:55.028777  1763 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:05:55.085121  1730 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:55.086113  1549 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:55.067399  1639 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:05:55.092559  1833 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:05:55.053848  1694 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.048655  1735 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.206346  1735 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.034440  1753 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:05:55.077068  1605 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.208750  1605 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:55.094570  1741 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:55.049520  1343 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:55.125362 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.046392  1666 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:55.097416  1824 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.110241  1751 consensus_queue.cc:1243] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:55.098526  1764 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:55.111865  1745 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:55.114188  1775 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.050390 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:55.194945  1839 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:05:55.196607  1653 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:55.197681  1575 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:55.198156  1829 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
W20260709 13:05:55.198976  1828 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 13:05:55.202200  1837 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:05:55.124842  1530 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.112792  1805 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:55.228709 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:55.252482 29908 raft_consensus.cc:2368] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:05:55.254367 29112 leader_election.cc:400] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:05:55.254788 29112 leader_election.cc:403] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:05:55.231966 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 181658us
I20260709 13:05:55.185552  1639 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:55.213517  1623 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.676s	user 0.012s	sys 0.001s
I20260709 13:05:55.234266 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:55.235539 29214 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:55.256551 29909 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:55.188062  1491 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:55.212496  1694 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:55.231117 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 223463us
I20260709 13:05:55.219138  1666 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.093035  1577 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:55.284646 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:55.250818 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 239489us
I20260709 13:05:55.233067  1746 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:55.251106 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 223627us
W20260709 13:05:55.231570 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 234279us
I20260709 13:05:55.226459  1824 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:55.257045 29902 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:55.178022  1794 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.240713  1549 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:55.237287  1764 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:55.296892  1776 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.005s	sys 0.002s
I20260709 13:05:55.256590  1530 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:55.263041  1639 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:55.264806 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.256268  1805 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.270167  1666 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:55.301124 29186 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:55.296909 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.306797 29113 leader_election.cc:400] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:05:55.325301 29113 leader_election.cc:403] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:05:55.306777 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 299512us
I20260709 13:05:55.328157 29910 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:55.328295 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:55.297935 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:55.319824 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60089.4us
I20260709 13:05:55.330722 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:55.312573  1848 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:55.313067  1653 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:05:55.311131  1789 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:05:55.392463  1680 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:55.271108  1757 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:55.393599  1845 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:05:55.337587  1849 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:05:55.394477  1842 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
W20260709 13:05:55.395619  1844 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:05:55.336118  1549 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:05:55.346796  1776 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:55.396181 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 63424.6us
I20260709 13:05:55.432313 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:55.432060 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 103376us
I20260709 13:05:55.396706  1629 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.387248  1673 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.437544 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:55.341588  1855 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 13:05:55.404834  1680 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.440726 29185 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:55.456938 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:55.349766  1852 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 13:05:55.430809  1740 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:55.468822 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 142253us
W20260709 13:05:55.457319  1739 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.725s	user 0.011s	sys 0.002s
I20260709 13:05:55.458549  1680 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:55.455463  1343 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:55.454367  1859 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:05:55.479738 29111 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:55.405603  1746 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.443185  1629 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.447705  1673 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:55.497743 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 58797.2us
W20260709 13:05:55.481127  1552 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.866s	user 0.009s	sys 0.005s
W20260709 13:05:55.505381  1857 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:05:55.506280  1862 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:55.509820 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:55.510550 29142 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:55.512393 29144 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:05:55.514492 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:55.505445  1854 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:05:55.476358  1739 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.524555 29071 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:05:55.543741 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:55.500663  1810 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:55.499354  1574 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:55.544853 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:55.500389  1861 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.521751 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:55.550311 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.550709 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:55.551546 29072 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:55.552836 29072 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:55.554023 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:55.481171  1860 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:05:55.498539  1751 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:55.518858 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:55.558061 29068 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:55.523736 29066 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:55.546026 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:55.547066 29902 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:55.522779 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:55.541148 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:55.563411 29071 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
W20260709 13:05:55.568740 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 74429.6us
I20260709 13:05:55.571101 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:55.572275 29072 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:55.546589  1539 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.771s	user 0.009s	sys 0.003s
W20260709 13:05:55.573570 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 59181.2us
W20260709 13:05:55.574154 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 244447us
I20260709 13:05:55.559715  1861 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.576500  1539 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.576125 29911 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:55.587195 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:55.589234  1653 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:55.590757  1805 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.501963  1859 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:05:55.548712  1810 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:55.594916  1639 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.549182  1574 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.599457 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:55.609366 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:55.538681  1552 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.639640 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:55.620942  1788 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.002s
I20260709 13:05:55.649334 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:55.649920 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:55.620316  1867 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:55.654564 29222 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:55.608176  1866 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:55.663347 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:55.664167 29910 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:55.626163  1810 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:55.655208  1860 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:55.640560  1870 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:55.665228 29073 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:55.638554  1574 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:55.658931  1580 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:55.624851  1852 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:05:55.660427  1549 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:55.664919  1751 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:55.652115  1740 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:05:55.643062  1775 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.770473 29067 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:05:55.735855 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2580) took 1009 ms. Trace:
I20260709 13:05:55.706719 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:55.755793 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 801) took 1005 ms. Trace:
I20260709 13:05:55.733496 29071 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 13:05:55.745780 28968 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 163) took 39810 ms (client timeout 15000 ms). Trace:
W20260709 13:05:55.767570 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 78241.6us
I20260709 13:05:55.809329 29911 rpcz_store.cc:276] 0709 13:05:54.750560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.750700 (+   140us) service_pool.cc:224] Handling call
0709 13:05:54.750979 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.607176 (+856197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.755513 (+148337us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b54005e92d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:55.755722 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1897272576}
I20260709 13:05:55.814368 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:55.808974 29146 rpcz_store.cc:276] 0709 13:05:54.726489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.734332 (+  7843us) service_pool.cc:224] Handling call
0709 13:05:55.735652 (+1001320us) spinlock_profiling.cc:243] Waited 68 ms on lock 0x7b54001cc354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:55.735787 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156438400}
I20260709 13:05:55.692595  1673 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:55.838739 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 179788us
I20260709 13:05:55.845064 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:55.703970  1752 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.623s	user 0.008s	sys 0.004s
I20260709 13:05:55.693593  1860 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:55.718354  1605 consensus_meta.cc:287] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.000s
I20260709 13:05:55.763938  1779 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:55.760108  1799 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:55.682382  1741 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.854593  1752 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:55.710211  1867 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.718590  1870 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.764899  1848 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:05:55.819635  1748 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.705367  1824 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:55.858707  1849 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:55.888876  1847 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:55.889319  1849 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:55.889516  1847 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:55.889816  1849 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:55.890564  1849 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.862349  1605 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:55.892916 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.894022  1605 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:05:55.896097  1763 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.896705  1727 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:55.896659  1763 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.897148  1727 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:55.897970  1727 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.898890  1763 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:55.899186  1727 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.899422  1763 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:55.811018 28968 rpcz_store.cc:269] 0709 13:05:15.935012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.935201 (+   189us) service_pool.cc:224] Handling call
0709 13:05:17.801523 (+1866322us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b5400018954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a77090 00007f7836b15f3a 00007f783c8cc1cc 00007f783c9ca37e 00007f783ae8b2a4 00007f783ae8b232 00007f783ae8b161 00007f783ae8b0dc 00007f783ae8a392 00007f7834e4848c 00007f7834e478c6
0709 13:05:55.745646 (+37944123us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:48.352695 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:48.591491 (+238796us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:48.601222 (+  9731us) write_op.cc:435] Acquired schema lock
0709 13:05:48.603483 (+  2261us) tablet.cc:662] Decoding operations
0709 13:05:48.729952 (+126469us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:48.730050 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:48.730075 (+    25us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:05:48.730745 (+   670us) tablet.cc:701] Row locks acquired
0709 13:05:48.730791 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:05:48.730957 (+   166us) write_op.cc:270] Start()
0709 13:05:48.731094 (+   137us) write_op.cc:276] Timestamp: P: 1783602348730888 usec, L: 0
0709 13:05:48.731132 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:05:48.731844 (+   712us) log.cc:844] Serialized 2898 byte log entry
0709 13:05:49.324452 (+592608us) op_driver.cc:464] REPLICATION: finished
0709 13:05:49.744099 (+419647us) write_op.cc:301] APPLY: starting
0709 13:05:49.755587 (+ 11488us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:49.826934 (+ 71347us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:49.827087 (+   153us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:50.488177 (+661090us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:50.488540 (+   363us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:05:50.488611 (+    71us) write_op.cc:312] APPLY: finished
0709 13:05:50.488986 (+   375us) log.cc:844] Serialized 73 byte log entry
0709 13:05:50.489231 (+   245us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:50.489354 (+   123us) write_op.cc:454] Released schema lock
0709 13:05:50.489421 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:50.489902 (+   481us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:50.489933 (+    31us) write_op.cc:454] Released schema lock
0709 13:05:50.617689 (+127756us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b180030f820. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20522 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
Metrics: {"mutex_wait_us":1405159,"reported_tablets":29,"spinlock_wait_cycles":1318365952,"child_traces":[["op",{"apply.queue_time_us":323153,"apply.run_cpu_time_us":17963,"apply.run_wall_time_us":801518,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33387,"num_ops":7,"prepare.queue_time_us":182437,"prepare.run_cpu_time_us":29515,"prepare.run_wall_time_us":689014,"raft.queue_time_us":207299,"raft.run_cpu_time_us":14027,"raft.run_wall_time_us":203965,"replication_time_us":593125,"spinlock_wait_cycles":6222720,"thread_start_us":488134,"threads_started":4,"wal-append.queue_time_us":425939,"wal-append.run_cpu_time_us":21926,"wal-append.run_wall_time_us":2236892}]]}
I20260709 13:05:55.876125  1673 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.901535  1673 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:55.906347 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.906689 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:55.818336  1747 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:05:55.862840  1799 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:55.867875  1653 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:55.688079  1864 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.152s
I20260709 13:05:55.875777  1878 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:05:55.899856  1763 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.921860  1763 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:55.891820  1849 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:55.912585  1666 consensus_meta.cc:287] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.574s	user 0.003s	sys 0.008s
W20260709 13:05:55.935781  1694 consensus_meta.cc:287] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.646s	user 0.011s	sys 0.001s
I20260709 13:05:55.937000  1694 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.751748  1794 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:55.749130  1764 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:55.713871  1629 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:55.688468  1876 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.165s
I20260709 13:05:55.815830  1557 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:05:55.877085 29213 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:55.947602 29213 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:55.872246  1870 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:55.932337  1745 consensus_meta.cc:287] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.689s	user 0.000s	sys 0.011s
I20260709 13:05:55.878086  1343 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:55.879854  1824 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:55.880689  1748 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.882105  1099 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:55.890484  1847 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:55.758291  1547 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:55.765405  1639 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:55.914835 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.705000  1539 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:55.713407  1552 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:55.872355  1751 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.861227  1860 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.938675  1805 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:55.962286 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 99130.7us
I20260709 13:05:55.948064  1735 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:55.942111  1653 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:55.949165  1580 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:05:55.875617  1867 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.949016  1752 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:05:55.965253  1861 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:55.967199  1547 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:55.955298  1745 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:55.947306  1666 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:55.969785  1763 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:55.998950  1580 consensus_queue.cc:1243] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:55.983069  1673 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.003436  1885 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.008545  1885 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:56.009788  1826 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.056s	user 0.009s	sys 0.003s
I20260709 13:05:56.010133  1810 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.010910  1885 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.013115 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:56.014536  1867 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:56.014966  1680 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.532s	user 0.006s	sys 0.004s
I20260709 13:05:56.015478  1680 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.017704  1680 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:56.018733  1530 consensus_meta.cc:287] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.691s	user 0.008s	sys 0.000s
I20260709 13:05:56.019173  1530 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.019328  1574 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.021034  1847 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.021665  1574 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:56.023178 29147 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:05:56.023696 29147 pending_rounds.cc:85] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 5
I20260709 13:05:56.024047 29147 pending_rounds.cc:107] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:05:56.024307 29147 raft_consensus.cc:2889] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:56.025672 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2387) took 1326 ms. Trace:
I20260709 13:05:56.026002 29147 rpcz_store.cc:276] 0709 13:05:54.699253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.836736 (+137483us) service_pool.cc:224] Handling call
0709 13:05:54.876672 (+ 39936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.024736 (+1148064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.025462 (+   726us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400190ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:56.025617 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2635691264}
I20260709 13:05:56.027338 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 841) took 1040 ms. Trace:
I20260709 13:05:56.027736 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:56.027709 29909 rpcz_store.cc:276] 0709 13:05:54.986808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.987277 (+   469us) service_pool.cc:224] Handling call
0709 13:05:54.987603 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.257214 (+269611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.027148 (+769934us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54004b50d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:56.027288 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617983744}
I20260709 13:05:56.028587 29148 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:56.029559  1756 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.986193  1741 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.026875  1810 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:55.986953  1735 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.035079 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2564) took 1277 ms. Trace:
I20260709 13:05:56.035480  1775 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.031847 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:56.037078 29139 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: [12.11-15.12]
I20260709 13:05:56.030090 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:56.030146  1756 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.044858 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:56.045046  1756 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.038785  1679 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:56.046118  1679 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:56.045931  1880 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:56.028069 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 680) took 1194 ms. Trace:
I20260709 13:05:56.042093 29213 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.046953  1880 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.047531 29213 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:56.046437  1864 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.048411  1864 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.049191 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:56.050547 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:56.050716  1864 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.051118  1750 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:56.052057  1824 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:56.051703  1750 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.052680  1824 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.046953 29073 rpcz_store.cc:276] 0709 13:05:54.834071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.835688 (+  1617us) service_pool.cc:224] Handling call
0709 13:05:54.836359 (+   671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.686695 (+850336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.027840 (+341145us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54002ace54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:56.028014 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1905811840}
I20260709 13:05:56.054524 29148 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:05:56.054666  1557 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:56.055146  1824 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.055318 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:56.056810 29073 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:05:56.038990  1547 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.035496 29214 rpcz_store.cc:276] 0709 13:05:54.757571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.760464 (+  2893us) service_pool.cc:224] Handling call
0709 13:05:54.760823 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.256997 (+496174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.034819 (+777822us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b54003a9dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:05:56.035016 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091300096}
I20260709 13:05:55.997664  1739 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.001077  1861 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:05:56.041874 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:56.035768 29217 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 9. Current term is 11. Ops: []
W20260709 13:05:56.042074  1881 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:05:56.032496  1653 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.058370 29068 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.046229 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:56.060236 29139 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:05:56.090035 29064 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.090672 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:56.091066 29142 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:56.092022 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:56.092173 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:56.092607 29139 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:56.093353 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:56.093720 29139 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.060441 29909 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:05:56.060467 29141 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:05:56.063820 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:56.095286 29148 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:05:56.096113 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:56.096832 29146 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.068542 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:56.098462 29147 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:56.060793 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:56.053707 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:56.070977 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:56.071123 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:56.100754 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.101161 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:56.102766 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:56.050791 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:56.104717 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 54014us
I20260709 13:05:56.105635 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:56.106704 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:56.046061  1605 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:56.055730  1824 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:56.081694 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:56.089358 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:56.107304 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:56.099653 29145 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:56.108831 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:56.108247 29140 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:05:56.109632 29073 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:56.045624  1756 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:05:56.113425 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:56.113714 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:56.117514 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:56.141377 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:56.118906 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 65508.6us
I20260709 13:05:56.072327  1694 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:56.142875 29213 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:56.137046 29141 raft_consensus.cc:2410] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 12 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:05:56.139550 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 68670.6us
I20260709 13:05:55.979842  1799 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.145100 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:56.143568 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2579) took 1138 ms. Trace:
I20260709 13:05:56.076786  1802 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:05:56.147361 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:56.089145  1776 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.096379  1547 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:56.152294 29220 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:56.152760 29213 rpcz_store.cc:276] 0709 13:05:55.004762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.020517 (+ 15755us) service_pool.cc:224] Handling call
0709 13:05:55.031339 (+ 10822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.877002 (+845663us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b540022de20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a84dc0 00007f7836b0f1c0 00007f7836b10231 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c
0709 13:05:56.041763 (+164761us) spinlock_profiling.cc:243] Waited 45.2 ms on lock 0x7b540022de20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a8b06f 00007f7836b03f60 00007f7836b057de 00007f7836b0fb49 00007f7836b10416 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62
0709 13:05:56.143279 (+101516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.143499 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556053632}
I20260709 13:05:56.181656 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:56.181890 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.182534 29220 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:05:56.182971 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:56.183348 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:56.152391 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:56.182562 29139 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:05:56.135967  1552 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:56.137943  1673 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:56.200649 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48315.7us
I20260709 13:05:56.167263 29064 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:56.153995 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.146812 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:56.168891 29218 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:56.119628  1805 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:56.170311 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:56.055160  1557 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:56.176118 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 79413.8us
I20260709 13:05:56.177245 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:56.004477  1763 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.134595  1740 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:56.184957 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:56.164790 29146 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:56.169570 29219 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:05:56.203879 29222 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
W20260709 13:05:56.205014 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 58164.7us
I20260709 13:05:56.177541  1848 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.141319  1629 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:56.208863 29214 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:05:56.305446 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:56.168306  1578 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:05:56.135499  1739 raft_consensus.cc:2764] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:05:56.148916  1741 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:05:56.315294 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 138506us
W20260709 13:05:56.323617  1888 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:05:56.324517  1731 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.324397  1789 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:56.135171  1605 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.208750  1885 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.125s
I20260709 13:05:56.327982  1557 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.121133  1775 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:05:56.168949  1876 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.144711  1639 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:56.208562 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.404297 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:56.404402 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:56.404950 29911 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.169422  1746 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:56.346550 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.168447  1824 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:56.143157  1872 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:56.362797  1748 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.364938  1745 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.341382  1740 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:56.314536  1762 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:56.366464  1530 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.340986  1878 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:56.372905  1629 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.424583  1878 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.425225  1763 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:56.361378  1848 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.208966  1552 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.426777  1552 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.427778  1848 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:56.360512  1827 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.407s	user 0.008s	sys 0.000s
I20260709 13:05:56.208390  1753 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.169s
I20260709 13:05:56.412288  1639 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.432152  1639 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.415549  1740 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:56.180192  1679 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.401548  1885 consensus_queue.cc:1243] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:05:56.202317  1776 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.318329  1756 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:56.397833  1739 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:05:56.449378  1739 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:56.201017  1810 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.184s
I20260709 13:05:56.405890  1860 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:56.444252  1776 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:56.412849  1802 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:56.451774  1802 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.452687  1605 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.454021  1802 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.418555  1876 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.454293  1853 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:56.454715  1853 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:56.456079  1876 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:56.440868  1763 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:56.456598  1880 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:56.456668  1763 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:56.457639  1763 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.458777  1763 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:56.459434  1739 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:56.460338  1739 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.461537  1739 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.462966  1878 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:56.455531  1853 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.464211  1853 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.465431  1629 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.448344  1666 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:56.447103  1756 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.467157  1756 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.469347  1735 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.481391  1750 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.483857  1679 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:56.492638  1870 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:56.496402  1799 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:56.204213 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:56.532868 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:56.533717 29064 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:56.536214 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:56.536764 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:56.538730 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:56.539803 29064 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 10.
I20260709 13:05:56.541231 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:56.541678 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:56.542189 29064 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:56.543289 29072 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:56.544574 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:56.545787 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:56.546803 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 202060us
I20260709 13:05:56.323784 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 684) took 1402 ms. Trace:
I20260709 13:05:56.628438 29069 rpcz_store.cc:276] 0709 13:05:54.921073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.950927 (+ 29854us) service_pool.cc:224] Handling call
0709 13:05:56.323578 (+1372651us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002ad354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:56.323725 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2649018112}
I20260709 13:05:56.629714 29067 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:56.631191 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:56.631357 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:56.632153 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:56.633154 29220 raft_consensus.cc:2368] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13.
I20260709 13:05:56.634155 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.635507 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.636804 29073 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:56.639261 29073 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:56.314556 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:56.403198 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.402878 29219 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:56.678764 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 137433us
I20260709 13:05:56.557968 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.680765 29221 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
I20260709 13:05:56.401619 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.684033  1629 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:56.558504 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.559631 29213 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:56.669256 29220 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:56.621313 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:56.622969  1099 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:05:56.660858 29069 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:05:56.681577 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:56.418340 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.683017 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.677846 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:56.677796  1733 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.668910  1850 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:56.665509  1727 consensus_meta.cc:287] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.766s	user 0.021s	sys 0.006s
I20260709 13:05:56.679605 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.698460  1735 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:56.705917 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:56.709729 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.710490 29221 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:56.705994 29222 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:05:56.712076 29221 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:56.713820 29221 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:56.713862 29222 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:05:56.718376  1735 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.722445  1735 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:56.740351 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:56.741580 29218 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:56.741828 29073 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:05:56.743270 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:56.728859 29902 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.039s	sys 0.043s
I20260709 13:05:56.732532 29214 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:56.750757 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.757792  1336 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:05:56.758064 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 137084us
I20260709 13:05:56.758491  1336 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:56.759500 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:56.698621  1824 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:56.760380  1781 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:56.760380  1733 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.768134 29219 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
I20260709 13:05:56.768668  1848 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.769673  1733 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.769371  1848 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.769925 29070 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:05:56.760260  1629 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:56.772998  1639 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:56.773245 29113 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:56.774000  1639 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:56.774456  1639 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:56.774957  1805 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:56.775458  1805 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.777300 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 829) took 1864 ms. Trace:
I20260709 13:05:56.777793 29217 rpcz_store.cc:276] 0709 13:05:54.912625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.912771 (+   146us) service_pool.cc:224] Handling call
0709 13:05:56.035598 (+1122827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.081968 (+ 46370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.776834 (+694866us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003a9dd0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:56.777104 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2581505792}
I20260709 13:05:56.779902  1848 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.760519  1600 tablet_bootstrap.cc:492] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=1 applied=7 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:05:56.782655  1552 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.784118  1805 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:56.789480 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:56.792229  1810 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:56.793263  1810 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:56.791342 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:05:56.759680 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 101021us
W20260709 13:05:56.763305 29902 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.040s	sys 0.044s
I20260709 13:05:56.795650 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:56.799125 29069 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:05:56.804464 29072 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:56.814452 29064 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:56.816052  1850 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:56.821173  1850 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:56.821897  1794 consensus_meta.cc:287] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.872s	user 0.013s	sys 0.000s
W20260709 13:05:56.824424 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 143124us
I20260709 13:05:56.824079  1336 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.825409 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:56.825860 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:56.826717 29909 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:56.827435 29903 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:56.828508 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:56.830302  1756 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.831305 29911 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:56.841816 29908 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:05:56.831939 29065 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:56.832782  1855 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:56.833813 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:56.834524  1530 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:56.836102  1847 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.785s	user 0.006s	sys 0.013s
I20260709 13:05:56.836912  1733 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:56.801674  1552 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.837594 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:56.837805  1629 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.838344  1336 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.859773 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:56.838721  1824 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:56.836143 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:56.851503  1756 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.870651  1733 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.871315  1756 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:56.871383  1733 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.874426  1733 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.852259  1557 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:56.878134 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:56.878207  1740 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.868398  1824 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.879741 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:56.880564 29142 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:05:56.882856  1539 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:56.883111  1547 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.682s	user 0.009s	sys 0.001s
I20260709 13:05:56.883654  1539 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:56.884831  1539 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.885365 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:56.886811  1848 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:56.887295  1848 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:56.854120 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:56.888309  1872 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:56.855412 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:56.775822  1639 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.890391  1639 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.892398 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:56.893182 29142 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:56.856374  1530 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.847908  1855 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.848445  1600 tablet_bootstrap.cc:492] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:05:56.862586 29220 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:56.868381 29185 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:56.869112  1629 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:56.869982 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:56.855481 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:56.887687  1848 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.916226  1552 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:56.916953 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63049.2us
W20260709 13:05:56.896714 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 375407us
I20260709 13:05:56.919956 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:56.920226  1762 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:56.899431 29906 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:56.920702 29217 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:05:56.921332  1746 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:56.921635 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.922053 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.922266 29219 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 10.
I20260709 13:05:56.922664 29222 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:56.920693 29216 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:56.923074  1746 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:56.896278  1855 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:56.939462 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.941886  1673 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:05:56.943553  1629 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:56.948118 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 79894.3us
I20260709 13:05:56.953043  1727 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:56.944497 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.951622  1860 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:56.920754  1762 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:56.945611  1740 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:56.916967  1552 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:56.948421  1867 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:56.952243 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:56.953153 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 83237.4us
I20260709 13:05:56.895890  1530 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:56.962951  1580 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:05:56.952525 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.956213  1746 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:56.959295  1673 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:56.975554 29067 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:56.967818 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.968367 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:56.973584 29186 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:56.996943 29911 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:56.910064  1547 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:56.966863  1849 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.028s	user 0.004s	sys 0.008s
I20260709 13:05:56.997896 29213 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:05:56.999720 29216 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:56.968467  1600 ts_tablet_manager.cc:1403] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 2.805s	user 0.041s	sys 0.032s
I20260709 13:05:56.985911  1762 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:56.995113  1867 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:57.025538 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.968181  1855 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:57.027609 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 90491.4us
I20260709 13:05:57.000943  1673 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:57.029573 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 811) took 1056 ms. Trace:
I20260709 13:05:57.036391  1547 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:57.040186 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 66822us
I20260709 13:05:57.032639 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:57.060385 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:56.975885 28967 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 11 to 12, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 12 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.039655  1547 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:57.022696  1867 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:57.069933  1763 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.604s	user 0.006s	sys 0.007s
I20260709 13:05:57.071086 28967 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 4 to 11. New cstate: current_term: 11 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:57.081518  1552 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:57.083386  1901 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:57.083906 29068 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.837s	user 0.011s	sys 0.005s
I20260709 13:05:57.016187  1336 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:05:57.063656 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.095468 29220 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:05:57.096943 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2395) took 1063 ms. Trace:
W20260709 13:05:57.092713 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 61993.3us
I20260709 13:05:57.114388 29112 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:57.098739 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52897.4us
I20260709 13:05:57.063886  1872 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:57.110388 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 190861us
I20260709 13:05:57.116782 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:57.111724 29067 rpcz_store.cc:276] 0709 13:05:56.025963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.657255 (+631292us) service_pool.cc:224] Handling call
0709 13:05:56.975365 (+318110us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.005245 (+ 29880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.079781 (+ 74536us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b540012aa50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:57.088867 (+  9086us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635156864}
I20260709 13:05:57.121461 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.093551  1824 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:57.117578  1547 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:57.121433  1733 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:57.148999 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:57.156925 29213 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:57.063650  1746 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.124768 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.153481 29903 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:57.123997 29068 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:57.097180  1727 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:57.151098 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:57.158074 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.043411  1580 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.178421 29113 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:57.179042 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:57.035537 29902 rpcz_store.cc:276] 0709 13:05:55.968578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.969079 (+   501us) service_pool.cc:224] Handling call
0709 13:05:55.969397 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.969726 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.969755 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:55.973862 (+  4107us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.933867 (+960005us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:56.961036 (+ 27169us) spinlock_profiling.cc:243] Waited 70 ms on lock 0x7b5400769420. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a82dbe 00007f7836a8d930 00007f7836a8d8d8 00007f7836a8d858 00007f7836a8d813 00007f7836a8ca9c 00007f783c978626 00007f783c972dd8 00007f7836ac97cb 00007f7836acc2c0
0709 13:05:56.989642 (+ 28606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.012916 (+ 23274us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:57.012980 (+    64us) raft_consensus.cc:1685] finished
0709 13:05:57.013000 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.024670 (+ 11670us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3130,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3604,"spinlock_wait_cycles":161010432,"thread_start_us":802496,"threads_started":1,"wal-append.queue_time_us":872535}
I20260709 13:05:57.182121 29072 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:05:57.121665  1901 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:57.183876 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:57.184692 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.184885  1653 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.160024  1547 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:57.157567  1733 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:57.201709 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 86110.2us
I20260709 13:05:57.203512 29039 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:57.211045 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2394) took 1179 ms. Trace:
W20260709 13:05:57.211768 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 203439us
I20260709 13:05:57.211498 29068 rpcz_store.cc:276] 0709 13:05:56.025667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.029921 (+  4254us) service_pool.cc:224] Handling call
0709 13:05:57.198565 (+1168644us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:57.196219 28967 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 8 to 12, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 12 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.216984 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.218164 29072 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:57.219836  1578 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.220593  1578 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.223253  1578 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:57.224119  1673 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.225414  1673 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:57.227728  1653 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.230331 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:57.231706 29220 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:57.160408  1810 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:05:57.206787  1547 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:57.210814  1336 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:57.209146  1763 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:05:57.235796 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 57478.8us
I20260709 13:05:57.183116  1872 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.204896  1733 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:57.242373 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 137692us
I20260709 13:05:57.244310  1731 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:57.192397  1805 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.242338 28967 catalog_manager.cc:5697] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 8 to 9, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 9 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:57.202430  1848 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.249166 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.255012  1580 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:57.264657 29902 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
W20260709 13:05:57.258893  1916 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.176s	user 0.002s	sys 0.000s
I20260709 13:05:57.260246  1918 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:57.260517 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:57.261875  1336 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:57.262486  1810 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:57.235826  1847 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:57.255368 28967 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 14 to 17, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 17 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.200584  1824 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.271250 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.274770 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:57.271294  1855 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:57.276983 29902 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:57.277455  1872 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:57.209335  1727 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.280369  1727 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:57.280838  1847 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.281464  1847 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:57.283104  1805 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:57.248122  1763 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:57.286602 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.290402 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:05:57.292696 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 169272us
I20260709 13:05:57.284438  1763 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:57.223709  1600 raft_consensus.cc:359] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 12 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.300880  1639 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:57.301378  1756 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:57.305498  1824 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:57.306061 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.307482 29071 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:57.293576 28967 catalog_manager.cc:5697] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 6 to 10, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 10 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:57.271710  1918 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:57.317763 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:57.318536 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 43877us
W20260709 13:05:57.321784 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 61487.3us
I20260709 13:05:57.321307  1848 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:57.338428 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:57.313858  1639 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:05:57.360657 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.354667  1557 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:57.372705  1557 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.373171  1557 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:57.376232  1799 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.376391  1917 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.376885  1799 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:57.377445 29071 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 687) took 1472 ms (client timeout 1547 ms). Trace:
W20260709 13:05:57.378042 29071 rpcz_store.cc:269] 0709 13:05:55.904853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.906433 (+  1580us) service_pool.cc:224] Handling call
0709 13:05:57.377192 (+1470759us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54001d8654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:05:57.377380 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548500736}
I20260709 13:05:57.379454  1799 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:57.380626  1727 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:57.380093  1901 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.383162 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:57.383816 29146 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:57.384802  1552 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.385479  1552 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.387084  1653 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:57.389225 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 74391.9us
I20260709 13:05:57.391511  1639 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.393276  1639 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:57.398615  1847 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:57.400283 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44003.3us
I20260709 13:05:57.401252 29910 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:05:57.402819 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:57.405263 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.406203 29905 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:57.363031 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.381054  1727 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:57.353504 28967 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:57.319370  1918 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:57.416673 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59374us
I20260709 13:05:57.412415 29907 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:57.421762 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:57.423591 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.425647 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:57.427487 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.428318 29911 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:57.428371  1902 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:57.430446 29037 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:57.431345  1629 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:57.431850  1629 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.432503  1629 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:57.420679  1848 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:05:57.427882  1756 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:05:57.420480  1805 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:57.436501  1805 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:57.437983  1653 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:57.387749  1552 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:57.440706 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.443399  1746 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:57.448767 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.451116 29908 raft_consensus.cc:2368] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13.
I20260709 13:05:57.429807  1902 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:57.449868 29907 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
W20260709 13:05:57.461735 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 104325us
I20260709 13:05:57.379463  1600 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:05:57.466969 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:57.460300  1902 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:57.469753 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.377381  1917 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.473786 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:57.410802  1336 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.110s
I20260709 13:05:57.479074 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:57.494448 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:57.496697 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 71317.9us
I20260709 13:05:57.500763  1735 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:57.508574 29071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43a52106dbee45398d36a633f281eef6"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39110
I20260709 13:05:57.509164  1810 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:57.509479 29071 raft_consensus.cc:606] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:57.509734  1810 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:57.509609  1902 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.510886  1810 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.512663  1902 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:57.512861 29071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:39110
I20260709 13:05:57.513491 29071 raft_consensus.cc:606] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:57.513852 29071 raft_consensus.cc:614] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:05:57.516165  1574 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.011s	sys 0.024s
W20260709 13:05:57.516501  1912 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 13:05:57.487008  1653 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.517763  1653 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.521275  1847 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:57.521345  1751 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.522279  1751 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.486361  1805 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.486615  1848 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:57.528424  1848 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:57.529763  1848 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.532843 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.533891 29072 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:57.481949  1756 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:57.482364  1746 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:57.540093 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 83241.9us
I20260709 13:05:57.483777 28967 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 6 to 10, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 10 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:57.541260 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 323971us
W20260709 13:05:57.542148 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68546.5us
I20260709 13:05:57.543311 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:57.546592 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.547362  1847 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:57.563546 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:57.544270  1762 consensus_meta.cc:287] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.000s
I20260709 13:05:57.550079 28967 catalog_manager.cc:5697] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.576671 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:57.588843  1751 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:57.574620  1847 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.590541 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:57.592247  1727 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.591872  1629 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.597621  1917 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:57.598058  1918 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:57.608415 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44937.1us
I20260709 13:05:57.551988  1902 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:57.615149 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:57.637077 29072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39110
I20260709 13:05:57.637820 29072 raft_consensus.cc:606] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:05:57.638222 29072 raft_consensus.cc:614] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:57.631433 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:57.641191 29146 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:57.616612  1336 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:57.577837  1810 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:57.620507  1847 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:57.625486  1653 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:57.623898 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:57.696002 29220 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:57.621601 28967 catalog_manager.cc:5697] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 7 to 8, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 8 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.630309  1727 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:57.621645  1731 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:57.631273  1552 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:57.538544  1547 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.644681 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:57.617610 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 71903.2us
I20260709 13:05:57.549935  1746 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.620898 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:57.715317 29213 raft_consensus.cc:2410] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:05:57.631461  1629 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:05:57.724777 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:57.666256  1850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.016s	sys 0.016s
I20260709 13:05:57.631642  1530 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:05:57.634281  1902 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.545658  1600 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:57.713958 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.706935 28967 catalog_manager.cc:5697] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.662286  1855 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:57.720148 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 86980.6us
W20260709 13:05:57.728598  1850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.016s	sys 0.016s
I20260709 13:05:57.719857  1731 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.710680  1552 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.619220  1762 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.623680  1756 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:57.725734 29911 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:57.744630 28967 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:57.738909  1902 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:57.764468 28967 catalog_manager.cc:5697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 10 to 11, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 11 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.767617 29064 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:05:57.750675  1552 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:57.770027 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 288249us
I20260709 13:05:57.785173 28967 catalog_manager.cc:5697] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 5 to 11, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 11 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.786213 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:57.801258 29220 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:57.812767 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:05:57.789307  1733 consensus_meta.cc:287] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.001s
W20260709 13:05:57.837057 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.758775  1855 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.816676  1629 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:57.767176  1756 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.850396 29037 leader_election.cc:400] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13.
I20260709 13:05:57.853727 29037 leader_election.cc:403] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:57.848696 28967 catalog_manager.cc:5697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 8 to 13, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 13 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:05:57.800086  1901 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 13:05:57.865314 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 51443us
I20260709 13:05:57.836053  1547 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:57.871353  1928 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:05:57.871889  1752 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:57.872577  1921 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:05:57.872792  1752 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.873291  1752 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:57.874254  1925 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:05:57.864506 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 129412us
I20260709 13:05:57.869174  1580 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:57.869910  1746 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:57.830685  1364 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:57.875831 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 401528us
I20260709 13:05:57.852384  1629 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:05:57.829418  1733 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:57.872195  1731 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:57.896274 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.898865 29064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:39110
I20260709 13:05:57.869249  1931 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:05:57.902412 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 200265us
I20260709 13:05:57.869212  1918 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:57.857178 28967 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 9 to 11, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131), now has a pending config: VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129), VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130), VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131), VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 11 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } } pending_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:57.883634  1849 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:05:57.909512 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.910209 28967 catalog_manager.cc:5697] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.912222 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:57.912909 29910 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:57.912129 28967 catalog_manager.cc:5697] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 6 to 9, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 9 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.914952 28967 catalog_manager.cc:5697] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 4 to 12, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 12 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:57.916352 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.917053 29902 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:57.917402 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:57.918249 29069 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:57.919061 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:57.920150 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:57.920600 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.921003 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:57.924067 29039 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:57.922358 28967 catalog_manager.cc:5697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 8 to 11, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 11 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:05:57.922309 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.925410 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.926962 29071 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:05:57.926057 28967 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:05:57.927575  1763 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.593s	user 0.012s	sys 0.000s
I20260709 13:05:57.928202 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:57.928874 29069 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:57.901854  1735 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.887243  1578 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.887549  1629 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:57.942821 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52036.6us
I20260709 13:05:57.928210 28967 catalog_manager.cc:5697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:57.898774  1547 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.951749 29069 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:57.956413 29907 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:05:57.964896 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:57.966194 29071 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 20: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:57.967340 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.964960 28967 catalog_manager.cc:5697] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:57.972568 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.931598  1735 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:57.980949 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 81423.7us
I20260709 13:05:57.986117 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:58.008910 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.005774 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.981894 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:58.013756 28967 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 8 to 10, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 10 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:05:57.966765  1918 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.017498 29902 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:05:58.045074 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:57.907055  1364 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:57.940933  1763 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.059563 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:58.060765 29071 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:57.971108  1578 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.020457  1918 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.017328  1901 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:58.072016 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 151971us
I20260709 13:05:58.075585 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.022909  1746 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:58.093060 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:58.094820 29910 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:05:58.099758 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:58.093830 29073 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:58.070472  1580 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:57.978789  1931 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:58.115765 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 349135us
I20260709 13:05:58.117748 29906 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:58.013295  1762 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.111766 29146 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:05:58.114768 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.121796 29213 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:58.120591 29071 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:05:58.102936  1600 ts_tablet_manager.cc:1434] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 1.087s	user 0.014s	sys 0.013s
I20260709 13:05:58.124147 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:58.086550  1574 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.130532 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:58.131068 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 174128us
I20260709 13:05:58.134124  1950 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:58.134727  1950 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.135102  1950 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:58.137553  1530 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:58.139590  1336 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.140264  1954 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.140764  1955 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:58.141041  1954 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.141557  1955 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.142231  1955 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.145649  1539 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:58.146140  1539 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:58.149160  1578 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:58.151047  1867 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.031s	user 0.005s	sys 0.012s
I20260709 13:05:58.151582  1867 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.154194  1867 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:58.140374  1336 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.161787  1336 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.121789  1855 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.168614 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:58.172236  1914 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:05:58.173318 29073 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:05:58.173702  1944 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:05:58.174868 29073 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15. Ops: []
W20260709 13:05:58.174994  1930 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:05:58.176687  1918 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.179126  1940 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.179850  1940 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:58.181044 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:58.181586  1927 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:05:58.182722 29037 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:58.183514  1940 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.184708  1918 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:58.149111  1850 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:58.186367  1850 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:58.186756  1918 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:58.187331  1918 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:58.187583  1850 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.191020  1847 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.193254  1364 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.194106 29905 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:05:58.194845 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:58.183885  1917 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:58.191996  1918 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.196223 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.192711 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:57.970718  1547 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.202694  1574 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.203336  1848 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:58.203747 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:58.149091  1950 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.208272  1917 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:58.213152  1735 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:58.214531  1939 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:05:58.258594  1740 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:58.282078  1673 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.056s	user 0.011s	sys 0.006s
I20260709 13:05:58.217003  1824 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:58.282176  1740 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.282781  1824 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:58.264410  1848 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.284502  1848 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:58.283867  1824 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.285292 29064 raft_consensus.cc:606] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:58.285915 29064 raft_consensus.cc:614] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:58.220857  1763 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.224941  1931 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.115341  1901 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:58.102391  1746 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:58.225520 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:58.254796  1756 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.226420 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:05:58.227139 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 97990.6us
W20260709 13:05:58.256827  1557 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.883s	user 0.014s	sys 0.001s
I20260709 13:05:58.231117 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:58.232065  1731 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:58.359952 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.227708 29909 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:58.223186 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:58.256826 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:58.362321 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 139220us
I20260709 13:05:58.281340  1872 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:58.217893  1727 consensus_meta.cc:287] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.507s	user 0.006s	sys 0.006s
I20260709 13:05:58.147318  1539 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.219096  1733 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.197260  1954 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.265913 29215 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:58.232798  1867 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:58.291893 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.298460  1824 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:58.305431 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:58.308317  1574 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.308744  1740 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.311058  1950 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.265267  1552 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.316860  1955 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.329212  1940 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:58.393088 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:58.393188 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:58.393919 29905 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
W20260709 13:05:58.335429  1810 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.636s	user 0.009s	sys 0.001s
I20260709 13:05:58.285296  1917 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:05:58.347431 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.374903  1756 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:58.129321  1600 ts_tablet_manager.cc:1467] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Tablet startup took 1122ms
I20260709 13:05:58.420091  1867 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:58.420519  1849 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13.
I20260709 13:05:58.420787  1867 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.421663  1746 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:05:58.423032  1867 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:58.350162 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:58.424134 29069 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
W20260709 13:05:58.424615 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 74565.1us
I20260709 13:05:58.425333  1849 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.425920  1849 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:58.419179  1600 ts_tablet_manager.cc:1469] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Trace:
0709 13:05:48.589537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.593191 (+  3654us) service_pool.cc:224] Handling call
0709 13:05:48.878233 (+285042us) ts_tablet_manager.cc:934] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
0709 13:05:50.518257 (+1640024us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:05:51.205208 (+686951us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:51.205350 (+   142us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:05:51.348654 (+143304us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:05:51.387348 (+ 38694us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:51.455985 (+ 68637us) ts_tablet_manager.cc:989] Replica registered.
0709 13:05:51.465206 (+  9221us) inbound_call.cc:177] Queueing success response
0709 13:05:54.163740 (+2698534us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:05:57.026779 (+2863039us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:05:57.220456 (+193677us) tablet_replica.cc:261] Starting instrumentation
0709 13:05:57.221257 (+   801us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":35602,"mutex_wait_us":29935,"raft.queue_time_us":128069,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"tablet-copy.queue_time_us":215782,"tablet-download-pool.queue_time_us":177,"tablet-download-pool.run_cpu_time_us":1450,"tablet-download-pool.run_wall_time_us":1266065,"thread_start_us":300579,"threads_started":3}
W20260709 13:05:58.227849  1942 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.156s	user 0.001s	sys 0.001s
I20260709 13:05:58.428416  1740 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:58.428715  1849 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.429322  1849 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:58.429083  1740 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.429884  1901 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.431483  1740 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.432348  1955 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:58.433707  1751 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:58.434222 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.261085 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:58.435210 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.360718 29219 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
W20260709 13:05:58.371004  1805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.003s	sys 0.003s
I20260709 13:05:58.376116 29216 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:05:58.360306  1557 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.377283  1958 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:58.362401 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 108461us
W20260709 13:05:58.390110 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 159189us
I20260709 13:05:58.375823  1673 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:58.359982  1948 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:58.339002  1752 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.375936  1931 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:58.387202 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 81860.8us
I20260709 13:05:58.412436 29064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:39110
W20260709 13:05:58.413342 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.435850 29148 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
W20260709 13:05:58.456550  1805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.004s	sys 0.003s
I20260709 13:05:58.400733  1954 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.429855  1849 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.457409  1872 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:58.400868  1574 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:05:58.457998  1872 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:58.458565  1574 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:58.435770 29071 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:05:58.459110  1574 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:58.416066 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:58.460088  1673 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.460484  1673 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:58.461243 29067 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:58.435210 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:58.462560  1557 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.448158 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:58.465456 29144 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:05:58.466625 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:58.467291 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.467607 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:58.468022 29221 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:05:58.467873 29874 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:58.468498 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:58.469218  1945 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:58.404273  1629 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.470417  1735 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:58.470963  1735 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.471395 29072 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
W20260709 13:05:58.438162  1902 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.656s	user 0.011s	sys 0.002s
I20260709 13:05:58.471870 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:58.403620  1731 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:58.472019  1902 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.473440  1731 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.474012 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:58.418074 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.440300 29065 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:58.475653 29213 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:58.437013  1953 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.437932 29141 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:58.476686  1953 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.416165  1756 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.469823  1945 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.474289  1902 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.502810 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:58.438371 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.414135  1917 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.506400 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:58.507164 29213 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:58.434307  1751 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:58.479177 29064 raft_consensus.cc:606] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:05:58.479213 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 77552.5us
I20260709 13:05:58.347976 29907 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:58.465842 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:58.476526 29143 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:05:58.504384 29215 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:05:58.515264 29064 raft_consensus.cc:614] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:58.509816 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.478166  1752 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:58.510339 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:58.455497  1850 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:58.441489  1552 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.516527 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:58.523694 29905 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:58.471419  1735 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:58.443746  1539 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:58.509195 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.546581 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:58.526028 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.511268 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.531320 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:58.540149 29219 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:58.541311 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:05:58.545966 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 257482us
I20260709 13:05:58.502156  1945 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:58.506081  1948 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:58.540210 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.547043 29218 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
W20260709 13:05:58.555635 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.563256 29907 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:58.564121 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.566224 29216 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:05:58.568270 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44603.7us
W20260709 13:05:58.570075 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.408789  1940 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.580964 29144 pending_rounds.cc:85] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 4
I20260709 13:05:58.566510 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.583132 29213 raft_consensus.cc:2410] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:05:58.503522  1805 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:05:58.574024 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.589165 29148 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:58.585320 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:58.589732 29214 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:58.457449  1954 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.587668 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:58.592078 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.554024  1917 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:58.596246  1938 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.596858  1938 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.599020  1938 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.599519  1938 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.600062  1364 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.600381 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.600615  1364 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.603082  1364 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.557155  1948 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.476440  1958 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.613381  1855 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:58.613783 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.613950  1847 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.614365  1918 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:58.615741  1547 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:58.542702  1574 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.573606  1752 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:58.616935  1965 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:58.617095 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:58.592814 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:58.615751 29140 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:05:58.618680 29905 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:58.581445 29144 pending_rounds.cc:107] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:05:58.567271  1849 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.653393 29142 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:58.559504  1902 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:05:58.592444  1733 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:58.653388 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 47731.8us
I20260709 13:05:58.657936 29144 raft_consensus.cc:2889] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:58.618119  1552 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.679581  1552 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.679812  1733 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.682379  1552 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.684581 29148 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:58.677278 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 112775us
I20260709 13:05:58.686952 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:58.618718  1948 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:58.625049  1855 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:58.683239  1953 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.682832  1733 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.690279 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:58.697795 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:58.633798  1629 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.701800  1940 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.703048  1673 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.703007  1751 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.706120  1673 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.707198  1574 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:58.708559  1867 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:58.631940  1578 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.660396  1965 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:58.661348  1547 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:58.637225  1918 consensus_queue.cc:1243] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:05:58.722704 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 191555us
I20260709 13:05:58.720697 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:58.685639  1752 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:58.721089 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:58.729440 29038 leader_election.cc:400] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13.
I20260709 13:05:58.733569  1629 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.631551  1938 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:58.738579 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:58.719111 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 202703us
I20260709 13:05:58.740850 29905 raft_consensus.cc:2368] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15.
I20260709 13:05:58.740921 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:58.739256  1931 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:58.773505 29140 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:58.774501 29038 leader_election.cc:403] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:05:58.775815 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.719326  1965 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:58.779359 29218 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:58.717001  1756 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.783183 29112 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:58.793365 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.795657 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:58.800208 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:58.808096 29909 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:58.807994  1943 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:58.784075  1848 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:58.812497  1960 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:05:58.812613  1752 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.816154  1364 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:05:58.816280 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 199248us
I20260709 13:05:58.808631  1933 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:58.818572 29219 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:58.846405  1953 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.820080  1673 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13.
I20260709 13:05:58.822286  1653 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:58.725031  1578 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.847201  1953 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.850467  1578 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:58.847685  1653 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:58.852353 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.824930  1580 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.851773 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:58.860102 29145 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:58.823397 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.862106 29142 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
W20260709 13:05:58.819141 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 104547us
I20260709 13:05:58.771133  1902 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:05:58.813449  1963 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:05:58.827543 29903 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
W20260709 13:05:58.811699  1966 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:05:58.865795  1902 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:58.839617 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 67800us
W20260709 13:05:58.840766 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.840941 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:58.818130  1810 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.868209  1902 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:58.867892  1810 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.869349 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:58.873042 29139 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:58.876353 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:58.877784 29113 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:58.782186  1629 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.879314 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:58.839289  1950 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:05:58.885067 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 102332us
I20260709 13:05:58.887530 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:58.818215  1931 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.888411 29903 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:05:58.844959  1855 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.904088 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:58.905740 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 113274us
I20260709 13:05:58.906061 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.906734 29215 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:58.907598 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.907596 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.908237 29215 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:05:58.911906 29037 leader_election.cc:400] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15.
I20260709 13:05:58.912287 29037 leader_election.cc:403] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:05:58.908149 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:58.880168  1933 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:58.914529 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:58.857757  1953 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.883611  1955 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:05:58.918455 29871 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.008s (SENT)
W20260709 13:05:58.920219 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 50954us
I20260709 13:05:58.889565  1673 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:58.838624  1943 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.923837  1956 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:58.924322  1956 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.924746  1956 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:58.925161  1848 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:58.927986  1970 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:05:58.929899  1824 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.001s
I20260709 13:05:58.934137 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.934856 29146 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:05:58.935007  1948 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.935788 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:58.936872 29146 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:05:58.939340  1980 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:05:58.939739  1948 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:58.941018 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2513) took 1025 ms. Trace:
I20260709 13:05:58.941558  1961 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:58.941485 29068 rpcz_store.cc:276] 0709 13:05:57.915389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.921164 (+  5775us) service_pool.cc:224] Handling call
0709 13:05:57.921509 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:57.979820 (+ 58311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.979867 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:58.016151 (+ 36284us) log.cc:844] Serialized 26 byte log entry
0709 13:05:58.392308 (+376157us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:58.392361 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.392458 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:58.940793 (+548335us) raft_consensus.cc:1685] finished
0709 13:05:58.940818 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:58.940948 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18035,"raft.queue_time_us":63982,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":8047,"thread_start_us":303024,"threads_started":1,"wal-append.queue_time_us":509166}
I20260709 13:05:58.948882  1938 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:58.940524  1973 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:05:58.874826  1580 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:58.960954 29112 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:58.922223 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:58.937832 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.949468  1849 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:05:58.956715 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:58.957742 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.957617  1945 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:58.922474 29188 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.836s (SENT) [suppressed 13 similar messages]
W20260709 13:05:58.980716 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 58260.5us
I20260709 13:05:58.959463 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:58.968609 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 46460.5us
I20260709 13:05:58.914134  1731 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:58.985569 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.985749  1364 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:58.986446 29140 raft_consensus.cc:2410] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:05:58.973820  1867 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:58.987407 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.988847 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.989250 29875 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:58.989639 29141 raft_consensus.cc:2410] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:05:58.990206  1977 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:58.990491 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:58.990679 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:58.990962 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.991247 29141 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:58.995234 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:58.992238  1977 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:58.996443  1977 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.996759 29141 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:05:58.996798  1977 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:58.997329 29141 pending_rounds.cc:85] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 6
I20260709 13:05:58.997678 29141 pending_rounds.cc:107] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 12.7
W20260709 13:05:58.997536 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 90297.5us
I20260709 13:05:58.974925  1810 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.976038  1855 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:59.004364 29142 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:05:59.005952 29142 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:58.960161  1805 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:58.969393  1954 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:58.915123  1950 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:59.008157  1805 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.008252  1364 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.978739 29219 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:59.011857  1629 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:59.011843  1986 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:58.982072 29213 raft_consensus.cc:2410] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:05:59.013226  1986 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:05:58.982462 29214 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:58.983168  1958 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:59.013926  1945 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:59.026961 29139 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:05:58.984984 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.027828 29218 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:05:59.029129 29903 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:59.015676 29140 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:59.035456 29143 raft_consensus.cc:2376] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12. Re-sending same reply.
I20260709 13:05:59.044170 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.045725 29142 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:05:59.014179  1986 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:58.979221  1733 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:58.979591  1763 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:05:58.974700  1968 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:05:59.003427  1943 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:59.008970  1364 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:58.931391  1933 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.009158  1950 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.008762  1805 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:58.983414  1580 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:05:59.017494 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 184671us
W20260709 13:05:59.021695 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.016940  1938 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.055583 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.022714  1731 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:59.056459  1733 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:59.063270 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.057797 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:05:59.085330 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.997938 29141 raft_consensus.cc:2889] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:59.097916 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:58.993546  1653 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.096375  1943 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:59.062287  1763 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.088160  1733 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:59.113760  1943 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:05:59.119040 29903 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:05:59.066977  1574 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:05:59.121842  1943 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:59.131820 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2520) took 1194 ms. Trace:
I20260709 13:05:59.132468 29066 rpcz_store.cc:276] 0709 13:05:57.937070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.170128 (+233058us) service_pool.cc:224] Handling call
0709 13:05:58.170625 (+   497us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:58.171110 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:58.171143 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:58.198285 (+ 27142us) log.cc:844] Serialized 52 byte log entry
0709 13:05:58.546658 (+348373us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:58.546720 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.546818 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:59.131490 (+584672us) raft_consensus.cc:1685] finished
0709 13:05:59.131519 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.131735 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6315,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":116,"thread_start_us":310016,"threads_started":1,"wal-append.queue_time_us":456271}
I20260709 13:05:59.114308  1849 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:59.088812  1938 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.198660  1947 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:59.185158  1849 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.204092  1731 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.273376  1947 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:05:59.286974  1947 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:59.313695  1948 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.316036  1574 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.062633 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.097627 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.316129  1948 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:05:59.332140 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 359671us
I20260709 13:05:59.305143  1653 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:59.336216  1574 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.357197 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.357242  1639 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:59.362890 29142 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:59.363135  1948 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:59.359140  1629 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.367808  1629 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.370340  1629 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:59.381224 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:59.359562  1917 consensus_meta.cc:287] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.741s	user 0.011s	sys 0.006s
W20260709 13:05:59.376647  1850 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.834s	user 0.014s	sys 0.005s
W20260709 13:05:59.381280  1872 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.921s	user 0.020s	sys 0.025s
W20260709 13:05:59.392202 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 77778.4us
I20260709 13:05:59.395452  1917 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:59.396281 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.397576 29142 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:59.403403 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:59.404341 29218 raft_consensus.cc:2410] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:05:59.407042 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:59.411317 29214 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:59.386685  1629 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:59.415494 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:59.415794 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:59.416173 29217 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:59.416282 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:59.416791 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:59.417493 29216 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:59.418322 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.370364  1977 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.422665 29220 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:59.422935 29139 raft_consensus.cc:2410] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 17 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:05:59.424633 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.425800  1849 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:59.408591  1855 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.427139  1855 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:05:59.430287 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.438446 29143 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:59.440270 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:59.432288  1989 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:05:59.446204  1855 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:59.434653  1735 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.874s	user 0.023s	sys 0.016s
I20260709 13:05:59.453693  1735 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.456204 29907 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:05:59.458168 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.391170  1872 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.469645 29142 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
W20260709 13:05:59.475584 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.477115 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:05:59.483292 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.423429  1850 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 13:05:59.489094 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.455902  1735 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:59.502100 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:59.507989 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:05:59.502337  1965 consensus_meta.cc:287] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.649s	user 0.005s	sys 0.011s
W20260709 13:05:59.523506  1995 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:05:59.532409 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:59.492498  1810 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:59.533357 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 53158.8us
I20260709 13:05:59.531008 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:59.536638 29217 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:05:59.536479 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.515287  1855 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:59.547676 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:59.547466 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:59.550750 29216 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:05:59.515868  1938 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:59.562727 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:59.561925 29910 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEARNER]: Advancing to term 14
I20260709 13:05:59.549568  1735 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
W20260709 13:05:59.568310  1997 log.cc:927] Time spent T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 13:05:59.570025  1994 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:59.536530  1810 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.549705  1975 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:59.532445  1727 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
W20260709 13:05:59.580693 29145 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.707s	user 0.016s	sys 0.026s
I20260709 13:05:59.583422 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:05:59.533063  1965 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.533912  1848 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:05:59.588424 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.589396  1977 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:59.591076  1872 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:59.574090  1849 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:59.597119 29907 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:05:59.599349 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.600015 29139 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:59.573555  1938 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.586330 29145 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
W20260709 13:05:59.615849  1805 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.033s
I20260709 13:05:59.624261  1805 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.605816 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:05:59.626266 29146 raft_consensus.cc:2368] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15.
I20260709 13:05:59.626550  1805 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:05:59.596972  1364 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.521s	user 0.018s	sys 0.003s
I20260709 13:05:59.627466  1849 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:59.627627  1364 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.628358  1849 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.629216  1849 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:59.575754  1810 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:59.633526  1763 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:59.618129 29139 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:05:59.623224  1574 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:59.637142 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:59.638213  1917 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:59.638715  1917 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:59.639137 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:05:59.639680  1917 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.641258  1965 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:59.645527  1947 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.649879 29146 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 15. Ops: []
I20260709 13:05:59.649763 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.651516 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.655097 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:59.664623  1735 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:59.665362  1735 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:59.666453  1735 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.604055  1975 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:59.668533  1975 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.670102 29219 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:59.681545  1364 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:05:59.682016 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.683205 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.684670 29145 raft_consensus.cc:2410] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:59.614540 29910 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:59.692122 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:05:59.694489 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.699730 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.700721 29140 raft_consensus.cc:2410] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:59.701622 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.702018  1629 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:59.702410 29140 raft_consensus.cc:2368] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:05:59.702998 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:05:59.703498 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.693745  1850 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:59.704794 29140 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:59.706458 29145 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:59.721427 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:59.724877 29216 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:05:59.725607 29139 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:59.723019 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:59.694568  1956 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.769s	user 0.017s	sys 0.021s
W20260709 13:05:59.694177  1733 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.557s	user 0.016s	sys 0.002s
I20260709 13:05:59.704752  1850 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:59.726552 29220 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:59.728232 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:05:59.702148  1961 consensus_meta.cc:287] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.760s	user 0.017s	sys 0.020s
I20260709 13:05:59.736258 29146 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:05:59.738448 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:05:59.738406 29871 leader_election.cc:336] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.547s (SENT)
I20260709 13:05:59.744844 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.744900 29871 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:05:59.747596 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.745716 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2418) took 1185 ms. Trace:
I20260709 13:05:59.650903  1938 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.658776  1943 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.753285 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:59.760710 29068 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:59.757283 29216 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:59.758405 29220 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:05:59.765695 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:59.766804 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:05:59.719447  1918 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:05:59.744354  1805 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:59.747256  1850 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:59.776199 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.770759 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:59.732842  1956 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.735929  1965 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:05:59.762233  1933 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:59.774056 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.774638 29909 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:59.744560  1961 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15.
I20260709 13:05:59.733453  1733 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:05:59.767907  1986 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.770332  1955 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:59.789175 29068 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:05:59.770890  1639 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:59.796218 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.772066  1763 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:59.798400 29218 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:59.800870 29146 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:59.774910  1947 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:59.772353  1855 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:59.803367 29147 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:59.808143  2015 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:59.809195  2015 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:59.809578  2015 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:59.813899  1639 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:59.814450  1956 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:05:59.815062  1639 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.815950  1955 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:05:59.817123 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:59.817183 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.817989 29068 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:59.818418 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.819043 29140 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:05:59.816905  1955 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.819816 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.820760 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.819846 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.824360 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.852819 29146 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:59.748890 29144 rpcz_store.cc:276] 0709 13:05:58.548833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.549083 (+   250us) service_pool.cc:224] Handling call
0709 13:05:58.564171 (+ 15088us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:58.834011 (+269840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:58.834043 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:58.856300 (+ 22257us) log.cc:844] Serialized 52 byte log entry
0709 13:05:59.359479 (+503179us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:59.546330 (+186851us) log.cc:844] Serialized 14 byte log entry
0709 13:05:59.733168 (+186838us) log.cc:844] Serialized 14 byte log entry
0709 13:05:59.733475 (+   307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.733579 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:59.733630 (+    51us) raft_consensus.cc:1685] finished
0709 13:05:59.733647 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.733800 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":160375,"prepare.queue_time_us":9696,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":161338,"raft.run_cpu_time_us":427,"raft.run_wall_time_us":40292,"thread_start_us":823279,"threads_started":3,"wal-append.queue_time_us":542535}
W20260709 13:05:59.830529 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 282911us
I20260709 13:05:59.803253  1986 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.840266  1850 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.840922 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.844787 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.861421 29142 raft_consensus.cc:2368] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:05:59.845553 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.862537 29064 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:59.846287 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.846539  1943 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:59.863363 29065 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:59.869187 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.846441 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.846963 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:05:59.847438  2001 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:05:59.847663 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:59.820760 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.854297 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:59.856289 29068 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:05:59.874081 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.859422 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.859769 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.865219 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.865381 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.875349 29146 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:59.872349 29067 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:59.875545 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.872972 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.874044 29069 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:59.878893 29073 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:59.861847 29066 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:05:59.880151 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.880283 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:59.872524 29147 raft_consensus.cc:2410] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:59.876633 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:05:59.880762 29145 raft_consensus.cc:2368] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16.
I20260709 13:05:59.881156 29218 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:59.877975 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.881629 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.882447 29145 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:05:59.882427 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.883960 29140 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:59.887759 29216 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:59.879863 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:59.888990 29141 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:05:59.890096 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.890777 29073 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:05:59.891232 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.883697 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:05:59.891803 29145 raft_consensus.cc:2376] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13. Re-sending same reply.
I20260709 13:05:59.896621 29140 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:05:59.896767 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:05:59.896767 29066 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:59.898509 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.898617 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:05:59.900481 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.900965 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.901043 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.901717 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.902619 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:05:59.897046 29140 pending_rounds.cc:85] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 10
I20260709 13:05:59.904852 29140 pending_rounds.cc:107] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:05:59.905089 29140 raft_consensus.cc:2889] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:59.908206 29146 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:05:59.909521 29218 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:05:59.909907 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:05:59.859030  1918 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:05:59.896054  1975 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:59.918464 29067 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:05:59.919031 29072 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:05:59.916857 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:59.921494 29068 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:05:59.922986 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.929069 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:05:59.931303 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.910694  1961 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.876520  1855 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.930079 29068 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:05:59.938272 29067 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:05:59.919526  2009 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:59.898263  1943 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:05:59.944564  1746 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:59.931193  1629 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:59.947508 29072 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:59.965576 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:59.898394  1810 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:05:59.985805 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.968209 29070 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:05:59.937718  1552 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:59.989923 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:05:59.995260 29070 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:05:59.995262 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:05:59.939311  1975 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:59.987452  1918 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:59.993495  1805 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.000974 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.007215 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.008806 29070 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:05:59.971570  1629 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:05:59.983750  2012 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:05:59.983563  2009 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:05:59.994292  1986 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:05:59.940229  1961 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.009838 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:00.032793 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.034240 29067 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:00.041522 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:00.042997 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:00.011137  1364 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:06:00.046762 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 51579.1us
I20260709 13:06:00.045120 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.045257 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:00.041834 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:00.019434  1917 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:00.054843 29142 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:00.049921 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:00.050516 29066 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:06:00.032655  1731 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:00.073747 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.051524 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:00.051627 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:00.012825  1805 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.079080 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:00.080616 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.084122 29068 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:00.081240  1918 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:00.083082 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:00.091327 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.091707 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:00.092940 29142 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:06:00.091344 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:00.095517 29141 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 11: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:00.097010 29145 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:00.092478 29073 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:00.038194  1336 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:06:00.111146 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:00.112177  2027 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:06:00.113042  2022 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:06:00.114369 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.109512  1578 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:06:00.118395  1578 consensus_queue.cc:1243] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:00.119493  1938 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:00.109316  1986 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:06:00.036775  2012 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.125363 29146 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:00.131309 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.132418 29068 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:00.108357  1731 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:00.073567  1746 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:06:00.161033  1746 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:00.161600  1746 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.162071  1746 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:00.150823 29143 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:06:00.153235  1917 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:00.164664 29147 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:00.156502  2012 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:00.158991  1731 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.099485  1931 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:06:00.126451  1751 raft_consensus.cc:2764] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:06:00.169376 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.176133 29068 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:00.133082  1950 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:00.179939 29068 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:00.150650  1986 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:00.194868 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.195818 29068 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:00.192467  2030 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.204751 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:00.198323  1849 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.568s	user 0.010s	sys 0.003s
I20260709 13:06:00.122961  1975 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.189913  1950 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:00.210290 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 131306us
I20260709 13:06:00.207647 29068 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:00.082423  2009 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.127408  1629 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.194047  1986 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:00.212702 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:00.216348 29067 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:00.203148  1727 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.207020  1810 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:00.211046  2030 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.211385  1849 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.222630 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:00.222090 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 105738us
I20260709 13:06:00.223397 29147 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:06:00.224452 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.249713 29067 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:00.250906  1805 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:00.225020  1727 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.254570  1727 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:00.227140  1961 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:00.256883  1961 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:00.256630  1950 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.258477  1950 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.227574  1629 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.260677 29064 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:00.261658 29111 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.262928 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.263177  1756 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:00.263942  1756 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:00.264506 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.269222  2030 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:00.271226  2014 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.271833  2014 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.272233 29147 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:00.274755  2014 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:00.274823  1746 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.278390  2015 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.281347 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:00.319237 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:00.239665  2020 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.246s	user 0.000s	sys 0.002s
I20260709 13:06:00.232438 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.324658 29072 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:06:00.229389  2026 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:06:00.243532  1733 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:00.311633 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:00.242523  1849 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:00.313446  2030 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:00.323400 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.227682  1850 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:00.326680 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.324056  1727 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:00.343590 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 130969us
I20260709 13:06:00.324077  2012 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:00.330724  2015 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.314342  1943 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:00.332034 29905 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 10.
W20260709 13:06:00.333304 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 222233us
I20260709 13:06:00.336261 29907 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:00.362871 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:00.363042 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.304400 29067 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:00.361301 29039 leader_election.cc:400] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:00.363350 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:00.363696 29039 leader_election.cc:403] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:06:00.363704 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:00.359172 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:00.364780 29911 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:00.368450 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.370858 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:00.338285  1850 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:00.379565 29907 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:00.381769 29072 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:00.371896 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:00.348855  1955 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:00.377084 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 94726.9us
I20260709 13:06:00.386298 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.358726  1727 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.240397  1986 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:00.438172 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 93784.6us
I20260709 13:06:00.402812 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.444059 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:00.361321  1731 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:00.376737  1653 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:00.445384 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:00.340231  2030 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.373199  1751 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:00.452916 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.361737  1943 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:00.373322  1574 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:00.374717  1989 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:00.458662 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:00.387581  2012 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:00.475340 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 66293.7us
I20260709 13:06:00.478091  1855 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:00.387655  2040 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:00.479353  1938 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:00.390237  1727 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:00.390089  1918 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:00.445060  1756 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:00.469240  1639 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:00.537278 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.395809  1733 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.122s
I20260709 13:06:00.450278  2032 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:00.456734  1751 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:00.514655 29911 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:00.497717  1989 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.374156  1948 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:06:00.376672  1580 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 13:06:00.451727  1653 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.454298  1961 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:00.456511  1810 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:00.446830  2042 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:00.479166 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:00.566002  1989 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:00.540241 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 96299.4us
I20260709 13:06:00.622212 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:00.612056  2050 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:06:00.542407 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.564169 29903 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
W20260709 13:06:00.624498  2044 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:06:00.625087  1943 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.528796  2051 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
I20260709 13:06:00.552232 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:00.517207  1986 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:00.521481  1855 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.652185  1855 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.653831 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.631659  1989 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.577032 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.655098 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.604637 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.540480  1639 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.438886  1955 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.565433  2040 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.568557  1751 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:00.631392  2041 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 13:06:00.523981  2023 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.541s	user 0.002s	sys 0.002s
I20260709 13:06:00.567557 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.693962 29072 raft_consensus.cc:2368] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 12. Current term is 13.
I20260709 13:06:00.671626  1917 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:00.625171  2042 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:00.567653  1574 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:00.632721 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 153668us
I20260709 13:06:00.612746  2012 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.644801  1746 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:00.646608  2030 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:00.631986  1975 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.648291 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.649421 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.706368 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.706995 29069 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:00.649644 29144 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:00.708315 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:00.649739 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.709051 29219 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:00.650345 29082 tablet_copy_service.cc:342] P a936878f59c840a6bd176a1d2d7347b1: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-b9574de46fcd490e84208e41907200b0 received from {username='slave'} at 127.0.0.1:40862
I20260709 13:06:00.654501 29064 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:06:00.655316 29068 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:00.656014  1639 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:00.533177  1918 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.656414 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:00.614087 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.697060  1917 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:00.664585  2014 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:00.663594  1989 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:00.664157  1933 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:06:00.531334  1850 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.668555  1756 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:00.632203  2056 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.670156  2039 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:00.575886  1653 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:00.655737 29073 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:06:00.686180 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:00.749117 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 63025.6us
I20260709 13:06:00.750092 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:00.626324  1943 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:00.756150  2009 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:00.757022  2009 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:00.758306  2009 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:00.664647  2045 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:06:00.698643 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:00.760679 29905 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:00.755611  1763 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:00.698681 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:00.698653 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 76548.7us
I20260709 13:06:00.769860 29039 leader_election.cc:400] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16.
I20260709 13:06:00.770188 29039 leader_election.cc:403] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:00.698318 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:00.772450 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 74221.5us
I20260709 13:06:00.774806 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:00.701248 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:00.775616 29072 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:06:00.709501 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:00.699381  1574 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.714005 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.705241  1552 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.715421 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:00.715714 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.716493 29082 tablet_copy_service.cc:434] P a936878f59c840a6bd176a1d2d7347b1: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-b9574de46fcd490e84208e41907200b0 on tablet b9574de46fcd490e84208e41907200b0 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:00.718894 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:00.714531 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.656970  2035 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:00.722659 29871 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.699666  2012 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.718077 29065 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:00.723629 29215 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:00.724792 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:00.710639  1975 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.726137 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 832) took 1150 ms. Trace:
I20260709 13:06:00.717811  1629 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:00.717239  2040 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:00.732223 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:00.736862 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 184774us
W20260709 13:06:00.797922 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 403608us
I20260709 13:06:00.736968  1746 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:00.737689  1961 raft_consensus.cc:1027] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: attempt to promote peer 7e78ffbcc98242068a874a5d9e341477: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:06:00.738006  2057 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:00.738744 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 162506us
I20260709 13:06:00.699898  2042 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:00.635123  1947 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:06:00.775835 29903 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:00.713017 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.776805 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:00.820139 29069 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:00.820325 29214 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:06:00.776995 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.778358  1727 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.746575  1756 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.780714 29905 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:00.723615  1918 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:00.830338 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.786497 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.794672 29221 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:00.831539 29907 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:00.799952 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:00.805224 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 82633.9us
I20260709 13:06:00.832432 29216 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:06:00.834275  1763 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.801728  2014 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:00.821105 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.824186 29064 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:06:00.825462 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:00.825793 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:00.796092 29213 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:00.797216 29148 rpcz_store.cc:276] 0709 13:05:59.575658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.575799 (+   141us) service_pool.cc:224] Handling call
0709 13:05:59.587372 (+ 11573us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:59.587727 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:59.587758 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:59.616634 (+ 28876us) log.cc:844] Serialized 52 byte log entry
0709 13:05:59.866634 (+250000us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:59.866679 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.866737 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:00.287761 (+421024us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:00.717369 (+429608us) raft_consensus.cc:1685] finished
0709 13:06:00.717391 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:00.726068 (+  8677us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11814,"raft.queue_time_us":8436,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":3854720,"thread_start_us":161310,"threads_started":1,"wal-append.queue_time_us":218391}
I20260709 13:06:00.797735  1965 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:00.878160 29143 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:00.784161  1850 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.828735 29188 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.802862  2010 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:00.881302 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:00.838042 29139 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:06:00.879312 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:00.881665 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:00.809085  2048 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.774833  2056 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.837181 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.877931 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.879235 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:00.880482 29911 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:00.910786 29065 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:06:00.881106 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:00.879774 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 79916.3us
W20260709 13:06:00.868912 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.795573  2059 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16.
I20260709 13:06:00.879371 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.913470 29064 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:00.883395 29069 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:00.836031 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:00.886874  2061 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:00.814819  1746 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:00.910826 29072 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:06:00.802596  1733 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:06:00.920151  2061 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.920650  2061 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.911108 29143 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:00.920914  1746 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.924602 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.831645  1756 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:00.925307 29073 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:00.797640  1731 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 13:06:00.927014 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.926995 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:00.918812  2048 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.924780 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:00.802943  2054 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:06:00.939702 29064 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:06:00.797061  1552 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:00.878172  1965 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.936189 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.937088 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.878608  2040 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:00.943367 29069 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:00.943749 29217 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:00.944588 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:00.950920 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:00.947789 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:00.951473 29069 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:00.836145  1629 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 13:06:00.928993  2039 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:00.957561 29213 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:06:00.958565 29064 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 13:06:00.964543 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.965269 29069 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:00.834836  1763 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.820199  2057 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:00.975749 29073 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:00.978067  1810 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:01.004195 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:01.005704 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 81016.8us
I20260709 13:06:01.008306 29064 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:06:00.950017  1965 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:01.012822 29064 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:06:00.836061  1917 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:01.020707 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.027660 29071 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:01.025781 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:01.047881 29222 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:01.005167  1954 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.002569  1947 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:01.051376 29139 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:06:01.010447  2059 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:01.059502 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:01.060964 29067 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:01.009956  1810 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:01.066208 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:00.979224  2040 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:01.072242 29217 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:06:01.072140 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:01.074939 29064 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:01.083694 29064 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:01.077418  1552 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:01.079373 29213 raft_consensus.cc:2368] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:01.072926 29217 pending_rounds.cc:85] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 5
I20260709 13:06:01.087188 29217 pending_rounds.cc:107] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:06:01.087476 29217 raft_consensus.cc:2889] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:01.076448  1578 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:01.036310  2014 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:01.086179  1931 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:06:01.091292  2060 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:06:01.091509  2063 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:06:01.091778  2060 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:01.092787  1975 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:01.092693  2060 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.060317  2059 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:01.101073  2056 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:01.101009  2059 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.103617 29067 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:01.010303  2062 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:06:01.063418  1733 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:06:00.784004  1849 consensus_queue.cc:1175] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e78ffbcc98242068a874a5d9e341477" responder_term: 17 status { last_received { term: 17 index: 13 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:01.069685  1810 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:01.144212 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:01.144263 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:01.057775  1855 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:01.149500 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:01.150600 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 321936us
I20260709 13:06:01.152901 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.154083 29217 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:01.107677  1578 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:01.160872 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:01.059865  1954 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.150274 29064 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:01.166256 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.167016 29215 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:01.168274 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:01.155054  2010 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:01.147586  2012 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:01.172955 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.175838 29064 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:01.176663 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 241480us
I20260709 13:06:01.178557 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:01.179175 29909 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:01.157760  1733 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:01.131875  2014 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.158064  1574 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:01.180055 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 152503us
I20260709 13:06:01.181377 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.184185 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.186586 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:01.188222 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:01.191061 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:01.192257 29217 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 11: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:01.148146  1731 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:06:01.196128 29215 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:01.197777 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:01.170924  2066 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:01.175165  1805 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:01.207991 29064 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:06:01.208580 29064 pending_rounds.cc:85] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 7
W20260709 13:06:01.203424 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.208832 29064 pending_rounds.cc:107] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:06:01.201565 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.209111 29064 raft_consensus.cc:2889] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:01.200202 29186 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:01.200568 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.193146  1948 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:01.222438 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:01.230456 29215 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:01.222587 29141 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:01.222150 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:01.199234  1574 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:06:01.226835 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55118.8us
I20260709 13:06:01.208962  1947 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:01.244112  2069 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 13:06:01.247077  1751 consensus_meta.cc:287] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
I20260709 13:06:01.246819  1580 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:01.248014 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:01.275071 29214 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:01.275522 29072 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:01.258426 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.224546 29217 raft_consensus.cc:1217] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 11.7->[13.8-16.10]   Dedup: 13.8->[14.9-16.10]
I20260709 13:06:01.263108 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:01.264353 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 102245us
I20260709 13:06:01.276937 29217 pending_rounds.cc:85] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 8
I20260709 13:06:01.209491  2083 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:01.277251 29217 pending_rounds.cc:107] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:06:01.277542 29217 raft_consensus.cc:2889] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:01.196664  2049 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 13:06:01.291503 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.278910 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.201843  1961 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:06:01.292400 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:01.218564  2071 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:01.293313 29141 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:01.292671 29215 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:01.296489 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:01.297338 29072 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:01.295725 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:01.304461 29215 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:06:01.218618  2056 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:06:01.403391 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.299787  2083 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:01.416113  1763 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:01.402608  1933 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:01.422422  2073 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.166s
I20260709 13:06:01.294000  1733 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.214042  1578 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:01.294844  2057 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:01.299577  2071 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:01.429981  1639 consensus_meta.cc:287] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.690s	user 0.009s	sys 0.002s
I20260709 13:06:01.212846  2066 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.426616  2040 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:01.402750 29221 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:06:01.483373 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.421407  2010 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:01.504536 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 251731us
W20260709 13:06:01.441545  1850 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.532s	user 0.019s	sys 0.005s
I20260709 13:06:01.550586 29215 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 11.
I20260709 13:06:01.436360  2048 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:01.587813 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:01.508509 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.512517 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.456283  1653 consensus_meta.cc:287] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.659s	user 0.009s	sys 0.003s
I20260709 13:06:01.589808  1653 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:01.520654 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:01.591149 29214 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:01.593557 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.500514  1933 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.572999 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:01.575204 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:01.440718  1948 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:01.286161  1751 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:01.538652 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 288461us
I20260709 13:06:01.587808 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.528631  1855 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:01.599004  2070 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
W20260709 13:06:01.608963 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.612205  2077 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
W20260709 13:06:01.612723  2074 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 13:06:01.616775 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 353746us
I20260709 13:06:01.624800 29144 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:01.582409  2012 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:01.626505 29219 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 13: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:01.596370  2084 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.627874 29070 raft_consensus.cc:2410] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 13:06:01.525897  1950 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.026s	sys 0.118s
I20260709 13:06:01.621488  2083 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:01.448442  2039 raft_consensus.cc:2764] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:06:01.532204  1639 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:01.629761  1943 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.019s	sys 0.045s
I20260709 13:06:01.638264 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:01.708861 29141 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:01.711529 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.714519 29188 leader_election.cc:400] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15.
I20260709 13:06:01.715485 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:01.751103 29221 raft_consensus.cc:2410] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:06:01.623634  1931 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:01.531111  1954 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:01.523643  2009 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:01.784828 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:01.678380 29222 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.618s	user 0.011s	sys 0.000s
I20260709 13:06:01.741671 29907 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:01.768036 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:01.786458 29222 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:01.713996  1933 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:01.637233 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:01.792788 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.748049 29186 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.013s	sys 0.027s
I20260709 13:06:01.756654 29218 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:01.793905 29186 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.013s	sys 0.027s
I20260709 13:06:01.747669 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:01.711078  1950 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.027s	sys 0.120s
W20260709 13:06:01.793241 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 156128us
I20260709 13:06:01.794831 29215 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:01.760825 29214 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
W20260709 13:06:01.630374  1986 consensus_meta.cc:287] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.974s	user 0.002s	sys 0.010s
I20260709 13:06:01.705493  1805 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:01.524008  2071 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:01.743824 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:01.796434  2092 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:01.819700  2092 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:06:01.748939  1943 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.019s	sys 0.046s
I20260709 13:06:01.797851 29140 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:01.799058 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:01.822700 29065 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:01.800591 29188 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:01.727689  1948 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.809360 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2612) took 1100 ms. Trace:
I20260709 13:06:01.809811  2087 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.118s
W20260709 13:06:01.812086  1756 consensus_meta.cc:287] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.869s	user 0.008s	sys 0.004s
I20260709 13:06:01.731052  1574 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
W20260709 13:06:01.812335  2072 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:06:01.813231  1805 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:01.718291  2084 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.816476  1751 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:01.824111 29220 rpcz_store.cc:276] 0709 13:06:00.708544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.709913 (+  1369us) service_pool.cc:224] Handling call
0709 13:06:00.926178 (+216265us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:06:00.926361 (+   183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.941222 (+ 14861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:00.941270 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:00.946579 (+  5309us) log.cc:844] Serialized 26 byte log entry
0709 13:06:01.174403 (+227824us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:01.174431 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.174475 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:01.808938 (+634463us) raft_consensus.cc:1685] finished
0709 13:06:01.808969 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.809278 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14431,"raft.queue_time_us":31273,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":14204,"spinlock_wait_cycles":4983168,"thread_start_us":160527,"threads_started":1,"wal-append.queue_time_us":257595}
I20260709 13:06:01.733174  2056 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:06:01.734925  2012 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.841434 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:01.850050 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 49583us
I20260709 13:06:01.854143 29213 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:01.867538 29188 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:01.826792  1948 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:01.841028  2070 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:06:01.863317 29221 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
W20260709 13:06:01.877146 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 133387us
W20260709 13:06:01.861397  2050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.144s	user 0.023s	sys 0.013s
I20260709 13:06:01.840378  1954 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:01.799185  2009 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.914391 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:01.847975  1756 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:01.859563  2039 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:01.919656 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:01.921721 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:01.862457  2085 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.825330  1931 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.882157  2101 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:01.924214 29909 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:01.917514 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 183240us
I20260709 13:06:01.926450 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:01.882520  1763 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:01.916904  2050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.203s	user 0.024s	sys 0.013s
I20260709 13:06:01.893283  2066 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:01.939488 29220 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:06:01.919382  2042 consensus_meta.cc:287] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.079s	user 0.007s	sys 0.005s
W20260709 13:06:01.913883  2061 consensus_meta.cc:287] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.993s	user 0.013s	sys 0.000s
W20260709 13:06:01.923811 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 172381us
I20260709 13:06:01.931792 29143 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
W20260709 13:06:01.942348 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.899452  1580 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:01.934099 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 66660.8us
I20260709 13:06:01.935076 29871 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:01.935705  2109 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:01.936842 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2560) took 1133 ms. Trace:
I20260709 13:06:01.936343  2112 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:01.954159 29064 rpcz_store.cc:276] 0709 13:06:00.804132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.228609 (+424477us) service_pool.cc:224] Handling call
0709 13:06:01.229194 (+   585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:01.229830 (+   636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:01.229874 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:01.249369 (+ 19495us) log.cc:844] Serialized 26 byte log entry
0709 13:06:01.752467 (+503098us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:01.752508 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.752576 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:01.936586 (+184010us) raft_consensus.cc:1685] finished
0709 13:06:01.936619 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.936784 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":21873,"thread_start_us":497977,"threads_started":1,"wal-append.queue_time_us":684859}
I20260709 13:06:01.957285 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2562) took 1044 ms. Trace:
W20260709 13:06:01.940685 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 145864us
I20260709 13:06:01.937503  1763 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:01.947507  1989 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:01.948686  2030 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:06:01.925251  1954 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.946822  2042 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:01.927296  2099 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:01.939016  2101 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:01.957811 29064 rpcz_store.cc:276] 0709 13:06:00.913206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.956381 (+1043175us) service_pool.cc:224] Handling call
0709 13:06:01.956755 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.956963 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:01.956991 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:01.957037 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:01.957069 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.957126 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.957228 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:02.030522 29147 raft_consensus.cc:1217] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 11.10->[12.11-16.12]   Dedup: 12.11->[16.12-16.12]
I20260709 13:06:01.959753 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:01.956353  2079 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:06:01.978724 29221 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:02.031131 29147 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: [12.11-16.12]
W20260709 13:06:02.032019 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 72314.4us
I20260709 13:06:01.970290 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2614) took 1261 ms. Trace:
I20260709 13:06:01.968421 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2559) took 1165 ms. Trace:
I20260709 13:06:02.034673 29067 rpcz_store.cc:276] 0709 13:06:00.802849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.139704 (+336855us) service_pool.cc:224] Handling call
0709 13:06:01.158637 (+ 18933us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:01.175272 (+ 16635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:01.175309 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:01.180997 (+  5688us) log.cc:844] Serialized 26 byte log entry
0709 13:06:01.636724 (+455727us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:01.636781 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.636882 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:01.968079 (+331197us) raft_consensus.cc:1685] finished
0709 13:06:01.968118 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.968296 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10705,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":7020,"thread_start_us":435430,"threads_started":1,"wal-append.queue_time_us":712043}
I20260709 13:06:01.940135  2050 consensus_queue.cc:1243] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:06:01.953935 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:01.981693 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.031663 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 843) took 1246 ms. Trace:
I20260709 13:06:01.938056  2103 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:02.033221 29222 rpcz_store.cc:276] 0709 13:06:00.709154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.709271 (+   117us) service_pool.cc:224] Handling call
0709 13:06:01.970008 (+1260737us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54002f62d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:01.970176 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624450432}
W20260709 13:06:02.033675 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 168141us
I20260709 13:06:02.089303 29146 rpcz_store.cc:276] 0709 13:06:00.785264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.785688 (+   424us) service_pool.cc:224] Handling call
0709 13:06:00.804827 (+ 19139us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:00.893057 (+ 88230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:00.893080 (+    23us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:00.893508 (+   428us) log.cc:844] Serialized 78 byte log entry
0709 13:06:01.108368 (+214860us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:01.108431 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.108518 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:01.963547 (+855029us) raft_consensus.cc:1685] finished
0709 13:06:01.963569 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.031500 (+ 67931us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30340,"raft.queue_time_us":83905,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"thread_start_us":150633,"threads_started":1,"wal-append.queue_time_us":286219}
W20260709 13:06:02.096232 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 142379us
I20260709 13:06:02.099361 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:02.096702 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 161699us
I20260709 13:06:02.099993 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:02.100566 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.101954  2078 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:02.102385  2101 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:02.104076 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 170710us
I20260709 13:06:02.104544  2042 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:01.930280  2085 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:01.952317  2061 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:01.973198  2108 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:01.988004  1918 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.109s	user 0.003s	sys 0.009s
I20260709 13:06:02.110461 29871 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:02.110997 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 316642us
I20260709 13:06:02.030192  1989 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:02.042721  1763 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:02.115702 29113 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:02.121690 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:02.107738  2109 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:02.042501  1931 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:02.115281  2112 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.114499  2084 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.113435  1639 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:02.168383 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 69072.5us
I20260709 13:06:02.138125 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:02.164462 29216 raft_consensus.cc:2368] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:02.165495  1727 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:02.119339  2060 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:02.129163  2108 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.173949 29038 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:02.129462  1810 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.971s	user 0.008s	sys 0.004s
I20260709 13:06:02.182266  2071 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.134593  2062 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:06:02.185359 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 69760.6us
I20260709 13:06:02.130450  2099 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:02.187086 29038 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:02.168536  1931 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:02.193475 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:02.134549  2104 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:02.195422  2111 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:02.196702  2073 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:02.174794  2084 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:02.200667  2114 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:06:02.202555  2042 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:02.188166  2010 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:02.184168  2113 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:02.204710  2012 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.187691  2103 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:02.206214  1653 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:02.209036  1955 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:02.209455  1933 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:02.211771 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.212744  1933 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:02.169155  2112 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.131570  1918 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.214998  1918 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.215778 29066 raft_consensus.cc:2368] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12.
I20260709 13:06:02.217556  2085 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.174311  2115 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:06:02.220054 29113 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:02.215500  2112 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:02.225775 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:02.233457 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 64132.4us
I20260709 13:06:02.230917 29065 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:02.241958 29067 raft_consensus.cc:1217] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 7.4->[9.5-16.8]   Dedup: 9.5->[12.6-16.8]
I20260709 13:06:02.242787 29067 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: [9.5-16.8]
I20260709 13:06:02.245930 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:02.246469 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:02.248576 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:02.250074 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:02.167465  1948 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:02.189167  2099 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.296012  2099 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:02.177143  2108 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:02.186229  2109 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:06:02.253232  2078 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:02.264834  1639 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:02.299450  1639 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.299865  1639 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:02.265012  2090 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:02.301168  2090 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:02.301574  2090 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:02.265643  2053 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.304358  2053 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.306249  2061 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:02.306924  2053 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.266263  1855 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:02.307687  1855 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.309958  1855 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.266804  2097 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:02.310743  2097 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.313112  2097 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.268925 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:02.313298  2078 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:02.192773  1810 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:02.314817 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:02.316973 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:02.228951  1989 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:02.317782 29911 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:02.318518  1989 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:02.270983 29148 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:02.321270 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:02.271687 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:02.324986  2124 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:06:02.326561 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2555) took 1526 ms. Trace:
I20260709 13:06:02.326774 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2564) took 1404 ms. Trace:
I20260709 13:06:02.326989 29073 rpcz_store.cc:276] 0709 13:06:00.800481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.935839 (+135358us) service_pool.cc:224] Handling call
0709 13:06:02.326385 (+1390546us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b540031e7d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:02.326499 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66061184}
I20260709 13:06:02.327643 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2642) took 1502 ms. Trace:
I20260709 13:06:02.328217  1727 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:02.328269 29140 rpcz_store.cc:276] 0709 13:06:00.825423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.825573 (+   150us) service_pool.cc:224] Handling call
0709 13:06:02.327344 (+1501771us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b54003d36d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:02.327527 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2113365760}
I20260709 13:06:02.231765  1918 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:02.327186 29064 rpcz_store.cc:276] 0709 13:06:00.922232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.030478 (+1108246us) service_pool.cc:224] Handling call
0709 13:06:02.030832 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:02.083558 (+ 52726us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:06:02.083637 (+    79us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:02.113975 (+ 30338us) log.cc:844] Serialized 26 byte log entry
0709 13:06:02.186435 (+ 72460us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:02.186479 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.186617 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:02.326574 (+139957us) raft_consensus.cc:1685] finished
0709 13:06:02.326595 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.326721 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85449,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":17058,"thread_start_us":19742,"threads_started":1,"wal-append.queue_time_us":126439}
I20260709 13:06:02.262933  2012 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:02.333012 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 87232.4us
I20260709 13:06:02.334210 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:02.275455 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:02.283732  2015 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:02.284718  1653 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:06:02.221578  2104 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:02.287468  2071 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.295287  2062 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:02.273895 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:02.353693 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 180761us
I20260709 13:06:02.331845 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:02.340263 29140 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:06:02.357666 29064 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:06:02.358155 29871 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:02.348119 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:02.349411 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:02.350627 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:02.254908  1965 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.214s	user 0.012s	sys 0.000s
I20260709 13:06:02.335466 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:02.358220 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:02.344611 29073 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:02.361321 29065 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:02.361344 29140 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:02.355943 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:02.262573  1849 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:02.365573 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:02.366990 29038 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:02.368362 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:02.369977 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:02.372952 29073 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:02.373749 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2443) took 1358 ms. Trace:
I20260709 13:06:02.374115 29073 rpcz_store.cc:276] 0709 13:06:01.014960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.372265 (+1357305us) service_pool.cc:224] Handling call
0709 13:06:02.372564 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.373509 (+   945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.373702 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:02.377550 29141 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:02.317026  1954 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.371470 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:02.344837  1810 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:02.351191  1918 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:02.350561  2101 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.386801 29142 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:06:02.384488 29065 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:02.387252 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:02.354339  1653 consensus_queue.cc:1243] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:06:02.393590 29064 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:02.395321 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:02.376569  2062 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:02.396356 29066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 706) took 1827 ms (client timeout 2117 ms). Trace:
W20260709 13:06:02.406961 29066 rpcz_store.cc:269] 0709 13:06:00.568521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.568630 (+   109us) service_pool.cc:224] Handling call
0709 13:06:02.396100 (+1827470us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400187c54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:02.396261 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3587138944}
I20260709 13:06:02.405167 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:02.377630  1727 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:02.400619  2088 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:02.401096  2111 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:02.396013  1954 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:02.401265 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:02.378763  2117 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:02.400015 29142 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:06:02.353336  1850 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:06:02.384960  2115 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:02.413087 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:02.378513  2112 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.354176  2015 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.390276  2083 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:02.428399 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:02.448755 29066 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:02.414992  1653 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:02.371599  1965 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:02.432557 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:02.447011  2083 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:02.428278  1954 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.404074  2101 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:02.425455  2071 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:02.430886  1727 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:02.429416  2115 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:02.522141 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.390903  1763 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.435114  1849 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.092s
W20260709 13:06:02.527225 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 140857us
W20260709 13:06:02.464210 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 95811.1us
I20260709 13:06:02.448119  2066 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:02.447731  2112 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.555773 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:02.617987 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 62334.1us
I20260709 13:06:02.618793 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:02.619064  2112 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:02.559715 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 127249us
W20260709 13:06:02.552626 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 88823.4us
I20260709 13:06:02.543241  2083 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:02.451897  2055 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 13:06:02.563159  2101 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:02.563365  1763 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:02.554423  2115 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:02.564100  2061 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:02.564689 29907 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:02.569880  1965 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:02.551229  2062 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:02.646035  2113 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:02.646514 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2552) took 1846 ms. Trace:
I20260709 13:06:02.646919 29071 rpcz_store.cc:276] 0709 13:06:00.799600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.028856 (+229256us) service_pool.cc:224] Handling call
0709 13:06:01.058819 (+ 29963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.136752 (+ 77933us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:01.152107 (+ 15355us) log.cc:844] Serialized 14 byte log entry
0709 13:06:01.956017 (+803910us) spinlock_profiling.cc:243] Waited 8.66 ms on lock 0x7b180043eb20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a 00007f7836b17a2a
0709 13:06:02.119718 (+163701us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:02.119770 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:02.119818 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.646247 (+526429us) spinlock_profiling.cc:243] Waited 77.7 ms on lock 0x7b54003edfd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:02.646292 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.646429 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5888,"prepare.queue_time_us":15109,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":1824,"raft.queue_time_us":23395,"spinlock_wait_cycles":204097024,"thread_start_us":157529,"threads_started":1,"wal-append.queue_time_us":1031837}
I20260709 13:06:02.648188 29068 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:02.648880 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:02.649497 29071 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:02.648592 29068 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:02.650390 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:02.650475  1917 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:02.650547 29068 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.651340 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2609) took 1943 ms. Trace:
I20260709 13:06:02.651684 29218 rpcz_store.cc:276] 0709 13:06:00.707611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.709546 (+  1935us) service_pool.cc:224] Handling call
0709 13:06:00.709811 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.794018 (+1084207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.651122 (+857104us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001338d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:02.651285 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2406805760}
I20260709 13:06:02.577550  2125 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:02.658191  2125 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.531767  2104 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:02.660964 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 240517us
I20260709 13:06:02.661218 29071 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:02.584780  1954 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.585515 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:02.594743  1918 raft_consensus.cc:1027] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: attempt to promote peer 7e78ffbcc98242068a874a5d9e341477: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:06:02.602509 29066 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
W20260709 13:06:02.605088  2121 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:06:02.616641  2066 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:02.627826 29911 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:06:02.635142 29905 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:06:02.638447  2133 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:02.639035  1653 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:02.656759 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:02.581573  2015 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.665889 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:02.682708 29066 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:06:02.666379 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.675253 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:02.688640 29140 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:02.689978 29145 raft_consensus.cc:2955] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Committing config change with OpId 12.8: config changed from index -1 to 8, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:06:02.686261 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.748530  2078 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:02.655560  2090 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.685424  2105 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:02.684357  2060 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:02.690080  2015 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:02.683465  2104 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:02.681324  2057 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:06:02.680991  2071 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:02.703249  2101 raft_consensus.cc:2955] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Committing config change with OpId 12.8: config changed from index -1 to 8, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:06:02.755043  1639 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:02.752159  2131 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:02.756733  1965 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:02.802560  1965 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.760362  1948 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:02.762137 29144 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:02.763643  2090 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.764240  2105 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:02.809616  2108 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.000s
I20260709 13:06:02.810261  2108 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.764635  2060 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.810643  2105 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.789657 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2441) took 1777 ms. Trace:
I20260709 13:06:02.819878 29067 rpcz_store.cc:276] 0709 13:06:01.011878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.036146 (+1024268us) service_pool.cc:224] Handling call
0709 13:06:02.241622 (+205476us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:06:02.243263 (+  1641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.789425 (+546162us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54003219d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:02.789586 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328068352}
I20260709 13:06:02.791886 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2605) took 1649 ms. Trace:
I20260709 13:06:02.793632 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2621) took 1393 ms. Trace:
I20260709 13:06:02.824707 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:02.779488 29071 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:02.829200 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2445) took 1812 ms. Trace:
I20260709 13:06:02.829617 29071 rpcz_store.cc:276] 0709 13:06:01.016714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.748657 (+1731943us) service_pool.cc:224] Handling call
0709 13:06:02.779235 (+ 30578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.828836 (+ 49601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.829126 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:02.793783 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2641) took 1968 ms. Trace:
I20260709 13:06:02.791105 29066 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
W20260709 13:06:02.788062 29216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2609) took 1614 ms (client timeout 1773 ms). Trace:
I20260709 13:06:02.833508 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:02.831549 29147 rpcz_store.cc:276] 0709 13:06:00.825157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.825921 (+   764us) service_pool.cc:224] Handling call
0709 13:06:02.030287 (+1204366us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:02.032252 (+  1965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.793514 (+761262us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54001cbe50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:02.793706 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2743555328}
I20260709 13:06:02.784955  2054 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:02.765004  2104 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:02.786772  1931 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.558s	user 0.008s	sys 0.002s
I20260709 13:06:02.759577  2078 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.897357  1931 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:02.898419 29910 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:02.899497  2078 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:02.900493 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 75872.8us
W20260709 13:06:02.832423 29216 rpcz_store.cc:269] 0709 13:06:01.174051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.174189 (+   138us) service_pool.cc:224] Handling call
0709 13:06:02.787858 (+1613669us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b540026c0d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:02.788004 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2252686720}
I20260709 13:06:02.822726 29221 rpcz_store.cc:276] 0709 13:06:01.399849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.915236 (+515387us) service_pool.cc:224] Handling call
0709 13:06:01.928754 (+ 13518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.032271 (+103517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.793312 (+761041us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b54000be854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:02.793522 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114460544}
I20260709 13:06:02.821661 29213 rpcz_store.cc:276] 0709 13:06:01.143016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.143948 (+   932us) service_pool.cc:224] Handling call
0709 13:06:02.791688 (+1647740us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54003395d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:02.791816 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1527044992}
I20260709 13:06:02.897163  2054 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:02.902527 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 84173us
I20260709 13:06:02.904621 29903 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:06:02.907624 29910 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:02.902774 29911 raft_consensus.cc:2955] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEARNER]: Committing config change with OpId 12.8: config changed from index -1 to 8, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:06:02.911026 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:02.912880 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:02.777871  1653 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:02.787215  2134 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.916810 29067 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:06:02.765565  2071 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:02.778669  1639 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:02.778832  2125 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:02.846796  1989 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.528s	user 0.014s	sys 0.000s
W20260709 13:06:02.923331 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 89954.2us
I20260709 13:06:02.925638 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2446) took 1908 ms. Trace:
I20260709 13:06:02.953715 29067 rpcz_store.cc:276] 0709 13:06:01.017476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.902391 (+1884915us) service_pool.cc:224] Handling call
0709 13:06:02.916475 (+ 14084us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:02.925074 (+  8599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.925548 (+   474us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:02.900604  2057 consensus_queue.cc:1243] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:02.902031  2104 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:02.908605  1961 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:02.846869 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:02.779525  2120 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:03.053193 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 434443us
I20260709 13:06:02.783383  2131 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.075836  2131 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.076421 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.077520  2131 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:03.078135  2131 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:02.920120  1965 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:02.769059  2103 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:02.919142  1653 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:02.923748  2055 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.936199  2030 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.061977 29066 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:02.942512  2012 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:02.936825  2015 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:03.066483 29069 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:02.942058  2134 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.070864  2125 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:03.070686  2115 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.072532  2071 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.073030  2108 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.059252  2105 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:03.060598  2060 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.056010  2032 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.120189  2032 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.121076  2134 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.122066  2015 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.122668  2032 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.125429  2054 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.112306  2012 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:03.126046  2115 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:03.127103  2012 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:02.781806  2099 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:02.946863  2113 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.132983  2105 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.133796  2113 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.135536  2105 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:03.085769 29070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 708) took 2417 ms (client timeout 1726 ms). Trace:
I20260709 13:06:03.148936 29071 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:03.061590  1961 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:03.149336 29070 rpcz_store.cc:269] 0709 13:06:00.647865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.648027 (+   162us) service_pool.cc:224] Handling call
0709 13:06:03.065213 (+2417186us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b5400383954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:03.065424 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2039841024}
I20260709 13:06:02.769385  2066 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:02.776226  1727 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.087147  2103 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:03.118481  1756 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:03.083511 29142 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.028s	sys 0.108s
I20260709 13:06:03.137749  1578 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:03.166254 29142 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.028s	sys 0.108s
W20260709 13:06:03.166821 29219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2616) took 2443 ms (client timeout 1679 ms). Trace:
I20260709 13:06:03.139583 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 727) took 2460 ms. Trace:
I20260709 13:06:03.168149 29069 rpcz_store.cc:276] 0709 13:06:00.679264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.002262 (+322998us) service_pool.cc:224] Handling call
0709 13:06:03.065818 (+2063556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.101977 (+ 36159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.139341 (+ 37364us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b5400383950. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:03.139484 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4744806656}
I20260709 13:06:03.140743  2071 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:03.102960  2055 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.119779 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2444) took 2104 ms. Trace:
I20260709 13:06:03.144194 29072 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:03.145632 29067 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:03.131644  2099 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.156169 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2449) took 2135 ms. Trace:
I20260709 13:06:03.174669  2115 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:03.167342 29219 rpcz_store.cc:269] 0709 13:06:00.715058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.715189 (+   131us) service_pool.cc:224] Handling call
0709 13:06:03.158068 (+2442879us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b540023c654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:03.158410 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922209024}
I20260709 13:06:03.152704  1961 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:03.187160 29070 rpcz_store.cc:276] 0709 13:06:01.020899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.154947 (+2134048us) service_pool.cc:224] Handling call
0709 13:06:03.155352 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.155600 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.155637 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:03.155693 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:03.155743 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.155816 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.156089 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.103392  2030 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.182161 29073 rpcz_store.cc:276] 0709 13:06:01.015559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.375052 (+1359493us) service_pool.cc:224] Handling call
0709 13:06:02.375453 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:02.536546 (+161093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:02.536588 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:02.556030 (+ 19442us) log.cc:844] Serialized 26 byte log entry
0709 13:06:02.949941 (+393911us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:03.080431 (+130490us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.080703 (+   272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.080774 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.119504 (+ 38730us) raft_consensus.cc:1685] finished
0709 13:06:03.119530 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.119713 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7551,"prepare.queue_time_us":133483,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"raft.queue_time_us":119877,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":13346,"thread_start_us":523972,"threads_started":2,"wal-append.queue_time_us":528107}
I20260709 13:06:03.086755  2120 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:03.257965  2083 consensus_meta.cc:287] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.000s
I20260709 13:06:03.257658  2104 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.258636  2060 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:03.254487 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.089620  1653 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.272754 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:03.266256 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:03.162410  2131 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.115221  2125 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.166417  2113 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.285624 29219 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:03.286830 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.269797 29070 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:03.288056 29219 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:06:03.272254 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.288067 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2568) took 1663 ms. Trace:
I20260709 13:06:03.288956 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.288115 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.289789 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.289647 29070 rpcz_store.cc:276] 0709 13:06:01.627029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.269121 (+1642092us) service_pool.cc:224] Handling call
0709 13:06:03.269446 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.287807 (+ 18361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.288013 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.290553 29218 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:03.290715 29139 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:03.290228 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.291322 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2571) took 1054 ms. Trace:
I20260709 13:06:03.291703 29070 rpcz_store.cc:276] 0709 13:06:02.236599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.290633 (+1054034us) service_pool.cc:224] Handling call
0709 13:06:03.290890 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.291042 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.291066 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:03.291105 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:03.291134 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.291185 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.291277 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:03.292627 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:03.293612 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.293970 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.296126 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:03.297916 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.299139 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.299950 29215 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:03.303259 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.303797 29220 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:06:03.304664 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.308655 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.301079 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.309825 29221 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:03.314514 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.269734  2145 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.317572 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:03.321174 29904 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:03.322441 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.325512 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.326305 29218 raft_consensus.cc:2368] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:03.326725 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:03.327862 29142 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:03.329348  2104 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:03.329561  2138 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.074s	user 0.002s	sys 0.000s
I20260709 13:06:03.332094  1731 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:03.332772  1731 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.335124  1731 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.299232  2078 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:03.337201  2078 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.341737  2120 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.341632  2088 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:03.343667  2032 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.298964  2035 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.302579  2060 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.344410  2032 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.301539  2062 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.345723  2062 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.346472  2060 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:03.347123  2032 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.306972  2099 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.348078  2099 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.350756  2099 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.306699  2105 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:03.344551  2035 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.352802 29215 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:06:03.295758  2049 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:03.362228 29215 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:03.364078 29218 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:03.364192 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.365056 29216 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:03.365367 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:03.363520 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:03.270915  2115 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.319203  1756 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.379082 29148 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:03.347412  2060 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.372963 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.343164  2088 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:06:03.351418  1653 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:03.385035 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:03.339824  2078 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:03.283849  2083 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.359784  2062 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:03.327032  2101 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:40850 (request call id 12) took 18154 ms. Trace:
I20260709 13:06:03.399363  1756 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.431347 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:03.432099 29909 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:03.432279 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.432672  2055 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:03.432839 29215 raft_consensus.cc:2376] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18. Re-sending same reply.
I20260709 13:06:03.433220  1756 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.409000 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:03.433841 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.434217 29147 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 10.
I20260709 13:06:03.435014 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.356014  2105 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:03.410986 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:03.415517 29216 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:03.421192 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:03.359846  2117 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.918s	user 0.013s	sys 0.000s
I20260709 13:06:03.367543  2104 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:06:03.386106  1955 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.428011  2057 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.379402  2049 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.430266  2035 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.428995  2030 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.430099  2101 rpcz_store.cc:276] 0709 13:05:45.144832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.145038 (+   206us) service_pool.cc:224] Handling call
0709 13:05:45.216444 (+ 71406us) log.cc:844] Serialized 506 byte log entry
0709 13:05:46.412068 (+1195624us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b1800305be0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b49d01 00007f7836a86691 00007f7836a66eaa 00007f7836a6dd53 00007f7836a6dcd9 00007f7836a6dc69 00007f7836a6dc31 00007f7836a6cdfd 000000000038f354 000000000038f199
0709 13:06:03.284900 (+16872832us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5875,"raft.queue_time_us":234165,"raft.run_cpu_time_us":45698,"raft.run_wall_time_us":794705,"spinlock_wait_cycles":23248256,"thread_start_us":378485,"threads_started":1,"wal-append.queue_time_us":870211,"wal-append.run_cpu_time_us":3321,"wal-append.run_wall_time_us":1054367}
I20260709 13:06:03.397840 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.410058  2060 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.453181 29148 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:03.457425 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.468324 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.469020 29065 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:03.435752 29215 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 12: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:03.429008 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2454) took 2602 ms. Trace:
I20260709 13:06:03.453044 29141 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:03.440313 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:03.438843 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:03.443400 29145 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:03.353796  2145 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.464361 29218 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 13: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:03.462762 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:03.471210 29143 rpcz_store.cc:276] 0709 13:06:00.826211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.921355 (+ 95144us) service_pool.cc:224] Handling call
0709 13:06:00.921652 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.945858 (+1024206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.428509 (+1482651us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540029f254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:03.428916 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2322528128}
I20260709 13:06:03.475857 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:03.476101 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:03.476707 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:03.477587 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.480109 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.480758 29065 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:03.481786 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.482483 29065 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:03.475383 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.483991 29215 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:03.489212 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:03.478801  2055 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.493573  2055 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.494112 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.495361 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.496984 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.498008 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.498787 29218 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:06:03.466041  2060 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:03.435936  2105 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:03.500171 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.508797 29222 raft_consensus.cc:2410] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:06:03.509610 29216 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:06:03.499882 29218 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:03.490027 29037 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:03.503126 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.506889 29220 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:03.441582  2125 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:03.438094  2115 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.484776  1756 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:03.460286  2049 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.441582  2083 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.456967  2117 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:03.458268  1955 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.512785 29113 leader_election.cc:400] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:03.515753 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:03.430647  1961 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.511128 29215 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:03.462544  1805 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:06:03.489634  1810 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:03.458508  2057 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.463035  2010 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:06:03.463567  2032 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:06:03.532013 29113 leader_election.cc:403] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:06:03.484678  2030 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:03.502311  1965 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:03.535094 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 832) took 1610 ms. Trace:
I20260709 13:06:03.484867  1336 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:06:03.527755  1756 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.544979 29066 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:03.525758  1552 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:03.524091  2055 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:03.541136 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 249232us
I20260709 13:06:03.507513  2104 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:06:03.545989 29140 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:03.552750 29221 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:03.540328 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 229868us
I20260709 13:06:03.543721 29065 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:03.543787 29907 rpcz_store.cc:276] 0709 13:06:01.925233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.925601 (+   368us) service_pool.cc:224] Handling call
0709 13:06:01.925828 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.643142 (+717314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.534685 (+891543us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54007102d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:03.535020 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1468790656}
I20260709 13:06:03.534639  1855 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:03.556468 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:03.534749  1810 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:03.576987 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.580091 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.580966 29065 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
W20260709 13:06:03.550172 29038 connection.cc:256] RPC call timeout handler was delayed by -1.0249915999993391s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
I20260709 13:06:03.550661  1364 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:03.545794  1965 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:03.586689 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.585296 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.592592 29065 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:06:03.543761  2030 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.576975  1855 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.552127  2117 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.549379  1552 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:03.596434 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.603402 29065 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:03.579490  1810 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:03.598476 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:03.579545  2015 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:03.550215  1756 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:03.581971  1805 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:06:03.657676 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 107133us
I20260709 13:06:03.604839 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.604858 29215 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:03.577522  1639 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:06:03.643002  1961 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.597409  2030 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:03.581595  1336 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:06:03.600513  2154 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.672633 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.603204  1855 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:03.607955  2015 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:03.640453  2010 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:03.650213  2115 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.604933  2134 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:03.652258 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2567) took 2028 ms. Trace:
I20260709 13:06:03.606899  1552 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:03.593703  1965 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:03.602208 29213 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:06:03.661873 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.663038 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.703104 29065 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:03.577333  2103 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.605542  2117 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.613863  2101 raft_consensus.cc:1064] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:06:03.676213 29215 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:03.716835 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.717599 29215 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:03.717768 29219 raft_consensus.cc:2410] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:06:03.705102 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.718348 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.718145  2101 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.719091 29215 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:03.719262 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.719810 29213 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:03.674873  2125 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:06:03.722345  2125 consensus_queue.cc:1243] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:06:03.684228  2154 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.703202  2134 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.724897  2098 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:03.725337  2154 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.692116  2010 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.726086  2010 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:03.726404  2042 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:03.726766  2134 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.726838  2042 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:03.727802  2042 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.728830  2042 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:03.729439  2014 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.729257  1855 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.731594  2117 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.699435  2115 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.733618  2115 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:03.734735  1955 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.707875 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.737689 29217 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:03.738685 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.694945 29069 rpcz_store.cc:276] 0709 13:06:01.624033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.185219 (+1561186us) service_pool.cc:224] Handling call
0709 13:06:03.186067 (+   848us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.253410 (+ 67343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.253452 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.270491 (+ 17039us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.480821 (+210330us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:03.480861 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.480928 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.651996 (+171068us) raft_consensus.cc:1685] finished
0709 13:06:03.652026 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.652198 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28610,"raft.queue_time_us":103,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":4914,"thread_start_us":167715,"threads_started":1,"wal-append.queue_time_us":280287}
I20260709 13:06:03.739042 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2474) took 1064 ms. Trace:
I20260709 13:06:03.739410 29140 rpcz_store.cc:276] 0709 13:06:02.674871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.696388 (+ 21517us) service_pool.cc:224] Handling call
0709 13:06:02.696785 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.553869 (+857084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.738763 (+184894us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b5400190ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:03.738984 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952400768}
I20260709 13:06:03.721287 29213 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
W20260709 13:06:03.740538  2158 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:06:03.740868 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.741245 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.741755 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2570) took 1796 ms. Trace:
I20260709 13:06:03.741851 29213 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:03.741999 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.742717 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.742153 29073 rpcz_store.cc:276] 0709 13:06:01.945153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.280212 (+1335059us) service_pool.cc:224] Handling call
0709 13:06:03.280853 (+   641us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.295981 (+ 15128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.296014 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.306413 (+ 10399us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.541913 (+235500us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:03.541967 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.542039 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:03.741560 (+199521us) raft_consensus.cc:1685] finished
0709 13:06:03.741580 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:03.741698 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45880,"raft.queue_time_us":15930,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":6405,"thread_start_us":159211,"threads_started":1,"wal-append.queue_time_us":286307}
I20260709 13:06:03.743624 29213 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:03.752467 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:03.739743 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.753355 29069 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:03.755374  2108 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:03.744395  2015 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.756600  2015 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:03.759124 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.760737  2053 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:03.761152  2053 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:03.761950  2053 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.762894  2053 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:03.766949  2103 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.768493 29069 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
W20260709 13:06:03.769155 29910 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 850) took 1368 ms (client timeout 1516 ms). Trace:
I20260709 13:06:03.769464 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:03.769531 29910 rpcz_store.cc:269] 0709 13:06:02.400865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.400997 (+   132us) service_pool.cc:224] Handling call
0709 13:06:03.768996 (+1367999us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b54007e0254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:03.769077 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105118976}
I20260709 13:06:03.770684 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.770781  1965 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.681679  2145 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.749704 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:03.745626 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2469) took 1391 ms. Trace:
I20260709 13:06:03.772441 29139 rpcz_store.cc:276] 0709 13:06:02.354294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.388417 (+ 34123us) service_pool.cc:224] Handling call
0709 13:06:02.786359 (+397942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.291155 (+504796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.744686 (+453531us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b5400190d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:03.745455 (+   769us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b5400190d50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:03.745574 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2053339264}
I20260709 13:06:03.773036  2145 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.751016 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2448) took 2732 ms. Trace:
I20260709 13:06:03.773586  2145 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.773949 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.773912 29066 rpcz_store.cc:276] 0709 13:06:01.018912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.102004 (+2083092us) service_pool.cc:224] Handling call
0709 13:06:03.108768 (+  6764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.545397 (+436629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.750828 (+205431us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b5400146754. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:03.750963 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1002429696}
I20260709 13:06:03.774621 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:03.775246 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.750919 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 848) took 1371 ms. Trace:
I20260709 13:06:03.721729 29221 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:03.776203 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2442) took 2763 ms. Trace:
I20260709 13:06:03.777140 29215 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:06:03.772082 29911 raft_consensus.cc:2376] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12. Re-sending same reply.
I20260709 13:06:03.777200 29064 rpcz_store.cc:276] 0709 13:06:01.012730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.366246 (+1353516us) service_pool.cc:224] Handling call
0709 13:06:02.393403 (+ 27157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.397652 (+  4249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.776032 (+1378380us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b54002ad850. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:03.776149 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61856000}
I20260709 13:06:03.776190 29905 rpcz_store.cc:276] 0709 13:06:02.379891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.389593 (+  9702us) service_pool.cc:224] Handling call
0709 13:06:02.389890 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.668047 (+278157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.750684 (+1082637us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54005e92d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:03.750853 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":528792448}
I20260709 13:06:03.778524 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.778785 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.779209 29064 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:03.778605  2145 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.779908 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.793015 29064 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:06:03.794317 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.796142 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.796425 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.775239 29907 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:03.797068 29064 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:06:03.781023 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:03.801805 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 836) took 1709 ms. Trace:
I20260709 13:06:03.801949  2040 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:03.755586 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:03.802631  2040 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:03.803309  2103 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:03.803651  2040 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.805370  2103 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:03.806775  1855 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.783464  2057 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:03.787983 29068 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:03.810046 29217 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:03.811095 29907 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEARNER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:06:03.802219 29903 rpcz_store.cc:276] 0709 13:06:02.092563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.092728 (+   165us) service_pool.cc:224] Handling call
0709 13:06:02.093186 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:02.905330 (+812144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.801570 (+896240us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b540051bb54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:03.801751 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1859252480}
I20260709 13:06:03.790210 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.812161  2009 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.701922  1805 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:03.771282 29069 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:03.813835 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.817967 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.819279 29068 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:06:03.785068 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.821336 29068 pending_rounds.cc:85] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 11
I20260709 13:06:03.821651 29068 pending_rounds.cc:107] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:06:03.821949 29068 raft_consensus.cc:2889] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:03.797068 29215 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:06:03.780766  2032 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:03.826682  2032 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:03.828088 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.819010 29064 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:03.819042 29065 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:03.830500 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 735) took 2996 ms. Trace:
I20260709 13:06:03.830713 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:03.824066 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 68566us
I20260709 13:06:03.831136 29068 rpcz_store.cc:276] 0709 13:06:00.834084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.948179 (+1114095us) service_pool.cc:224] Handling call
0709 13:06:01.948510 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:02.648093 (+699583us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540005d320. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a84dc0 00007f7836b0f1c0 00007f7836b10231 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c
0709 13:06:03.787549 (+1139456us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b580092c3d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a690aa 00007f7836aeb447 00007f7836b03fe7 00007f7836b057de 00007f7836b0fb49 00007f7836b10416 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4
0709 13:06:03.830146 (+ 42597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:03.830423 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2194791552}
I20260709 13:06:03.832458 29038 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:03.833305 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.834939 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.834693 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:03.836364 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.813438  1805 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:03.837811  2034 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:06:03.790822  2147 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:03.831585 29065 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:06:03.850152 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:03.844548  1746 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:03.852991 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.824703  1855 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:03.861639  1855 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.854238 29068 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:06:03.867537 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.857779  2032 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:03.859490  2132 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.100s	user 0.001s	sys 0.007s
I20260709 13:06:03.854503  2140 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
W20260709 13:06:03.860422 29144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2459) took 1591 ms (client timeout 2024 ms). Trace:
W20260709 13:06:03.877940 29144 rpcz_store.cc:269] 0709 13:06:02.268989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.314561 (+ 45572us) service_pool.cc:224] Handling call
0709 13:06:03.860218 (+1545657us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54002e8954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:03.860354 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1027282688}
I20260709 13:06:03.840826  2147 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:03.878386 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.835997  1364 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:03.829846  2126 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:06:03.879869 29068 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:03.852730  2117 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.883864 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.885617 29068 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 11.
I20260709 13:06:03.887807 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:03.892648 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.834610  2105 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.890926 29068 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:03.835541  1756 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.855113 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:03.845625  2010 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.837179  2030 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:03.916599 29064 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
W20260709 13:06:03.921367 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 71295.7us
I20260709 13:06:03.952211 29871 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:03.953758 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.954838 29215 raft_consensus.cc:2368] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:03.955381  2108 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:03.955895 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.962306 29215 raft_consensus.cc:2376] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13. Re-sending same reply.
I20260709 13:06:03.961354 29142 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:03.960340 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.964409 29222 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:03.965613 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.966944 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.894167  1855 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.974866 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:03.976537 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.977387 29219 raft_consensus.cc:2376] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20. Re-sending same reply.
I20260709 13:06:03.979936 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:03.973505 29222 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:03.981525  2010 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:03.984012  2105 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:03.960273  2132 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:06:03.992134  2042 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.993031  2042 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.996016  2147 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:03.997504  2147 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:03.997570  1756 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.949030  2126 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:04.002789  1965 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:04.003928 29871 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.636s (SENT)
I20260709 13:06:04.004999  2030 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.879755  1552 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:04.006278 29065 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:04.007301  1552 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:03.893806  2117 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:04.010198  2117 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:04.011405 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:04.012291 29066 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:04.012457 29215 raft_consensus.cc:2410] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:06:03.895545 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:04.013842 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.014366 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:04.013859 29068 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:04.018793 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:04.019388 29907 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:03.956010  2108 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:04.015372  2057 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.030754 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:04.034889 29068 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:03.995795  2042 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:04.040383 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:04.041330 29906 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:04.045140 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.054006 29907 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:04.026433  2088 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:04.057060 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:03.974282  2115 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
W20260709 13:06:04.060933 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 392800us
I20260709 13:06:04.071656 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.072727 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.073886 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.072481 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:04.078490 29140 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:06:04.028228  2053 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:06:04.086944 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.088546 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:04.090430 29144 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:04.092633 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:04.094025 29871 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.727s (SENT) [suppressed 42 similar messages]
I20260709 13:06:04.045614  2057 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:04.126854  2145 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:06:04.140863 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:04.144510  2108 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:04.096597  2166 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:06:04.061830  2054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.015s	sys 0.006s
W20260709 13:06:04.152012  2054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.016s	sys 0.006s
I20260709 13:06:04.081758  2117 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:04.152575  2103 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:04.153146  2117 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:04.154166 29069 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:04.154507  2117 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:04.091650  1810 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:04.160238  2053 consensus_queue.cc:1243] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:06:04.160821  2042 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:04.160800 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:04.161695  2042 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:04.162055 29911 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:04.162799  2042 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:04.095088  1653 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:04.163268  2129 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.501s	user 0.000s	sys 0.009s
I20260709 13:06:04.163873  1653 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:04.166668 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.167480 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 79019.5us
I20260709 13:06:04.168399 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:04.063834  2088 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:04.171617 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 447437us
I20260709 13:06:04.176437 29906 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:04.096305  2060 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
W20260709 13:06:04.184584 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.185384 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.185268 29113 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:04.171849  1855 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:04.171627  2042 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:04.151376  2166 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:04.177956  1364 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:04.181559  1965 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:04.192106  2105 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:04.196130  1948 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:04.208159  1653 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:04.211326  1948 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:04.221705  2088 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:04.222187  1948 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:04.225129 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2561) took 3420 ms. Trace:
I20260709 13:06:04.225608 29072 rpcz_store.cc:276] 0709 13:06:00.805149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.298202 (+493053us) service_pool.cc:224] Handling call
0709 13:06:01.298571 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.184137 (+1885566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.224851 (+1040714us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b5400068bd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:04.225057 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4243700480}
I20260709 13:06:04.227396 29072 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:04.228325 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2583) took 1403 ms. Trace:
I20260709 13:06:04.228741 29071 rpcz_store.cc:276] 0709 13:06:02.824648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.830528 (+  5880us) service_pool.cc:224] Handling call
0709 13:06:04.228123 (+1397595us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540031e7d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:04.228263 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":540240128}
I20260709 13:06:04.230508 29071 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:04.233780 29064 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:04.234488 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2584) took 1406 ms. Trace:
I20260709 13:06:04.234794 29064 rpcz_store.cc:276] 0709 13:06:02.828276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.233176 (+1404900us) service_pool.cc:224] Handling call
0709 13:06:04.233461 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.234230 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.234416 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.238643 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2582) took 1617 ms. Trace:
I20260709 13:06:04.239144 29071 rpcz_store.cc:276] 0709 13:06:02.621177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.229770 (+1608593us) service_pool.cc:224] Handling call
0709 13:06:04.230151 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.230829 (+   678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.238276 (+  7447us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.240095 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2587) took 1291 ms. Trace:
W20260709 13:06:04.245712 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.240136 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2577) took 1853 ms. Trace:
I20260709 13:06:04.253129 29072 rpcz_store.cc:276] 0709 13:06:02.387200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.226873 (+1839673us) service_pool.cc:224] Handling call
0709 13:06:04.227201 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.227802 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.240089 (+ 12287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.254329 29064 rpcz_store.cc:276] 0709 13:06:02.948268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.239189 (+1290921us) service_pool.cc:224] Handling call
0709 13:06:04.239556 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.239730 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.239756 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:04.239795 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:04.239823 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.239888 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.240041 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.201918  1364 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:06:04.235694  1810 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:04.295782 29071 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:04.297876 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2588) took 1003 ms. Trace:
I20260709 13:06:04.298310 29071 rpcz_store.cc:276] 0709 13:06:03.294280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.276814 (+982534us) service_pool.cc:224] Handling call
0709 13:06:04.295536 (+ 18722us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.297565 (+  2029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.297815 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:04.303265  1805 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:04.305555  1805 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:04.307264  1653 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.303966  1364 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:04.322479 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2573) took 2082 ms. Trace:
I20260709 13:06:04.323068 29070 rpcz_store.cc:276] 0709 13:06:02.240099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.292571 (+1052472us) service_pool.cc:224] Handling call
0709 13:06:03.292784 (+   213us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.299290 (+  6506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:03.299320 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:03.299865 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:06:03.750108 (+450243us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b180034fe40. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:06:04.176540 (+426432us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:04.176594 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.176663 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.308093 (+131430us) raft_consensus.cc:1685] finished
0709 13:06:04.308125 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.322396 (+ 14271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38414,"raft.queue_time_us":7023,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":7763,"spinlock_wait_cycles":10257664,"thread_start_us":425835,"threads_started":1,"wal-append.queue_time_us":996255}
W20260709 13:06:04.328326 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.333001 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 238996us
I20260709 13:06:04.309023  1653 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:04.340641 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.342712 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2586) took 1510 ms. Trace:
I20260709 13:06:04.343225 29067 rpcz_store.cc:276] 0709 13:06:02.832360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.056120 (+223760us) service_pool.cc:224] Handling call
0709 13:06:04.342512 (+1286392us) spinlock_profiling.cc:243] Waited 68.2 ms on lock 0x7b54002ad354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:04.342638 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156964096}
W20260709 13:06:04.380658 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.288622  2166 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:04.371459 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 200441us
W20260709 13:06:04.411092 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.371330  1805 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:04.424361 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.387249  2057 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:04.427062  2034 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.437450 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2576) took 2050 ms. Trace:
I20260709 13:06:04.470384 29072 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:04.474153 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2644) took 1178 ms. Trace:
I20260709 13:06:04.474586 29220 rpcz_store.cc:276] 0709 13:06:03.296121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.304439 (+  8318us) service_pool.cc:224] Handling call
0709 13:06:04.473948 (+1169509us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b5400355cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:04.474096 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463589248}
I20260709 13:06:04.475713 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:04.476980 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:04.478019 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:04.478534 29220 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:04.483392 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.484339 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2643) took 1188 ms. Trace:
I20260709 13:06:04.530362 29221 rpcz_store.cc:276] 0709 13:06:03.295841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.351834 (+ 55993us) service_pool.cc:224] Handling call
0709 13:06:04.484120 (+1132286us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54003a93d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:04.484267 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376484096}
I20260709 13:06:04.489202 29905 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:06:04.500885 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.502295  2055 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:04.505323  2059 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:04.559289  2083 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:04.560027  2083 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:04.561149  2083 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:04.563016 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:04.512372  2172 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:06:04.513474 29065 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:06:04.513655  2175 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:06:04.514003 29071 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:04.514564 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:04.521870 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:04.548720 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.569619 29142 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:04.570863 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.550076 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.550313 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.573479 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.574754 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:04.575434 29140 raft_consensus.cc:2393] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:04.553805 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.576862 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.555437 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.578336 29143 raft_consensus.cc:2410] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:06:04.543335  2055 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:04.559458 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:04.511427 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.585402 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.565295 29037 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:04.566824 29065 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 16. Ops: []
I20260709 13:06:04.588333 29065 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:04.589511 29065 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:06:04.589982 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.590651 29065 raft_consensus.cc:1217] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 11.10->[12.11-17.13]   Dedup: 12.11->[15.12-17.13]
I20260709 13:06:04.591140 29065 pending_rounds.cc:85] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 11
I20260709 13:06:04.591399 29065 pending_rounds.cc:107] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:06:04.591641 29065 raft_consensus.cc:2889] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:04.580232 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:04.582438 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 92104us
I20260709 13:06:04.590683 29144 raft_consensus.cc:2393] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:04.596692 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.584273 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:04.597633 29147 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:04.598237 29220 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:04.599346 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:04.601203 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:04.601913 29220 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:04.605209 29221 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:06:04.605741 29221 pending_rounds.cc:85] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 7
I20260709 13:06:04.605981 29221 pending_rounds.cc:107] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:06:04.606209 29221 raft_consensus.cc:2889] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:04.454711  2034 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:04.638299 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:04.640084 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:04.567694  1918 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:04.592056  2054 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:04.596720  2040 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.650094 29144 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:06:04.650022 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 45841.5us
I20260709 13:06:04.663887 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:04.654461 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.438031 29068 rpcz_store.cc:276] 0709 13:06:02.386886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.061130 (+1674244us) service_pool.cc:224] Handling call
0709 13:06:04.080849 (+ 19719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.081109 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:04.371977 (+290868us) log.cc:844] Serialized 14 byte log entry
0709 13:06:04.436993 (+ 65016us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:04.437062 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:04.437125 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.437216 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.437376 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":146,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":4620,"thread_start_us":309988,"threads_started":2}
I20260709 13:06:04.638164  2145 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.681951 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2589) took 1386 ms. Trace:
I20260709 13:06:04.683050 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:04.684764 29144 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:06:04.648168  2172 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:06:04.686319 29185 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:04.684556 29071 rpcz_store.cc:276] 0709 13:06:03.295436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.299287 (+1003851us) service_pool.cc:224] Handling call
0709 13:06:04.513822 (+214535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.568197 (+ 54375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.658848 (+ 90651us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54002cbad0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:04.681874 (+ 23026us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438780672}
I20260709 13:06:04.681391 29220 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:04.644681  2071 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.691437 29220 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:04.603185  2060 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:06:04.695762 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:04.741782 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:04.743950 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48234.5us
I20260709 13:06:04.745229 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:04.647226  2108 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:06:04.653847  1918 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:04.746948 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:04.719549  2182 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:06:04.758234 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.714692  1855 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:04.700474  2147 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:04.785247 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 398881us
I20260709 13:06:04.782770 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:04.670724  2054 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:04.797585 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:04.694123  2145 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:04.685626  2040 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:04.747437  1653 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:06:04.819559 29141 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:04.818348 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:04.775350  2156 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:04.798616  2171 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.361s	user 0.000s	sys 0.002s
I20260709 13:06:04.791144  2182 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.791257  2103 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:04.796892  2147 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:04.683787  2055 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:04.715795  2071 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:04.843097 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.834262  2182 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:04.803766  2057 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 13:06:04.857359  2177 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:06:04.863719  2034 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:04.865970  1918 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:04.819681  2154 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:06:04.867543 29222 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.008s	sys 0.035s
W20260709 13:06:04.904795 29222 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.009s	sys 0.036s
I20260709 13:06:04.904590 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.871044 29144 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:04.876847 29140 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:06:04.878861 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.906844 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.831971  2191 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:04.882238 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:04.882635 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 468407us
W20260709 13:06:04.898797 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 333562us
W20260709 13:06:04.912482 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:04.900003 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 378225us
I20260709 13:06:04.901759 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.840934  2147 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:04.866961  2042 consensus_meta.cc:287] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.002s
I20260709 13:06:04.905737 29147 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:06:04.918149 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:04.934042 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.934741 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:04.935348 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.907900 29144 raft_consensus.cc:2410] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 16 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:06:04.897279  2014 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:04.936777 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.939827 29142 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:06:04.942956 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 358753us
W20260709 13:06:04.901690  2193 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:06:04.948621 29220 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:04.950073 29148 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:04.968935 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.907002 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:04.896740  2009 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:04.975845 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:04.900143  2055 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:05.017656 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:05.013340 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.021116 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 103052us
I20260709 13:06:05.022416 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:05.025184 29147 raft_consensus.cc:1217] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Deduplicated request from leader. Original: 8.6->[11.7-13.8]   Dedup: 11.7->[13.8-13.8]
I20260709 13:06:05.026157 29147 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:06:05.021126  2196 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:05.031489  2196 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:04.904551  2154 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:04.908850  2108 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:04.909660  2191 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:04.946786  2117 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:05.006969  1805 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:05.029137  2034 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.994845  2014 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:05.018805  2009 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.119052 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2598) took 1633 ms. Trace:
I20260709 13:06:05.120952 29070 rpcz_store.cc:276] 0709 13:06:03.485547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.332198 (+846651us) service_pool.cc:224] Handling call
0709 13:06:04.332625 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.338188 (+  5563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.338234 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.369566 (+ 31332us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.627220 (+257654us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:04.627286 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.627365 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.118737 (+491372us) raft_consensus.cc:1685] finished
0709 13:06:05.118768 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.118961 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":388,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":4983,"thread_start_us":255087,"threads_started":1,"wal-append.queue_time_us":312662}
I20260709 13:06:05.036772  2088 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:05.047519  2098 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:05.050783  2154 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:05.116808  1727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.054s	sys 0.086s
W20260709 13:06:05.096820  2177 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:06:05.044407  2196 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:05.136018 29069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.020s	sys 0.022s
I20260709 13:06:05.079308  2191 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:05.160727 29069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.022s	sys 0.022s
I20260709 13:06:05.154363 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2654) took 1185 ms. Trace:
I20260709 13:06:05.171751 29219 rpcz_store.cc:276] 0709 13:06:03.968608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.978307 (+  9699us) service_pool.cc:224] Handling call
0709 13:06:03.978662 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:03.978930 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:03.979269 (+   339us) log.cc:844] Serialized 14 byte log entry
0709 13:06:04.784757 (+805488us) log.cc:844] Serialized 14 byte log entry
0709 13:06:05.153341 (+368584us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:05.153471 (+   130us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:05.153596 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.153983 (+   387us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.154242 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":113123,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":78,"raft.queue_time_us":29086,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":32704,"thread_start_us":1172396,"threads_started":3,"wal-append.queue_time_us":480852}
W20260709 13:06:05.136366  1727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.054s	sys 0.086s
I20260709 13:06:05.124037  2014 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:05.116469  1805 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:05.130945  2009 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:05.059175  2108 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:05.172298  2098 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:05.193670 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:05.099077  2117 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:05.203444 29143 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
W20260709 13:06:05.236944 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 454264us
W20260709 13:06:05.209724 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.237438 29038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.002s	sys 0.028s
I20260709 13:06:05.229996 29147 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
W20260709 13:06:05.238060 29038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.002s	sys 0.028s
I20260709 13:06:05.120224  2034 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:05.232690 29139 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.000s	sys 0.029s
I20260709 13:06:05.199443 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:05.242820 29139 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.000s	sys 0.029s
I20260709 13:06:05.243083 29146 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
W20260709 13:06:05.246192 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.239017 29185 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:05.249109 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:05.242422 29217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.010s	sys 0.019s
W20260709 13:06:05.250703 29217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.010s	sys 0.019s
I20260709 13:06:05.273510 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:05.275056 29065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.000s	sys 0.034s
I20260709 13:06:05.275146 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2658) took 1077 ms. Trace:
W20260709 13:06:05.275557 29065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.000s	sys 0.034s
I20260709 13:06:05.275780 29217 rpcz_store.cc:276] 0709 13:06:04.198197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.198385 (+   188us) service_pool.cc:224] Handling call
0709 13:06:04.198758 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.199036 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:04.596135 (+397099us) log.cc:844] Serialized 14 byte log entry
0709 13:06:05.274803 (+678668us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:05.274853 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:05.274890 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.274948 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.275087 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23213,"prepare.queue_time_us":291352,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"thread_start_us":1029260,"threads_started":2}
I20260709 13:06:05.138684  2088 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:05.289978 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:05.196234  1805 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:05.229018  2054 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:05.290884 29147 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
W20260709 13:06:05.196013  2184 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.010s	sys 0.029s
I20260709 13:06:05.300688 29146 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:06:05.276625  2156 raft_consensus.cc:2764] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:06:05.285409  2192 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.387s	user 0.002s	sys 0.000s
W20260709 13:06:05.289178  2059 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.730s	user 0.014s	sys 0.000s
W20260709 13:06:05.322065 29216 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.015s	sys 0.025s
W20260709 13:06:05.322561 29216 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.015s	sys 0.025s
I20260709 13:06:05.288165  2042 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:05.296717  2040 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:05.348647 29875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.015s	sys 0.028s
I20260709 13:06:05.354503 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:05.355036 29875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.016s	sys 0.028s
W20260709 13:06:05.355768 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 558242us
I20260709 13:06:05.356727 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:05.308981  2071 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:05.294806  2103 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:05.355396 29146 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
W20260709 13:06:05.388119 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 476511us
I20260709 13:06:05.273344  2098 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:05.346369  2042 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:05.388168  2059 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.365640  2206 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:05.414408  1364 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:06:05.407352  2199 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 13:06:05.417315  1756 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:05.424366  2220 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:05.425077  2220 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:05.435254  2083 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:05.365216  2054 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:06:05.437121  2034 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:05.437286  2054 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:05.437923  2054 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:05.438920  2054 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:05.441180 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2672) took 1100 ms. Trace:
I20260709 13:06:05.441812  2059 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:05.444547  2054 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:05.446671 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2455) took 1845 ms. Trace:
I20260709 13:06:05.447208 29066 rpcz_store.cc:276] 0709 13:06:03.601033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.575219 (+974186us) service_pool.cc:224] Handling call
0709 13:06:04.581201 (+  5982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.808610 (+227409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.808656 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:04.808716 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:04.808780 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.446419 (+637639us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54002080d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:05.446469 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.446605 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522340224}
I20260709 13:06:05.450377  1948 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:05.453449  2206 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:05.368821  2012 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:06:05.454537  2088 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:05.441840 29220 rpcz_store.cc:276] 0709 13:06:04.340893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.720611 (+379718us) service_pool.cc:224] Handling call
0709 13:06:04.721331 (+   720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.949405 (+228074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.440917 (+491512us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540009fbd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:05.441113 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521136384}
W20260709 13:06:05.467126 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 444963us
I20260709 13:06:05.469583 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:05.479161 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2676) took 1054 ms. Trace:
I20260709 13:06:05.480105 29217 rpcz_store.cc:276] 0709 13:06:04.426329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.300848 (+874519us) service_pool.cc:224] Handling call
0709 13:06:05.478429 (+177581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.478703 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.478743 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:05.478802 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:05.478847 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.478922 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.479078 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:05.241950 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:05.486331 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 244475us
I20260709 13:06:05.487344 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:05.348466  2207 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:06:05.491669 29220 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:06:05.495209 29038 leader_election.cc:400] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:05.496779 29038 leader_election.cc:403] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:05.498756 29038 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:05.500685 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 227239us
I20260709 13:06:05.500978 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:05.503860 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:05.490067  2195 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.591s	user 0.000s	sys 0.008s
I20260709 13:06:05.523855 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:05.520556 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51255.5us
I20260709 13:06:05.406741  2042 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:05.517949  2209 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.102s	user 0.001s	sys 0.001s
W20260709 13:06:05.577787 29871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.003s	sys 0.010s
W20260709 13:06:05.578426 29871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.003s	sys 0.010s
W20260709 13:06:05.578823 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 555026us
W20260709 13:06:05.522037  2198 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.332s	user 0.003s	sys 0.000s
W20260709 13:06:05.522173  2215 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.002s
W20260709 13:06:05.518253  2208 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.125s	user 0.001s	sys 0.002s
W20260709 13:06:05.518164  2201 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.098s	user 0.000s	sys 0.002s
I20260709 13:06:05.548305 29217 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:05.553910 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:05.636307 29070 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:05.637360 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:05.638381 29070 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:05.566221 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:05.641083 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:05.642371 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:05.521886  2216 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.073s	user 0.001s	sys 0.001s
I20260709 13:06:05.555809 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:05.650643 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:05.277447  2115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.000s	sys 0.034s
I20260709 13:06:05.521854  1918 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:05.517995  2218 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.067s	user 0.000s	sys 0.002s
I20260709 13:06:05.558945 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:05.518913  2088 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:05.522471  2211 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:05.706171  2211 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:05.518435  2219 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:05.558437  2213 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:05.523644  2222 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:05.551730  2059 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:05.569286  2166 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.005s	sys 0.008s
I20260709 13:06:05.558772  2220 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:05.617445 29072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.017s	sys 0.034s
W20260709 13:06:05.717643 29072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.017s	sys 0.034s
I20260709 13:06:05.490203  2103 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:05.523104  2034 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 13:06:05.639611 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:05.731774 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.033s	sys 0.072s
I20260709 13:06:05.615195  2117 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:05.614521  2214 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:05.661406 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2665) took 1379 ms. Trace:
W20260709 13:06:05.662588  2200 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 13:06:05.615386  2108 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:05.623353  2223 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:06:05.681350  2055 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.000s
W20260709 13:06:05.682742  2147 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.676s	user 0.011s	sys 0.001s
I20260709 13:06:05.688429  2210 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.690723 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2663) took 1417 ms. Trace:
I20260709 13:06:05.769116 29218 rpcz_store.cc:276] 0709 13:06:04.282579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.700375 (+417796us) service_pool.cc:224] Handling call
0709 13:06:04.743631 (+ 43256us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:04.744163 (+   532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.744208 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:04.775341 (+ 31133us) log.cc:844] Serialized 52 byte log entry
0709 13:06:05.242869 (+467528us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:05.274685 (+ 31816us) log.cc:844] Serialized 14 byte log entry
0709 13:06:05.274882 (+   197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.274950 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.661111 (+386161us) raft_consensus.cc:1685] finished
0709 13:06:05.661133 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.661327 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35451,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":58175,"raft.run_cpu_time_us":468,"raft.run_wall_time_us":29374,"spinlock_wait_cycles":22979968,"thread_start_us":413957,"threads_started":1,"wal-append.queue_time_us":641178}
I20260709 13:06:05.777680 29073 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:05.783281 29221 rpcz_store.cc:276] 0709 13:06:04.273311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.607237 (+333926us) service_pool.cc:224] Handling call
0709 13:06:04.607558 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.643015 (+ 35457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.643057 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.643596 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.908828 (+265232us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:04.908902 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.908991 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.433615 (+524624us) spinlock_profiling.cc:243] Waited 37.1 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:05.689802 (+256187us) raft_consensus.cc:1685] finished
0709 13:06:05.689836 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.690647 (+   811us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42629,"raft.queue_time_us":36412,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":22876,"spinlock_wait_cycles":85387520,"thread_start_us":208731,"threads_started":1,"wal-append.queue_time_us":632039}
I20260709 13:06:05.691177  2212 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:05.695186 29068 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:05.708267 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:05.715159 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 741) took 2118 ms. Trace:
I20260709 13:06:05.788089 29071 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
W20260709 13:06:05.717547  2166 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.006s	sys 0.008s
W20260709 13:06:05.657339  2115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.225s	user 0.000s	sys 0.038s
W20260709 13:06:05.720858 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 354731us
W20260709 13:06:05.711797 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 355154us
I20260709 13:06:05.635447  2060 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.652969 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:05.805756 29217 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:06:05.702776  1918 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:05.758857 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.033s	sys 0.072s
I20260709 13:06:05.645785  2071 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:06:05.710839  2219 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.711148  2213 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.702378  2156 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.731948  2059 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:05.784394  1849 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.779356 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2661) took 1228 ms. Trace:
I20260709 13:06:05.788023 29067 rpcz_store.cc:276] 0709 13:06:03.596772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.344671 (+747899us) service_pool.cc:224] Handling call
0709 13:06:04.372347 (+ 27676us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.372743 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.372784 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.385271 (+ 12487us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.670445 (+285174us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:04.670496 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.670568 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.426274 (+755706us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:05.709699 (+283425us) raft_consensus.cc:1685] finished
0709 13:06:05.709734 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.715070 (+  5336us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4548,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":3907,"spinlock_wait_cycles":6510848,"thread_start_us":284789,"threads_started":1,"wal-append.queue_time_us":445764}
W20260709 13:06:05.792362 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 248179us
I20260709 13:06:05.792580 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:05.790882  2223 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:05.706892  2211 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:05.735062  2220 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.786693  1653 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.814246  1918 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:05.812111 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 836276us
I20260709 13:06:05.812646 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:05.813670 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:05.856146 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:05.764577  2227 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:05.869094  2227 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:05.872677 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 748) took 2274 ms. Trace:
I20260709 13:06:05.856634 29068 raft_consensus.cc:2376] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12. Re-sending same reply.
W20260709 13:06:05.790047  2009 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.555s	user 0.014s	sys 0.000s
W20260709 13:06:05.807685 29219 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.021s	sys 0.033s
I20260709 13:06:05.711840  2088 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:05.820246 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:05.716413  2207 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.889211 29218 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
W20260709 13:06:05.845227 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52719us
I20260709 13:06:05.824293 29139 rpcz_store.cc:276] 0709 13:06:04.551219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.551370 (+   151us) service_pool.cc:224] Handling call
0709 13:06:04.551758 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.552231 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.552285 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.552832 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:06:05.243572 (+690740us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:05.560341 (+316769us) log.cc:844] Serialized 14 byte log entry
0709 13:06:05.561253 (+   912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.561334 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.779089 (+217755us) raft_consensus.cc:1685] finished
0709 13:06:05.779114 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.779287 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":561878,"raft.queue_time_us":1260,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":31148,"thread_start_us":912539,"threads_started":2,"wal-append.queue_time_us":880595}
I20260709 13:06:05.828637  2108 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:05.809531 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:05.859476  2230 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:05.859577  1653 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:05.860558  2214 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.813364  1855 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:06:05.865914  2014 consensus_meta.cc:287] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.000s
I20260709 13:06:05.822934  2156 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:05.816496  2213 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:05.866981  2220 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:05.875577 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:05.875758 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 752) took 1910 ms. Trace:
I20260709 13:06:05.876453 29067 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:05.877800 29219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.021s	sys 0.033s
I20260709 13:06:05.787425  2147 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:05.890350 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:05.813632  2182 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.910s	user 0.016s	sys 0.000s
I20260709 13:06:05.873502 29069 rpcz_store.cc:276] 0709 13:06:03.598231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.492270 (+894039us) service_pool.cc:224] Handling call
0709 13:06:04.492867 (+   597us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.493246 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.493278 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.493821 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 13:06:05.162751 (+668930us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:05.163134 (+   383us) log.cc:844] Serialized 14 byte log entry
0709 13:06:05.163493 (+   359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.163752 (+   259us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.872439 (+708687us) raft_consensus.cc:1685] finished
0709 13:06:05.872462 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.872611 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":243,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":184372,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":13887,"thread_start_us":667948,"threads_started":1,"wal-append.queue_time_us":957660}
I20260709 13:06:05.814500  2060 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:05.829983  1849 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:05.790319  2210 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:05.891793 29071 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:05.846324  2226 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:06.041464  2224 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:06:03.890878 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:05.912451 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 99897.9us
I20260709 13:06:06.047240 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2674) took 1658 ms. Trace:
W20260709 13:06:05.923420 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 65290.4us
I20260709 13:06:05.818035  2237 raft_consensus.cc:2764] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:06:05.816668  2219 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:05.942791 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:06.136300 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.075089 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:06.136747 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.137393 29911 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:06:06.048146 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 157914us
I20260709 13:06:06.048904 29068 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:06:06.139947 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:06.140235 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:06.140775 29218 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:06.059460 29065 rpcz_store.cc:276] 0709 13:06:03.965438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.590187 (+624749us) service_pool.cc:224] Handling call
0709 13:06:04.590484 (+   297us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:05.275953 (+685469us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:05.275984 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:05.276590 (+   606us) log.cc:844] Serialized 52 byte log entry
0709 13:06:05.733733 (+457143us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:05.733786 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.733879 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:05.875478 (+141599us) raft_consensus.cc:1685] finished
0709 13:06:05.875506 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:05.875688 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19525,"raft.queue_time_us":314468,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":29554,"thread_start_us":1124515,"threads_started":2,"wal-append.queue_time_us":583443}
I20260709 13:06:06.071313 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:06.040117  1805 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.005s
I20260709 13:06:06.141922 29069 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
W20260709 13:06:05.942162  2083 consensus_meta.cc:287] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.004s
W20260709 13:06:05.914579  2217 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 13:06:06.048352 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:06.033773  2088 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:05.962723  2061 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:06.143187 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:06.152787 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:06.168618 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:05.827795  2059 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:05.966254  2154 consensus_meta.cc:287] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.812s	user 0.008s	sys 0.005s
W20260709 13:06:06.036047  2202 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.682s	user 0.003s	sys 0.005s
I20260709 13:06:06.074440  2235 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:05.905184  2108 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:05.938124  2098 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:06.142871 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:06.168880  2196 consensus_meta.cc:287] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.978s	user 0.007s	sys 0.007s
I20260709 13:06:06.169723 29145 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
W20260709 13:06:06.134398  2054 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.004s
I20260709 13:06:06.143187 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:05.958062  2042 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:06.210638 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 67513.6us
I20260709 13:06:06.142261 29219 rpcz_store.cc:276] 0709 13:06:04.388599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.174639 (+786040us) service_pool.cc:224] Handling call
0709 13:06:05.175066 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.231132 (+ 56066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.231292 (+   160us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:05.247306 (+ 16014us) log.cc:844] Serialized 26 byte log entry
0709 13:06:06.046682 (+799376us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:06.046733 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.046817 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.046879 (+    62us) raft_consensus.cc:1685] finished
0709 13:06:06.046896 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.047035 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3543,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":14960,"thread_start_us":744622,"threads_started":1,"wal-append.queue_time_us":677072}
I20260709 13:06:06.211939  2042 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:06.192595  2235 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:06.212895  2235 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:06.212963  2083 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:06.213811  2083 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:05.914201  2207 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.216168  2012 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:06.094233  2182 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.217895  2207 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.031909  2223 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.218801  2182 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:06.222680 29221 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:06.226423 29219 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:06:06.227619  2231 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.259s	user 0.000s	sys 0.000s
I20260709 13:06:06.228222  2235 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:06.229681  2239 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:06:06.232982 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:06.233916 29219 raft_consensus.cc:2410] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:06:06.235035  2235 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:06.236071  2235 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:06.237607  1653 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:06.239814  2220 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.236449  2235 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:06.240805  2235 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:06.242841  2210 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:06.162607  2184 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:06.163347  1805 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.147745  2228 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:06:06.079949 29874 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.002s	sys 0.011s
W20260709 13:06:06.037426  2191 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.850s	user 0.014s	sys 0.000s
I20260709 13:06:06.072474  2226 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:05.892475  2009 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.196188 29071 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
W20260709 13:06:06.187637  2084 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.008s	sys 0.018s
I20260709 13:06:06.193176  2098 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:06.201489 29069 raft_consensus.cc:2410] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:06.202821  2059 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.205035  2147 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:06.067507  2230 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:06.206367  1948 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.755s	user 0.014s	sys 0.000s
I20260709 13:06:06.067246  2014 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.193569  2108 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:06.209057  2196 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.057366  2040 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.215004  2083 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.272346  2182 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:06.297372  2105 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.019s	sys 0.035s
W20260709 13:06:06.271677 29874 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.009s	sys 0.017s
I20260709 13:06:06.179554  2071 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:06.276636 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:06.172578  2061 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:06.186357  2088 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:06.165807  2227 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:06.281679 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 296751us
I20260709 13:06:06.071858  2214 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.246383  1805 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:06.302356 29220 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:06.199604  2115 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:06:06.274207  2228 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:06.183494  2154 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:06.279731  2084 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.008s	sys 0.018s
I20260709 13:06:06.170094 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:06.374262 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 204217us
I20260709 13:06:06.310896  2230 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:06.311657 29145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2669) took 1754 ms (client timeout 2207 ms). Trace:
I20260709 13:06:06.320600  2009 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:06.322809  2014 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.325299 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 871) took 2592 ms. Trace:
I20260709 13:06:06.382720 29216 rpcz_store.cc:276] 0709 13:06:03.732583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.742800 (+ 10217us) service_pool.cc:224] Handling call
0709 13:06:04.484798 (+741998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.543826 (+ 59028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.543871 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.589423 (+ 45552us) log.cc:844] Serialized 26 byte log entry
0709 13:06:05.322996 (+733573us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:05.323041 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.323112 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.241126 (+918014us) raft_consensus.cc:1685] finished
0709 13:06:06.241148 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.324908 (+ 83760us) spinlock_profiling.cc:243] Waited 742 ms on lock 0x7b5400355cd0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:06.325183 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64841,"raft.queue_time_us":79161,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":21798,"spinlock_wait_cycles":1705742080,"thread_start_us":676876,"threads_started":1,"wal-append.queue_time_us":938332}
I20260709 13:06:06.342403  2182 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.327639 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2494) took 1307 ms. Trace:
I20260709 13:06:06.386637 29148 rpcz_store.cc:276] 0709 13:06:05.020876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.039234 (+ 18358us) service_pool.cc:224] Handling call
0709 13:06:06.326873 (+1287639us) spinlock_profiling.cc:243] Waited 6.1 ms on lock 0x7b5800003098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:06:06.327110 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.327319 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:06.327347 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:06.327390 (+    43us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:06.327427 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.327481 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.327582 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14022528}
I20260709 13:06:06.329221  2196 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.341828  2040 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.333259  2250 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.279601  2098 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:06.380544  2220 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:06.339677 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 784785us
I20260709 13:06:06.340111 29064 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:06:06.343953 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 205786us
I20260709 13:06:06.397784 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:06.398000 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:06.275604  2191 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:06.339257  2105 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.022s	sys 0.036s
I20260709 13:06:06.343511  2071 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.376560 29067 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:06.375159 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:06.403251 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:06.378221 29145 rpcz_store.cc:269] 0709 13:06:04.557155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.576484 (+ 19329us) service_pool.cc:224] Handling call
0709 13:06:06.311462 (+1734978us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54003a57d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:06.311592 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3661640064}
I20260709 13:06:06.384649 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:06.326623  2178 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.893s	user 0.004s	sys 0.008s
W20260709 13:06:06.330657 29073 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 739) took 2734 ms (client timeout 1636 ms). Trace:
I20260709 13:06:06.380344  2055 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:06.188534 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:06.412725 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 224254us
I20260709 13:06:06.350562 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:06.400753 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:06.407663 29073 rpcz_store.cc:269] 0709 13:06:03.596250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.773715 (+177465us) service_pool.cc:224] Handling call
0709 13:06:06.330450 (+2556735us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400186854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:06.330574 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3585796352}
I20260709 13:06:06.414110 29071 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:06.396243  1918 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.002s
I20260709 13:06:06.420337 29213 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:06.428200 29875 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:06.428556 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:06.434331 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:06.435043 29073 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:06.404793  2071 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:06.404461  2196 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:06.407985  2088 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:06.415413  2055 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.419556  2246 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:06.436902  2115 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:06.424283  2084 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:06.466857 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:06.403306  2220 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.457554 29071 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:06.423692  1948 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:06.465322 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:06.453280 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:06.467578 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:06.430169  2227 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.750470 29148 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:06.452759  2156 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:06.491768 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:06.644789 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2665) took 2089 ms. Trace:
I20260709 13:06:06.608047  2108 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.489054 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:06.650166  1849 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:06.654673 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2473) took 1120 ms. Trace:
W20260709 13:06:06.608985 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.460269  2246 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:06.633788  2145 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:06.552485  2154 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:06.441658  2088 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.531674 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2655) took 2530 ms. Trace:
I20260709 13:06:06.563884 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2673) took 2177 ms. Trace:
I20260709 13:06:06.641336  2219 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.632543  2253 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:06.459878  2055 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:06.437124  2125 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:06.501605  2191 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:06.514078  2185 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.715s	user 0.003s	sys 0.000s
I20260709 13:06:06.562222  2060 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:06.393340  2228 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.603816  2057 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.391219  2061 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.493397  2220 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.531462  2105 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.571815  1727 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:06:06.469465  2115 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.467685  2042 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:06:06.396647  2250 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.489626  2213 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.543102  2182 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:06.450253  2252 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:06:06.449582  2249 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
W20260709 13:06:06.494555  2211 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.622s	user 0.009s	sys 0.000s
I20260709 13:06:06.495131  2214 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.467264  2084 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.442358  2083 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:06:06.860486  2057 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.878508  2084 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:06.879285  2071 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.484524  2255 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:06.484918  2196 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:06.880530  2255 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:06.880910  2255 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:06.881433  2071 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:06.881588  2255 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.632067  2226 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.882956  2226 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:06.741680  2098 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.886071  2098 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:06.887593  2219 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.624228  2210 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:06.887992  2211 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.888427  2210 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.889181  2108 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.889293  2250 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.890219  2211 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:06.890441  2210 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:06.572914  2235 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.891041  2213 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.892505  2235 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.726246  2222 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.892869  2115 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.893328  2222 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.866235  2105 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:06.895470  2222 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:06.722810  2204 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.415s	user 0.008s	sys 0.025s
I20260709 13:06:06.869606  2228 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:06.788259  2156 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:06.851383  2246 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:06.895845  2105 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.849664  2088 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:06.864979  2125 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:06.750574  2227 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.726285  2230 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:06.896160  2204 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.594s	user 0.015s	sys 0.033s
W20260709 13:06:06.902575  2257 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.465s	user 0.002s	sys 0.000s
I20260709 13:06:06.864593  2269 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
W20260709 13:06:06.908782  2260 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:06:06.914186  2230 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.915515  2227 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.917078  2230 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:06.605185 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 737) took 3009 ms. Trace:
I20260709 13:06:06.953509 29064 rpcz_store.cc:276] 0709 13:06:03.595665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.339590 (+743925us) service_pool.cc:224] Handling call
0709 13:06:06.336198 (+1996608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.397493 (+ 61295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.604914 (+207421us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b5400186850. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:06.605109 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4585645440}
I20260709 13:06:06.573634 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:06.957496 28967 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 274) took 28500 ms (client timeout 15000 ms). Trace:
I20260709 13:06:06.957636 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:06.959088 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 385364us
I20260709 13:06:06.960779 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:06.961145 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:07.010283 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 49819.5us
I20260709 13:06:06.594475 29073 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:06.477171 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:07.050380  2244 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:07.051209  2244 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:06.483341 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:06.484073 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:06.786348 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:07.103863 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:07.104420 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.108484  2147 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:07.109556  2147 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.110324  2055 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:06.607448 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:06.966202 29218 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:06.966441 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 754) took 1446 ms. Trace:
I20260709 13:06:07.118129  2098 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:06.967067 29142 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:06.969408 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2471) took 1435 ms. Trace:
W20260709 13:06:06.754444 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 301235us
I20260709 13:06:06.850724 29213 rpcz_store.cc:276] 0709 13:06:04.386229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.127632 (+741403us) service_pool.cc:224] Handling call
0709 13:06:05.157187 (+ 29555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.420859 (+1263672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.563524 (+142665us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54002f58d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:06.563820 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2904460160}
I20260709 13:06:06.562132 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:06.726718 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:06.799855 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2660) took 1235 ms. Trace:
I20260709 13:06:07.144704 29220 rpcz_store.cc:276] 0709 13:06:05.564332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.650232 (+ 85900us) service_pool.cc:224] Handling call
0709 13:06:06.799610 (+1149378us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b540016c954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:06.799767 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1496475904}
I20260709 13:06:07.145946 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.147346 29216 raft_consensus.cc:2410] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:06:07.147166 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:06.847048 29222 rpcz_store.cc:276] 0709 13:06:04.000912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.001091 (+   179us) service_pool.cc:224] Handling call
0709 13:06:04.001450 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.002056 (+   606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.002090 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.002531 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.320816 (+318285us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:04.905502 (+584686us) log.cc:844] Serialized 14 byte log entry
0709 13:06:04.914064 (+  8562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.914172 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.307391 (+1393219us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:06.515955 (+208564us) raft_consensus.cc:1685] finished
0709 13:06:06.531414 (+ 15459us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.531605 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65317,"prepare.queue_time_us":169868,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":8660,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":3421,"spinlock_wait_cycles":3442944,"thread_start_us":885759,"threads_started":2,"wal-append.queue_time_us":414893}
I20260709 13:06:07.152824  2040 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:07.153075 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:07.154306 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.155098 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:06.728618 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 128998us
I20260709 13:06:07.162505 29222 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:07.164728 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:06.469273 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:07.174230 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 566695us
I20260709 13:06:06.972520 29141 rpcz_store.cc:276] 0709 13:06:04.555383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.555498 (+   115us) service_pool.cc:224] Handling call
0709 13:06:04.555769 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.824771 (+269002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.644481 (+1819710us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54003b4054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:06.644711 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606074240}
I20260709 13:06:07.174813 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:06.973730 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:07.176717 29217 raft_consensus.cc:2368] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15.
I20260709 13:06:07.177989  2279 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:06.482407 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:06.975246 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:06.963240 29069 rpcz_store.cc:276] 0709 13:06:05.534281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.142640 (+608359us) service_pool.cc:224] Handling call
0709 13:06:06.654392 (+511752us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b540012a054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:06.654597 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21436800}
W20260709 13:06:06.976795 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.181805 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:06.979993 29875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.029s	sys 0.042s
I20260709 13:06:06.989529 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:06.991755 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2476) took 2181 ms. Trace:
W20260709 13:06:06.958472 28967 rpcz_store.cc:269] 0709 13:05:38.456985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:38.477244 (+ 20259us) service_pool.cc:224] Handling call
0709 13:06:06.957077 (+28479833us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:58.616160 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:58.616830 (+   670us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:58.616951 (+   121us) write_op.cc:435] Acquired schema lock
0709 13:05:58.616988 (+    37us) tablet.cc:662] Decoding operations
0709 13:05:58.886726 (+269738us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:58.920413 (+ 33687us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:58.921090 (+   677us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:05:58.977462 (+ 56372us) tablet.cc:701] Row locks acquired
0709 13:05:58.977520 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:05:58.977728 (+   208us) write_op.cc:270] Start()
0709 13:05:58.977921 (+   193us) write_op.cc:276] Timestamp: P: 1783602358977637 usec, L: 0
0709 13:05:58.977957 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:05:59.097304 (+119347us) log.cc:844] Serialized 9720 byte log entry
0709 13:05:59.650072 (+552768us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b180030ad80. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:05:59.849405 (+199333us) op_driver.cc:464] REPLICATION: finished
0709 13:05:59.997520 (+148115us) write_op.cc:301] APPLY: starting
0709 13:06:00.011860 (+ 14340us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:00.630928 (+619068us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:00.630981 (+    53us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:00.673061 (+ 42080us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:00.673780 (+   719us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=48,mrs_lookups=0
0709 13:06:00.673887 (+   107us) write_op.cc:312] APPLY: finished
0709 13:06:00.674831 (+   944us) log.cc:844] Serialized 212 byte log entry
0709 13:06:00.676108 (+  1277us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:00.676529 (+   421us) write_op.cc:454] Released schema lock
0709 13:06:00.699880 (+ 23351us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:00.793905 (+ 94025us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:00.793953 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":38044,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":85973,"apply.run_cpu_time_us":40790,"apply.run_wall_time_us":765522,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104447,"num_ops":24,"prepare.queue_time_us":315833,"prepare.run_cpu_time_us":30161,"prepare.run_wall_time_us":783725,"raft.queue_time_us":181819,"raft.run_cpu_time_us":1885,"raft.run_wall_time_us":12210,"replication_time_us":871301,"spinlock_wait_cycles":8361856,"thread_start_us":527662,"threads_started":4,"wal-append.queue_time_us":289522,"wal-append.run_cpu_time_us":15340,"wal-append.run_wall_time_us":2026281}]]}
W20260709 13:06:06.993320 29070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2474) took 1458 ms (client timeout 1692 ms). Trace:
I20260709 13:06:06.966284 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:07.013177  1653 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:06.426033  1918 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.186818  2059 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:07.187637  2059 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:07.189391  1918 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:07.190675  1653 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:06:06.865947  1727 consensus_queue.cc:1243] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:06:07.191879 29875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.029s	sys 0.043s
W20260709 13:06:07.192548 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 759018us
I20260709 13:06:06.879770  2057 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:06.896133  2061 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:06.900211  2125 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:07.196386 29065 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:07.197021 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:06.605221  2134 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:06:06.751618  1948 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:07.206493 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:06.961975 29064 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:07.040088 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:07.019295  2042 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.051759  2244 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:07.023614  1805 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:07.076845  2083 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.080054 29071 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:06:07.084337  2222 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:07.117131 29066 rpcz_store.cc:276] 0709 13:06:05.520329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.521154 (+   825us) service_pool.cc:224] Handling call
0709 13:06:05.523045 (+  1891us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.874553 (+351508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.874582 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:05.875038 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:06:06.266855 (+391817us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:06.266895 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.266980 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.965825 (+698845us) raft_consensus.cc:1685] finished
0709 13:06:06.965868 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.966183 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29166,"raft.queue_time_us":126306,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":46800,"thread_start_us":713369,"threads_started":2,"wal-append.queue_time_us":489388}
W20260709 13:06:06.969564 29186 connection.cc:256] RPC call timeout handler was delayed by -1.2128295430002254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
I20260709 13:06:07.119091 29067 rpcz_store.cc:276] 0709 13:06:05.533650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.191057 (+657407us) service_pool.cc:224] Handling call
0709 13:06:06.277669 (+ 86612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.402686 (+125017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.968721 (+566035us) spinlock_profiling.cc:243] Waited 98.6 ms on lock 0x7b54001d9054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:06.969170 (+   449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226838912}
I20260709 13:06:07.118784  2098 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:07.212414  2042 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.219784 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:07.220806 29067 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:07.220850  2098 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:07.222477  2098 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:07.215808  2224 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:06:07.219357  2246 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:07.224269  2057 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.224476 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:07.225042  2057 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:07.225626 29066 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:07.226053  2246 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:07.216167  1805 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:07.226912 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:07.140717 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.155316 29218 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:06:06.970908 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2659) took 2769 ms. Trace:
I20260709 13:06:07.158460  2211 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:07.157581 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:06.719282 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 56799.4us
I20260709 13:06:07.164670 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.179656 29214 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 13: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:07.178851 29874 leader_election.cc:400] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15.
I20260709 13:06:07.197701 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:07.183254 29143 rpcz_store.cc:276] 0709 13:06:04.810465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.810926 (+   461us) service_pool.cc:224] Handling call
0709 13:06:04.811356 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.236986 (+425630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.990851 (+1753865us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b540027ced4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:06.991388 (+   537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":900918144}
I20260709 13:06:07.198205 29219 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
W20260709 13:06:07.183512 29070 rpcz_store.cc:269] 0709 13:06:05.534573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.639309 (+104736us) service_pool.cc:224] Handling call
0709 13:06:06.992740 (+1353431us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b54002d3f54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:06.993075 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1530594688}
I20260709 13:06:07.206437  1653 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:07.245680 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:07.233126  2055 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:07.266891 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:07.267534  2265 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:07.268353  2265 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 18 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:07.231864 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.268628 29874 leader_election.cc:403] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:06:07.232558 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.269320 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:07.269790 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 91323.2us
I20260709 13:06:07.270172  2009 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:07.270609 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:07.271248 29218 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:07.222473  2042 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:07.203876 29911 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:06:07.272447 29908 raft_consensus.cc:2410] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:06:07.272917 29065 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:07.216521  2083 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.216866 29073 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:07.219230  2222 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:07.276124 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 60360.2us
I20260709 13:06:07.202402 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.276515  2276 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:06:07.277347 29905 raft_consensus.cc:2410] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:06:07.277501  2276 consensus_queue.cc:1243] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 16.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:07.278904  2224 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:07.242187 29069 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:07.279572  2276 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:07.281522  2276 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:07.228860 28969 catalog_manager.cc:5697] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 9 to 11, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 11 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:07.282728  2276 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.227830  2057 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:07.209596 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 743) took 3612 ms. Trace:
I20260709 13:06:07.279315  2224 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:07.233251 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.235392 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.230455 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.244822 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.209200  1948 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.178978  2147 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:07.269634  2265 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:07.275835 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.276254  2222 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:07.275301 29215 rpcz_store.cc:276] 0709 13:06:04.201131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.201312 (+   181us) service_pool.cc:224] Handling call
0709 13:06:04.201693 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.202056 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.202085 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.202681 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.430395 (+227714us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:04.430438 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.430511 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.513313 (+2082802us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:06.970082 (+456769us) raft_consensus.cc:1685] finished
0709 13:06:06.970125 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.970627 (+   502us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":4232,"spinlock_wait_cycles":10764672,"thread_start_us":222575,"threads_started":1,"wal-append.queue_time_us":482483}
I20260709 13:06:07.208724  2059 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:07.228120  1805 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.285499 29910 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:07.200892 29216 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:07.268663 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.202831 29220 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:07.285583 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.294749 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:07.287607 29904 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:07.286345 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.268672  2014 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:06:07.254336  2084 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.308722 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:07.310003 29070 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:07.310163 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:07.310804  2084 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:07.311029 29066 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:07.312970 29066 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:06:07.313606 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2483) took 1770 ms. Trace:
I20260709 13:06:07.313913 29066 rpcz_store.cc:276] 0709 13:06:05.542929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.312005 (+1769076us) service_pool.cc:224] Handling call
0709 13:06:07.312490 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.313294 (+   804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.313501 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:07.314478 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:07.314932 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:07.316210 29903 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:07.285470  2224 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.317741 29222 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:07.319212  2224 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:07.319294  1805 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:07.320106  2059 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.320729  2059 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.322968  2059 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:07.324281 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 52396.4us
I20260709 13:06:07.326570 29219 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:07.329159 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:07.330080 29112 connection.cc:256] RPC call timeout handler was delayed by -1.1353447470000901s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
W20260709 13:06:07.332020 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 174795us
I20260709 13:06:07.333745 29214 raft_consensus.cc:2410] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:06:07.315675 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:07.332724 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 212992us
I20260709 13:06:07.336890 29073 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:07.338563 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:07.340627 29064 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:07.293397  2276 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:07.343760 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.345270 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:07.314778  2083 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:07.283886 28969 catalog_manager.cc:5697] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:07.348182 29064 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:07.351306 29067 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:07.355729 29220 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:06:07.356349 29220 pending_rounds.cc:85] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 11
I20260709 13:06:07.362036 29214 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:06:07.358371 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:07.369300 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:07.356634 29220 pending_rounds.cc:107] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:06:07.375454 29073 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:07.284492 29072 rpcz_store.cc:276] 0709 13:06:03.597275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.471515 (+874240us) service_pool.cc:224] Handling call
0709 13:06:04.471877 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.534460 (+ 62583us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:04.934282 (+399822us) log.cc:844] Serialized 14 byte log entry
0709 13:06:05.726168 (+791886us) log.cc:844] Serialized 14 byte log entry
0709 13:06:05.726351 (+   183us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:06.652569 (+926218us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b480027c258. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783ba36aa1 00007f783ba2db37 00007f783ba2d9ca 00007f783bb2526f 00007f783bb24f37 00007f783bb3955c 00007f783bb39508 00007f783bb39488 00007f783bb39443 00007f783bb386ec 00007f783aa62a96
0709 13:06:06.985361 (+332792us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b2c000014c8. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:06:06.986625 (+  1264us) log.cc:844] Serialized 26 byte log entry
0709 13:06:06.986895 (+   270us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:07.168340 (+181445us) log.cc:844] Serialized 14 byte log entry
0709 13:06:07.169613 (+  1273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.179989 (+ 10376us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:07.185709 (+  5720us) raft_consensus.cc:1685] finished
0709 13:06:07.185726 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.209395 (+ 23669us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7690,"prepare.queue_time_us":372142,"prepare.run_cpu_time_us":184,"prepare.run_wall_time_us":1780,"raft.queue_time_us":106003,"raft.run_cpu_time_us":3888,"raft.run_wall_time_us":903411,"spinlock_wait_cycles":17212928,"thread_start_us":1188755,"threads_started":3,"wal-append.queue_time_us":802680}
I20260709 13:06:07.374275 29220 raft_consensus.cc:2889] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:07.381354 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 98426.1us
I20260709 13:06:07.382684 29219 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:07.393514 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 60998.6us
I20260709 13:06:07.394687 29064 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:07.392927 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.380487  2211 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:07.388903 29216 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:07.406013 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.418598 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:07.403790  2211 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.410353 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:07.425836  2266 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:07.427722  2125 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:07.428673 29219 pending_rounds.cc:85] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 6
I20260709 13:06:07.444145 29219 pending_rounds.cc:107] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:06:07.444231 29910 raft_consensus.cc:2368] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:06:07.431226 29072 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:06:07.431396  2255 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:07.436412 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 50909.4us
I20260709 13:06:07.423720 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:07.444510 29219 raft_consensus.cc:2889] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:07.446420 29112 connection.cc:256] RPC call timeout handler was delayed by -1.0100599390007119s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
I20260709 13:06:07.435812 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:07.451097 29071 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:07.450402 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.423179 28969 catalog_manager.cc:5697] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 6 to 15, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 15 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:07.453248 29068 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:06:07.445708  2211 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:07.491729 29218 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:06:07.504302  1727 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.443503  2222 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:07.473166 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:07.474539 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 69871.8us
I20260709 13:06:07.475175  1948 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:07.476150 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:07.477246 29068 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:06:07.480312 29909 raft_consensus.cc:2368] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:07.480895 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.480942 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:07.483181 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.483577 28969 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index 5 to 8, term changed from 11 to 12, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129), VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130), VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131), VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 12 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:07.492244 29067 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:07.496141 29906 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 13: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:07.443517  2266 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.481252  1364 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:06:07.423255  2156 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:07.502080  2088 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.500291  2244 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:07.528193 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 59305.9us
I20260709 13:06:07.526325 29073 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:07.520197 29064 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:06:07.523552 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.495419  2125 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:07.538866 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.504906  2147 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:07.538021 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 73398.1us
I20260709 13:06:07.546912 29066 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:07.549258 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.556092  1727 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.563261  1364 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:07.564662 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:07.565862 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 96943.7us
I20260709 13:06:07.572707 29072 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:07.519814  2266 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:07.588291 29908 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:07.588805  2117 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:07.589394  2117 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:07.589497  2244 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:07.589651 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.589912  2117 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:07.591452 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.592170 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:07.592597 29064 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:07.594308  2088 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:06:07.595536 29067 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:06:07.596329 28969 catalog_manager.cc:5697] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 4 to 11, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 11 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:07.598968 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.598409 28969 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:06:07.600323 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:07.573359 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 43661.1us
W20260709 13:06:07.565704  2282 log.cc:927] Time spent T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:06:07.580484  2134 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.585135  2222 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:07.607791  1948 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:07.537388  2009 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15.
I20260709 13:06:07.632602 29907 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:07.593886  1364 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.572115  2147 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.603960 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2687) took 1260 ms. Trace:
I20260709 13:06:07.636248 29214 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:07.636233 29217 rpcz_store.cc:276] 0709 13:06:06.343520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.177752 (+834232us) service_pool.cc:224] Handling call
0709 13:06:07.178370 (+   618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:07.180183 (+  1813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:07.180228 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:07.204374 (+ 24146us) log.cc:844] Serialized 26 byte log entry
0709 13:06:07.351146 (+146772us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:07.351185 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.351255 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:07.603574 (+252319us) raft_consensus.cc:1685] finished
0709 13:06:07.603606 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.603815 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":163,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":10708,"thread_start_us":141929,"threads_started":1,"wal-append.queue_time_us":198119}
I20260709 13:06:07.603896 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:07.638484 29909 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
W20260709 13:06:07.639492 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.559530  1727 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:07.641793 29070 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:07.644452  2145 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:07.645035  2145 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:07.646617 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:07.648770 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.650238 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.650877 29902 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:07.651398  2040 raft_consensus.cc:2764] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:06:07.616523 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:07.617393 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 222348us
I20260709 13:06:07.617671  2276 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:07.617502  2134 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.620860  2125 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:07.624435 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.625416 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.627245 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.652279 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.615504 29905 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:07.634351 28969 catalog_manager.cc:5697] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:06:07.663512 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.635342  1364 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:07.669363 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.669770  2125 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:07.672825 29073 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:07.631091  1948 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.681550 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.684151 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.687162 29910 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 11: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:07.668663  2276 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.698150 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.699215 28969 catalog_manager.cc:5697] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 11, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 11 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:07.703006 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.696815  2276 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:07.708328 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.686262  2098 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.705080  2042 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.715157  2098 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.716979 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.693982  2265 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.721400 29910 raft_consensus.cc:2368] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:06:07.715698  2042 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:07.727480  2042 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:07.729097  2098 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:07.730002  2134 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:07.731858 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.702266  2059 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:06:07.740340  2125 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:07.744400 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 65856.9us
I20260709 13:06:07.746354  2059 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:07.747663 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2485) took 1331 ms. Trace:
I20260709 13:06:07.747887 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:07.748621 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.748160 29069 rpcz_store.cc:276] 0709 13:06:06.416191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.330187 (+913996us) service_pool.cc:224] Handling call
0709 13:06:07.330688 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:07.331138 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:07.331172 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:07.331746 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:06:07.539301 (+207555us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:07.539337 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.540139 (+   802us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:07.747430 (+207291us) raft_consensus.cc:1685] finished
0709 13:06:07.747449 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.747605 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5093,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":8874,"spinlock_wait_cycles":172288,"thread_start_us":205587,"threads_started":1,"wal-append.queue_time_us":387801}
I20260709 13:06:07.749655 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:07.750798 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:07.752224 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:07.753254 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:07.646644  2145 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.760931  2009 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:07.761127  2211 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:07.761720  2014 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:07.763365 29185 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:07.765147  2228 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:06:07.765268 29112 leader_election.cc:336] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 1.526s (SENT)
I20260709 13:06:07.765686 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.765774  2098 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.766651 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.766855 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.787144 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.767421 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.767551 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.768198 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:07.768885 29069 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:06:07.789674 29906 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 13: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:07.790498 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.764953 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:07.769099 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.769603 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:07.771095 29147 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:07.781814 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.783260 29904 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:07.795619 29908 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:06:07.797829 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.797055 29907 raft_consensus.cc:2368] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:07.827667  2061 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:07.784895  2228 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:07.835486 29071 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:07.797475 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:07.829396 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:07.822628 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.837551 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:07.830242 29142 raft_consensus.cc:2376] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16. Re-sending same reply.
I20260709 13:06:07.796840 28969 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:07.847232 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:07.848173 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60769.1us
I20260709 13:06:07.847374 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:07.849093 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 883) took 1351 ms. Trace:
I20260709 13:06:07.849677 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:07.840924  2061 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:07.850977  2061 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:07.851089  2057 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:07.851656  2057 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:07.852149  2057 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:07.853778  2260 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:06:07.856276  2012 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.772208  2117 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:07.841179 29213 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:07.782464  2255 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 13:06:07.890973 29217 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:07.894989 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.836388  2236 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.849774 29221 rpcz_store.cc:276] 0709 13:06:06.423338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.181531 (+758193us) service_pool.cc:224] Handling call
0709 13:06:07.182102 (+   571us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:07.198759 (+ 16657us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:06:07.387582 (+188823us) log.cc:844] Serialized 14 byte log entry
0709 13:06:07.518263 (+130681us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:07.563003 (+ 44740us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b480021c258. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783ba36aa1 00007f783ba2db37 00007f783ba2d9ca 00007f783bb2526f 00007f783bb24f37 00007f783bb3955c 00007f783bb39508 00007f783bb39488 00007f783bb39443 00007f783bb386ec 00007f783aa62a96
0709 13:06:07.583087 (+ 20084us) log.cc:844] Serialized 52 byte log entry
0709 13:06:07.583259 (+   172us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:07.583494 (+   235us) log.cc:844] Serialized 14 byte log entry
0709 13:06:07.648456 (+ 64962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.648552 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:07.774292 (+125740us) raft_consensus.cc:1685] finished
0709 13:06:07.774313 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.774471 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75342,"prepare.queue_time_us":278023,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":97,"raft.queue_time_us":73712,"raft.run_cpu_time_us":3835,"raft.run_wall_time_us":39480,"spinlock_wait_cycles":6886272,"thread_start_us":245081,"threads_started":2,"wal-append.queue_time_us":174514}
I20260709 13:06:07.795454  2265 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:07.865449  2059 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:07.916153 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:07.917438 29220 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:06:07.920197 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.913040 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:07.934789 29213 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:07.938903 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:07.939882 29069 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:07.940379 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:07.941001 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:07.942138 29216 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:06:07.944717 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55331.6us
W20260709 13:06:07.950635 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 185381us
I20260709 13:06:07.950986 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:07.861912  2098 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:07.954612 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:07.964651  2286 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:06:07.966261 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:07.882233  2012 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.976616  2228 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:07.905387  2156 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:07.916755 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:07.988344 28969 catalog_manager.cc:5697] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 8 to 15, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 15 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:07.994021 29910 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:07.868606  2042 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:06:08.000679 29217 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:08.007153 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.007879 29067 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:08.011960 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:08.014380 29068 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:08.017365 29221 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:08.020570 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.020948 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.021504  2083 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:08.021996 29142 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:08.022360  2083 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:08.023547 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:08.024438 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:08.024922 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:08.029353 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:08.029742 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.030344 29142 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.030985 29146 raft_consensus.cc:2410] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 18 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:06:08.029408 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.032650 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.032965 29141 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:08.034812 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.037475  2083 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:08.038538 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.039331 29146 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
W20260709 13:06:08.046782 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.047071 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:08.047423 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.051245 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:08.063905 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.074565 29070 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:08.054328 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:08.079547 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 330003us
I20260709 13:06:08.075470 29069 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:08.009821  2156 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.093493  2156 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:08.096233 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:08.096993 29068 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:08.097159 29141 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:08.098825  2269 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.191s
I20260709 13:06:08.099818  2269 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:08.100271  2269 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:08.100600  2228 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:08.104236 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:07.981595  2266 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:08.108654 29066 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:08.110474  2266 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:08.110010 29216 raft_consensus.cc:2955] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Committing config change with OpId 12.8: config changed from index -1 to 8, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
W20260709 13:06:08.113222 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.009720  1364 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:08.087087 29142 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:08.090941 28969 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 12 to 13, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 13 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:08.092955  2211 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:08.130182 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.119541 29906 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:08.128815 29139 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:08.137641 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.138087 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.140477 28969 catalog_manager.cc:5697] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 12 to 14, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 14 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:08.142978 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.143247 28969 catalog_manager.cc:5697] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 8 to 9, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 9 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:08.144994  2057 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.145761 28969 catalog_manager.cc:5697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 11, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 11 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:08.147512  2057 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:08.149686  2147 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:08.151233 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.152551 29066 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:08.153311 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.154211 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.156999 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:08.157476 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.156381 28969 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 11 to 15, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 15 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:08.159572  2117 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:08.160733 29068 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:08.160842 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.010027  2279 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 13:06:08.165217 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.176647 29904 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
W20260709 13:06:08.165460 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.003274  2236 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.173288 29064 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:08.178006 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:08.119565  1364 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.113801  2224 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:06:08.174981 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:08.182562 29142 raft_consensus.cc:2393] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:08.182451 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.178200 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.182516 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:08.186125 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 50735.4us
I20260709 13:06:08.187136 29910 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:06:08.188772 29112 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:08.191012 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:08.213974 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 443966us
I20260709 13:06:08.139143  2088 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:08.217813 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:08.219515 29907 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:08.212374 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.222290 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:08.223230 29064 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:08.222590 28969 catalog_manager.cc:5697] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 11 to 12, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 12 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:08.227571 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:08.216789 29065 raft_consensus.cc:2376] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18. Re-sending same reply.
I20260709 13:06:08.231134 29140 raft_consensus.cc:2410] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 18 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:06:08.185833  1364 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:08.174823  2211 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.233259 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:08.235010 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.235684 29908 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:08.238994 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:08.236361 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:08.237754 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 62884.1us
I20260709 13:06:08.241051 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.246691 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.248243 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.249230  2098 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:08.251255 29146 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:06:08.252521 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.253207 29908 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:08.254081 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:08.254328  2236 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:08.248288  2269 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.257170 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:08.257583  2269 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:08.238363 29902 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:08.264330 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:08.264572 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.264568  2265 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:08.266079 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.273479 29905 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 14: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:08.267396 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.275068 29142 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:08.276108  2117 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:08.276649  2117 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.277037  2117 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.217734  2295 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:06:08.278401  2295 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:08.278892  2295 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:08.279093 29145 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:08.280094  2295 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:08.281986  2295 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:08.282744  2061 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.283985 29214 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:08.284157 29141 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:08.284684 29139 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:08.285313 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:08.268466 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.287227  2012 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:08.268743 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.288810 29066 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:08.289229 29906 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:08.289336 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.289930 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.290036 29903 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:08.290830 29907 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:08.291584 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.292268 29903 raft_consensus.cc:2376] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15. Re-sending same reply.
I20260709 13:06:08.292191 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:08.294709  2276 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.000s
I20260709 13:06:08.296241 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:08.296615 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 107906us
I20260709 13:06:08.296968 29145 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:08.297571  2279 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:08.300238  2253 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:08.301657 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:08.303021 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:08.303764 29221 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:08.304297 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.304749 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:08.305497 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.305684 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:08.306322 29070 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:08.306330 29064 raft_consensus.cc:2376] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12. Re-sending same reply.
I20260709 13:06:08.239202  2088 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:08.308267 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:08.308650 29073 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:08.308804 29069 raft_consensus.cc:2376] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14. Re-sending same reply.
I20260709 13:06:08.309950 29073 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:06:08.310197  2156 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.311308 29222 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:08.311681 29142 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:08.311102 28969 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 10 to 11, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 11 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:08.312681 28969 catalog_manager.cc:5697] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 10 to 14, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 14 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:08.314293 28969 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 12 to 13, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 13 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:08.317656 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.318531 29065 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 18 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:06:08.295307  2276 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.322165  2009 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:08.328341 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 72124.5us
I20260709 13:06:08.329941 29906 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:08.324088 29143 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:08.323698 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.329595 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.343847 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:08.346848 29220 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:08.353154 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.355265 29145 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:08.356623 29144 raft_consensus.cc:2368] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13.
I20260709 13:06:08.357733  2276 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:08.358954  2088 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.364252 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:08.365777 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:08.366341 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.365015 28969 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 9 to 14, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 14 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:08.378365 29147 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:08.379629 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.380327 29072 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:08.385663  2061 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:08.398444 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:08.400467 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.326378  2014 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.407574 29903 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:08.408509 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:08.409624  2269 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:08.410243 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:08.414093 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 408202us
I20260709 13:06:08.415091 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.417092 28969 catalog_manager.cc:5697] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 7 to 15, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 15 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:08.421171 29064 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
W20260709 13:06:08.426663 29037 connection.cc:256] RPC call timeout handler was delayed by -1.3058240190002834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
I20260709 13:06:08.428880 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.429947 29903 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:08.432408 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:08.433281 29903 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:08.316841  2211 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:08.324589  2009 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:08.477494 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 79150.2us
I20260709 13:06:08.481338 29871 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:08.487175 28969 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 10 to 11, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 11 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:08.494043 29185 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:08.494405 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 84243us
I20260709 13:06:08.495175 28969 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 12 to 15, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 15 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:06:08.499009 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.455247  2228 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:06:08.519026 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 44442.4us
I20260709 13:06:08.455346  2059 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:08.490671  2224 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
W20260709 13:06:08.546178 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 315177us
I20260709 13:06:08.537897 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:08.519953  2302 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:08.452682  2014 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:08.534363  2057 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:06:08.557427 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 298367us
I20260709 13:06:08.558564 29218 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:08.559289 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:08.518707  2276 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:06:08.567188 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:08.586189 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 51053.5us
I20260709 13:06:08.586424  2009 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:08.577553  1805 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:06:08.587212 29112 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:08.589152 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:08.593972  2211 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:08.594739  2211 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:08.595260  2211 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:08.580248  2057 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:08.596001  2303 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:08.596972  2057 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:08.598585  2308 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:06:08.567226  2306 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:06:08.600749  2042 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:08.601724  2042 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:08.602609  2042 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:08.606873 29112 leader_election.cc:400] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
W20260709 13:06:08.611867 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 61889.9us
W20260709 13:06:08.613869  2309 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:06:08.617578 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2694) took 1329 ms. Trace:
I20260709 13:06:08.618044 29219 rpcz_store.cc:276] 0709 13:06:07.288460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.412078 (+123618us) service_pool.cc:224] Handling call
0709 13:06:07.428297 (+ 16219us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:07.524836 (+ 96539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:07.524865 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:07.542848 (+ 17983us) log.cc:844] Serialized 52 byte log entry
0709 13:06:08.008716 (+465868us) spinlock_profiling.cc:243] Waited 9.43 ms on lock 0x7b18003064c0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:06:08.213876 (+205160us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:08.213917 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.213993 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:08.617196 (+403203us) raft_consensus.cc:1685] finished
0709 13:06:08.617219 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:08.617485 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72323,"raft.queue_time_us":98,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":17582,"spinlock_wait_cycles":21696384,"thread_start_us":204972,"threads_started":1,"wal-append.queue_time_us":858230}
I20260709 13:06:08.616395  2211 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.580487  2156 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:08.570207  2310 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:06:08.629637  2305 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:06:08.569718  2255 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:08.630903  2304 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:08.607452 29112 leader_election.cc:403] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:08.632547 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:08.550119  2059 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:08.561225  2125 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
W20260709 13:06:08.644026 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.644914 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.655491 29906 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:08.656678  2224 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:06:08.657114  2084 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:08.657498  2224 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:08.658423 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:08.658777  2253 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:08.659391  2253 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:08.660019  2312 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:08.660365  2253 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.662009  2253 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:08.663697  2307 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:06:08.664217 29064 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:08.665490 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.633211  2276 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:08.660277 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.636073  2310 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:08.668529 29067 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
W20260709 13:06:08.684712 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 116577us
I20260709 13:06:08.686265 29903 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:08.658694  2224 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:06:08.638741  2304 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:08.726315 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:08.726715 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 68373us
I20260709 13:06:08.736604 29906 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:08.690083 29213 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:06:08.672286  2059 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:08.741395 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 152316us
I20260709 13:06:08.738962 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:08.748482 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.639357  1727 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:06:08.733619 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:08.751907 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 270930us
W20260709 13:06:08.684682  2145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.015s	sys 0.015s
I20260709 13:06:08.753996 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.754796 29907 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:08.738819 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.755898 29911 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:08.757123 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:08.639834  1364 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.752030 29909 raft_consensus.cc:2376] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18. Re-sending same reply.
I20260709 13:06:08.672307  2310 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:08.752737 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 163236us
I20260709 13:06:08.684564  2156 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.734642 29220 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:08.753680 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:08.674346  2117 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.667857  2042 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.695278 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:08.763336 29906 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:06:08.764076 29214 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:08.733487  2269 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:08.754267  2145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.015s	sys 0.015s
W20260709 13:06:08.768864 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 158257us
I20260709 13:06:08.774560 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:08.770344 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:08.739423  2228 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:06:08.789773 29148 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 17. Ops: []
I20260709 13:06:08.752488  2312 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:08.791428 29222 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:08.767228  2156 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.800724 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.792363  2315 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:08.796808  2014 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:08.812223  2319 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.812784  2319 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:08.813738 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60148.8us
I20260709 13:06:08.814685 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:08.814898  2319 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:08.815625 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:08.792539 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:08.793915  2276 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:08.796782 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:08.767263 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:08.818245 29213 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.804562  2042 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:08.806200  2117 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:08.822347 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:08.754099  1805 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:06:08.829314 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:08.831034 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:08.834338 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.816634  2057 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:08.829538 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:08.834911  2014 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:06:08.837327 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.837800 29215 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:08.838018 29067 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:08.830607  2315 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:08.839545  2315 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:08.840049  2315 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.841311 29217 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:08.843897  1364 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:08.844591 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52138.5us
I20260709 13:06:08.845983 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:08.847653  2117 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:08.848538 29214 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:08.852556 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:08.834961  2303 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:06:08.801973  2312 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:08.853598 29072 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:08.852772  2156 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:08.859373  2057 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:08.886238 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 68463.5us
I20260709 13:06:08.855154  2312 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.872584  2236 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:08.875797  2014 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:08.876264 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:08.907802 29906 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:08.877375 29909 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:08.878311 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:08.879889 29907 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:08.863776  2211 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:08.829542  2304 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:08.881119 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51882.6us
I20260709 13:06:08.881383 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:08.882581  2042 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:06:08.882210 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:08.870095 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.884902 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:08.890167 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 60700.5us
I20260709 13:06:08.902973  2236 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:08.932523 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47687.8us
I20260709 13:06:08.932765  2304 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:08.926220 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:08.931803 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 48463.3us
I20260709 13:06:08.909790 29911 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:08.912645 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.912906 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:08.904294 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:08.894268  2057 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:08.935263 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:08.935206 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 119448us
I20260709 13:06:08.952479 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:08.882266  2276 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:08.894556  2125 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:08.917191  2295 consensus_meta.cc:287] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.004s
I20260709 13:06:08.934147  2319 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:08.929112  2312 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.909744  1727 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:08.996605  2332 raft_consensus.cc:2955] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index 8 to 10, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:08.894066  2059 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:06:09.006001  2269 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:06:09.008033 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 72840.2us
I20260709 13:06:08.967741  2253 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.962167  2276 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.003305 29213 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:08.978192  2319 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:09.030431 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 143396us
I20260709 13:06:08.980170  2145 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:08.932893  2236 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.042912  2313 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:09.043785  2313 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.029564  2269 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:09.044415  2269 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:09.045562  2269 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:08.973887  2303 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:06:09.046904  2269 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:09.046109  2327 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:06:09.042932  2236 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.056458  2315 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.056970  2246 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:09.057554  2246 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:09.058537  2246 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:09.059468 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.059540  2266 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:09.060261  2266 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:09.061597 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:09.037539  2319 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.058939  2315 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:09.074400 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.077147 29186 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.078250  2211 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:09.079845  2211 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:09.080727  2211 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:09.081197  2211 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:09.082345  2211 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:09.085808  2125 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:09.086561  2125 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:09.140596 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 79096.1us
W20260709 13:06:09.141440 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 106588us
I20260709 13:06:09.059733  2246 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:09.037771  2276 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:09.001919  2295 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:06:09.158174 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:09.159027 29218 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:09.039616  2145 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:09.170034 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.004693  2088 consensus_queue.cc:1059] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.146s
W20260709 13:06:09.201745 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 124685us
I20260709 13:06:09.202870 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:09.205528 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 270550us
I20260709 13:06:09.174418  2145 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:09.168690  2211 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.221727 29186 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.175151  1805 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:09.172044  1948 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:09.226169 29037 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:09.223008  2266 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:09.237150  2266 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:09.197824  2147 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:09.245486 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 427691us
I20260709 13:06:09.256978 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:09.259105 29218 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
W20260709 13:06:09.261171 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 118495us
I20260709 13:06:09.246626 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:09.269609 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:09.271559 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 130178us
I20260709 13:06:09.272688 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.278499  2326 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:09.279398  2326 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:09.284240  2326 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:09.293474 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 46947.7us
W20260709 13:06:09.298286 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 501579us
I20260709 13:06:09.302454 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:09.240238  1948 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.213573  2319 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:09.233702  1805 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:09.249111  2061 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:09.317433 29215 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:06:09.340057 29904 raft_consensus.cc:2368] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15.
I20260709 13:06:09.341812 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:09.330559 29875 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.533s (SENT) [suppressed 64 similar messages]
W20260709 13:06:09.343029 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 71962.7us
I20260709 13:06:09.345003 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.344710 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:09.346432 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 436645us
I20260709 13:06:09.315675  1948 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:09.349455 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.326808  2310 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.003s
I20260709 13:06:09.270992  2315 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:06:09.387342 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.392360 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:09.398425 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 141582us
I20260709 13:06:09.399283 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:09.363219  2313 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:09.414418 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 72832.6us
I20260709 13:06:09.419857 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:09.420293 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:09.385739  2295 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:09.393992  2342 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:09.389882  2346 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:09.409559  2228 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:09.436923 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:09.417778  2269 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:09.368602  2310 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.413797  1727 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.420881  2083 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:09.447212 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:09.449152 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 49944.4us
W20260709 13:06:09.449429 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 176833us
W20260709 13:06:09.450835 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 106214us
W20260709 13:06:09.451464 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 120937us
I20260709 13:06:09.451795 29112 leader_election.cc:400] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:09.452368 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.426586  2125 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:09.459877 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.438383  2325 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:09.441085  2084 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:09.468452 29213 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:09.440001  2228 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:09.470419 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 248738us
I20260709 13:06:09.443970  2059 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:09.476567  2083 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:09.477188  2338 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:06:09.479368  2341 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:06:09.502027 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 54919.2us
I20260709 13:06:09.502614  2228 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.503091 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:09.504065  2315 raft_consensus.cc:2764] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:06:09.504129  2337 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:06:09.505538 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 54300.6us
I20260709 13:06:09.507169 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:09.507385  2134 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:09.508236  2348 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:09.509564  2134 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:09.510871  2134 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:09.484205  2326 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:09.456105  2342 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:09.488425  2313 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:06:09.528481 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 76809.8us
I20260709 13:06:09.476125  2295 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:09.493752 29186 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:09.504065 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.511631 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:09.512784 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:09.481532  2310 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:09.517545 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 50839.6us
I20260709 13:06:09.520083 29215 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:09.497248  2084 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:09.466917  2325 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:09.485813 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 46248.8us
I20260709 13:06:09.454810  1727 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.549496 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:09.554234 29217 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:09.558357 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:09.560232 29066 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.529350  2228 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.570158 29068 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:09.573812  2117 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.574281 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:09.574373  2117 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.575110  1727 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:09.575198 29070 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.585095 29112 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:09.586521 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:09.587355 29067 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:09.589314 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:09.548525  2313 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:09.558315  2325 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:09.577082  2117 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:09.540594  2315 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:06:09.571327  2319 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:09.604346 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:09.607466 29069 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:09.577138  2061 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:09.552050  2236 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
W20260709 13:06:09.620427 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 75293.8us
W20260709 13:06:09.621083 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 67386.9us
I20260709 13:06:09.624181  2336 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:09.624851  2336 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:09.625370 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.625980  2316 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:09.627275  2334 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:09.627858  2334 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.629590  1805 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.630594  2334 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:09.631237  2156 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:09.631749  2156 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:09.630626  1805 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:09.633486  2156 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:09.625283  2336 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:09.637359 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:09.637490  2353 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:09.639084  2353 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.638664 29215 raft_consensus.cc:2955] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 8 to 10, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:09.641505  2353 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:09.644579 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 133004us
W20260709 13:06:09.644809 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 156136us
I20260709 13:06:09.646164 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:09.646039 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:09.648237  2084 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.658289 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:09.665104 29908 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:09.664819  2342 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:09.662981 29871 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.604959  2147 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:09.598829  2346 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:09.673542 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:09.617327  2314 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:09.681398 29070 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:09.685141 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:09.686064 29070 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
W20260709 13:06:09.691020 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 86732.9us
I20260709 13:06:09.620142  2061 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:09.719413 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.728654 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:09.688611  2276 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.001s
I20260709 13:06:09.744609  2276 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:09.745479  2352 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:06:09.747129  2276 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:09.723237 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:09.735311  2359 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:09.753413 29069 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:09.682914  2014 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:09.619814  2083 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.684018  2334 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:09.744065  2266 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.506s	user 0.007s	sys 0.006s
W20260709 13:06:09.726368 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 80409.8us
I20260709 13:06:09.780781 29067 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:09.781318 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.756978 29066 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
W20260709 13:06:09.765026 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 141838us
I20260709 13:06:09.686269  2353 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:09.678974  2224 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:09.672601  2316 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:09.716945  2213 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:09.782933  2042 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:09.788378  2269 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.712734  2145 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:09.790956  2362 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:06:09.757742  2314 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:09.791965  2358 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:06:09.793409  2336 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:09.787732  2211 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.538s	user 0.006s	sys 0.006s
I20260709 13:06:09.757426  2295 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:09.825639 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 108058us
W20260709 13:06:09.832233 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 97477.9us
I20260709 13:06:09.846356 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.832719 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:09.711253 29903 raft_consensus.cc:2955] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 8 to 10, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:09.783288  2276 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.808380  2347 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:09.828293  2213 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:06:09.843740 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 62529.3us
W20260709 13:06:09.825325  2246 consensus_meta.cc:287] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.003s
I20260709 13:06:09.812011  2224 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:09.858667 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.860582 29147 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:09.861665 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 76243us
I20260709 13:06:09.732625  2337 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 13:06:09.875766 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.856977  2088 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:09.880316 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:09.882908 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.885020 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:09.866129  2224 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:09.861320  2084 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:09.879292  2253 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.841s	user 0.013s	sys 0.000s
I20260709 13:06:09.842074  2269 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:09.867079  2326 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:09.897240  2147 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:09.901839  2057 consensus_meta.cc:287] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.934s	user 0.012s	sys 0.000s
W20260709 13:06:09.902053 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.904268  2343 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:09.906328  2084 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.912395 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:09.944350 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:09.916963 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 241600us
I20260709 13:06:09.920853 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:09.921617 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:09.891520  2211 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:09.929419 29069 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:09.949805 29147 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:09.950709 29073 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:06:09.951692 29908 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:09.932973 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:09.881948  2295 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.882440  2088 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.861178  2276 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.946734 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:09.928294 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 254821us
I20260709 13:06:09.964112 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:09.892772  2236 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:09.892211  2370 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:09.840334  2313 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:09.961403 29146 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:09.924085  2253 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:09.966549 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:09.977429 29147 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:09.917650  2117 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:06:09.967353 29185 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:09.941442  2343 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:09.953307  2348 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:10.008271 29908 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:06:10.008646 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:10.009737 29213 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:09.947623  2364 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:09.984216  2349 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.548s	user 0.006s	sys 0.000s
I20260709 13:06:09.944139  2057 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.027536  2057 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:09.990022 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 113753us
I20260709 13:06:09.950646  2315 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:10.086207 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 62834us
I20260709 13:06:10.076431 29186 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:09.977906  2326 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:10.087626 29186 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:09.992223  2369 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:06:09.949316  2334 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:06:09.966295  2236 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.093181  2117 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.100608  2236 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:09.952147  2325 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:10.111660 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.994313  2372 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:06:09.994668  2354 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:09.995019  2314 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.971356  2370 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.125662 29148 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:09.966931  2211 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:10.000675  2156 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:10.001055  2269 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:10.001631 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:09.962502  2088 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:10.011668 29215 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 13:06:10.015810 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52124.8us
I20260709 13:06:10.015610 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:10.018190 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50920us
I20260709 13:06:10.022413  2147 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:10.159372 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.160212 29142 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:10.028127 29216 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:09.975625  2134 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.013s
W20260709 13:06:10.005533  2312 consensus_meta.cc:287] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.973s	user 0.009s	sys 0.005s
W20260709 13:06:10.028127 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 95267.6us
I20260709 13:06:10.084784 29213 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:09.990751  2367 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:10.090276 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 881) took 1204 ms. Trace:
I20260709 13:06:10.091509  2343 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.093569 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.095479  2310 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:10.380218  2343 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:10.380394  2310 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:10.380990  2310 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:10.096421  2042 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:10.096824  1727 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:06:10.074620  1948 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.690s	user 0.013s	sys 0.000s
I20260709 13:06:10.099172 29908 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:09.953943  2084 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:09.971861  2332 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:10.105283 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:10.405171 29909 raft_consensus.cc:2376] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19. Re-sending same reply.
I20260709 13:06:10.106510 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:10.107386  2059 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:10.107468 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:09.972069  2316 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.112556 29905 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:10.114006 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:10.376585  2366 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.413761 29910 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:10.121734 29071 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:10.128043 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.128228 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:10.134825 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:10.100811  2117 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.111426  1805 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.111112  2364 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:10.138870 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:10.140926 29902 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:10.141112 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:10.138211 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:10.141677 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.437251 29073 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:10.150988 29065 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:10.157755  2325 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:10.439644 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 301493us
I20260709 13:06:10.440550 29185 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:10.158494  2336 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:10.163057 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:10.176023 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.301350  2312 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.307045  2246 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.323442  2376 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:06:10.506845 29067 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:10.319207  2213 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.172098  2083 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:10.321594  2371 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
W20260709 13:06:10.526443 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 83328.2us
W20260709 13:06:10.330397  2377 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:06:10.317848 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.337002  2276 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:10.400866  1948 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.244750 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.278801 29066 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:06:10.265390 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:10.532827 29142 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:06:10.280606 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2727) took 1122 ms. Trace:
I20260709 13:06:10.255980 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:10.129140  2370 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:10.276268 29069 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:06:10.534946 29073 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
W20260709 13:06:10.329524  2061 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.000s
I20260709 13:06:10.247467 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.158974  2147 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:10.541404  2147 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:10.513420  2312 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.273495  2211 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.545936  2312 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:10.535266  2346 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:10.338590  2353 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.006s
I20260709 13:06:10.548419  2353 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:10.256754 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.552985 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.553664 29139 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:10.554754 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.556123 29219 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:10.319417  2304 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 1.386s	user 0.006s	sys 0.008s
I20260709 13:06:10.557416  2319 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.133765  2342 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.152s
I20260709 13:06:10.558207  2319 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:10.376736 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.559315 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.559695 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.376267 29904 rpcz_store.cc:276] 0709 13:06:08.872404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.880785 (+  8381us) service_pool.cc:224] Handling call
0709 13:06:10.075552 (+1194767us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54007102d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:10.076354 (+   802us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929067904}
I20260709 13:06:10.396620 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:10.339710  2295 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:10.560792 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 315114us
I20260709 13:06:10.379050  2134 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:10.561764  2134 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:10.562259  2134 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:10.399776 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:10.564740 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 165064us
I20260709 13:06:10.406278 29214 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:10.412787 29144 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:10.415225 29072 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:10.415350 29906 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:10.572181  2253 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:10.376415  2279 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.572763  2253 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:10.573185  2279 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.574193  2253 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.437991 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.409787 29217 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:06:10.540185  2084 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:10.578629  2224 consensus_meta.cc:287] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.000s
W20260709 13:06:10.439644 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64570.5us
I20260709 13:06:10.400548  2269 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.452001 29065 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:06:10.575740  2279 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:10.408830  2059 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:10.436395  2337 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.412808  2125 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.416043  2265 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.518281 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.519081  2196 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.520152 29213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:10.614248 29213 raft_consensus.cc:606] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:10.614761 29213 raft_consensus.cc:614] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:06:10.620352 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 88452.5us
I20260709 13:06:10.526027 29143 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11.
W20260709 13:06:10.527319  2380 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:06:10.504369  2042 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:10.505597  2364 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:10.279663  2347 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:10.530576 29148 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:10.533257 29216 rpcz_store.cc:276] 0709 13:06:09.157994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.160000 (+  2006us) service_pool.cc:224] Handling call
0709 13:06:09.174169 (+ 14169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:10.161278 (+987109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.280401 (+119123us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b540009fbd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:10.280554 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1963414784}
I20260709 13:06:10.536794 29073 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 13:06:10.538683  2057 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:10.530313  2355 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:10.539192  2061 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.540346  2117 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:10.540596 29069 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: [16.7-16.7]
I20260709 13:06:10.558586  2342 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:10.651154 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:10.440874 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:10.664927 29070 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:10.578687 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:10.579183 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 438111us
I20260709 13:06:10.666666 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:10.567848 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.580976  2156 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:10.449167  2303 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:06:10.582612 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.582768 29147 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:10.599677  2276 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:06:10.538146 29910 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
W20260709 13:06:10.584728 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.585517 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.587395 29141 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:06:10.593478  2224 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.589893 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:10.695819 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.446714  1805 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.699997 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:10.700986  2387 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.183s	user 0.000s	sys 0.003s
I20260709 13:06:10.701604  1805 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:10.512145  2326 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:10.591619 29145 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:10.703003 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.704993 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.443751  2332 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.593369 29214 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:10.645272  2057 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:10.599408 29221 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:06:10.510768  2246 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.538151  2315 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:10.563236  2134 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.717976 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.592860  2084 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.639555 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.571280  2083 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.649012  2383 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:10.649144 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:10.722823 29067 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:10.649888 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.647094  2393 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:10.653452 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:10.723955 29143 raft_consensus.cc:2376] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 11. Re-sending same reply.
I20260709 13:06:10.133383  2145 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:10.596791  2334 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:06:10.438944  2367 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.113s
I20260709 13:06:10.669719 29906 raft_consensus.cc:2368] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15.
I20260709 13:06:10.590293  2236 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.629006  2213 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:10.672516 29072 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:10.732807 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.596863  2269 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.734243 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.675840 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.557503  2304 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:10.415885  2345 consensus_queue.cc:1175] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e78ffbcc98242068a874a5d9e341477" responder_term: 13 status { last_received { term: 10 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 12. Current term is 13. Ops: [12.11-12.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:10.680121 29218 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:10.680478  2303 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:10.746896  2236 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:10.747004  2303 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:10.721697  2382 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:10.748514  2303 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.748984  2310 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.750540 29222 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:10.750578  2303 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:10.751314  2145 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:10.751717  2145 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:10.752322  2332 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:10.752655  2145 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.753213 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:10.753818 29067 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:10.753914  2145 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:10.757047 29146 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:10.758752 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:10.760006 29066 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:10.761940 29066 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:06:10.731244  2084 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.772212  2084 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:10.772840  2134 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:10.774607  2156 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:10.778246 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 911) took 1620 ms. Trace:
I20260709 13:06:10.778776 29221 rpcz_store.cc:276] 0709 13:06:09.158269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.160079 (+  1810us) service_pool.cc:224] Handling call
0709 13:06:09.199205 (+ 39126us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.710974 (+1511769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.777870 (+ 66896us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400339ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:10.778161 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3214605056}
I20260709 13:06:10.742424  2147 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.781872 29220 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:10.783381  2147 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:10.785138 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.786790 29148 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43a52106dbee45398d36a633f281eef6"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:06:10.787613 29148 raft_consensus.cc:606] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Received request to transfer leadership to TS 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:10.790103 29148 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:06:10.791188 29148 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:10.795212  2382 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:10.796491 29143 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:10.757172  2310 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:10.802948  2224 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:10.805892  2246 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:10.758049  2304 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.809963  2304 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:10.811172 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:10.812088 29065 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:10.737296  2383 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.814419 29217 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:10.721868  2315 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.686527 29139 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:10.689314  2319 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:10.612979  2337 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.690673 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 885) took 1036 ms. Trace:
W20260709 13:06:10.694031  2088 consensus_meta.cc:287] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.000s
I20260709 13:06:10.696907 29142 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:10.697614 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:10.699632  2364 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.454344 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:10.622634  2014 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:06:10.708168 29214 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:06:10.466846  2366 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.612439  2125 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.706643 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:10.815856  2383 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:10.818481  2315 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:10.824646  2228 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:10.828653  2394 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:10.845369  2228 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.708366 29064 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:10.847483  2228 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:10.848194 29145 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:10.786806 29037 leader_election.cc:400] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15.
W20260709 13:06:10.848958 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 62388.8us
I20260709 13:06:10.650705  2042 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:10.850195  2042 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.851027 29067 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:06:10.621930  2265 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.852995 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:10.853737  2265 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:10.854156  2059 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.674243  2355 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:10.855513  2059 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:10.855657  2355 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:10.712970 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46407.7us
I20260709 13:06:10.832068  2337 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:10.716547 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 126735us
I20260709 13:06:10.859949 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:10.860116  2337 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:10.860641 29218 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:10.861207 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:10.832504  2088 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.862818 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:10.708475  2313 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:10.717325  2326 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.864279 29215 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:10.836814 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 389199us
I20260709 13:06:10.664650  2342 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:10.627033  2196 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:10.863850 29222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79cdfc6a725145348b70034422ef477b"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:10.866415 29222 raft_consensus.cc:606] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:10.866794 29222 raft_consensus.cc:614] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:10.716754  2316 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:10.837673  2319 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:06:10.709794  2348 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.630s	user 0.004s	sys 0.012s
I20260709 13:06:10.717633  2370 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.718776 29144 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:10.715683  2061 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:10.736914 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:10.738288 29907 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:10.741006 29140 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:10.745620  2269 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:10.845667  2394 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:10.854636 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:10.833607 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:10.956043 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 62688.6us
I20260709 13:06:10.968871 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:10.969782 29213 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:10.838459 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:10.855786  2359 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:10.839811  2147 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:10.838881 29141 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:06:10.719663 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 97570.6us
I20260709 13:06:10.975660 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:10.977119 29221 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:10.763661 29066 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:10.709131 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:10.861766 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:10.946154 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 149199us
I20260709 13:06:10.832242 29911 rpcz_store.cc:276] 0709 13:06:09.654700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.655637 (+   937us) service_pool.cc:224] Handling call
0709 13:06:09.666411 (+ 10774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:09.666851 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:09.666882 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:09.667398 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:06:09.924894 (+257496us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:09.924934 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:09.925000 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.320785 (+395785us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:10.674804 (+354019us) raft_consensus.cc:1685] finished
0709 13:06:10.674824 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.690605 (+ 15781us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6480,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":37060,"spinlock_wait_cycles":12118912,"thread_start_us":257109,"threads_started":1,"wal-append.queue_time_us":286757}
I20260709 13:06:10.838675 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:10.956585 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62097.5us
W20260709 13:06:10.978961 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 140369us
I20260709 13:06:10.957811  2383 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:06:10.835301  2398 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:06:10.843214 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:10.972993 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2648) took 1004 ms. Trace:
I20260709 13:06:10.968178 29065 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:10.830171  2246 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.733487 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:10.883074  2319 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:10.950362  2348 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:10.841987  2390 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.166s	user 0.004s	sys 0.000s
I20260709 13:06:10.982157 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.983955 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:10.986616 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:11.011677 29142 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:10.987902 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:10.989207 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:11.012275 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:10.978412 29871 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:10.839613  2366 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:10.843712  2125 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:10.864311  2088 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:10.839335  1948 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:10.991220 29145 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:10.992352 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:10.993423 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 284345us
I20260709 13:06:11.017341 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:11.018225 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.998379 29219 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:10.999444 29072 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:10.977020 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:10.949201  2156 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:11.002341 29188 leader_election.cc:400] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13.
I20260709 13:06:11.020539 29188 leader_election.cc:403] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:06:11.019550 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:11.021961 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:11.024376 29073 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:06:11.024711 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47785.3us
I20260709 13:06:11.025969 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:10.860706  2337 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:10.975198  2359 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:11.029066 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:11.031569 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:11.055802 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.044872 29073 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:06:11.092504 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 63513us
I20260709 13:06:11.093603 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:11.097168 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:11.097321 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:11.099324 29070 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:06:11.033967 29069 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:06:10.996378  2383 consensus_queue.cc:1243] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:10.998521  2269 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:11.107143 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:11.108361 29219 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:11.034590 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:11.030794  2084 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:10.998739  2224 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
W20260709 13:06:11.115223 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.033108  2295 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:11.032953  2125 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:11.022456  2304 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:11.032670  2014 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:11.096081  2397 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:06:11.117051 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:11.027432  2156 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:11.016767  2147 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:11.030593  2236 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:10.833289  2382 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.104416  2312 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:11.146109 29070 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:11.133711 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:11.177423 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:11.112824  2253 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:11.109804  2269 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.167428 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:11.169581 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:11.246658 29064 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:11.246587 29071 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:11.025995  2061 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:06:11.113061  2394 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.089118  2279 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:06:11.227758 29219 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:06:11.181301 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 877) took 1038 ms. Trace:
I20260709 13:06:11.332239 29141 rpcz_store.cc:276] 0709 13:06:10.142355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.756190 (+613835us) service_pool.cc:224] Handling call
0709 13:06:10.838543 (+ 82353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:10.975582 (+137039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.180979 (+205397us) spinlock_profiling.cc:243] Waited 81.9 ms on lock 0x7b54003467d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:11.181207 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188406656}
I20260709 13:06:11.001817  2334 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 13:06:11.120692  2295 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.229453 29067 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:11.116941  2088 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13.
I20260709 13:06:11.091367  2342 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.003526  2246 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.249800 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:11.249948 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:11.124545  2304 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.253731 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:11.164374  2156 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:11.315083 29222 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:11.232245  2312 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.203099  2236 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.118409  2084 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.347786 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 869) took 1313 ms. Trace:
I20260709 13:06:11.230386  2253 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.173311  2125 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:11.353407 29071 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:06:11.355460  2398 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 13:06:11.359171 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 379578us
I20260709 13:06:11.358034 29065 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:11.375721 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:11.170253  2276 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.183s
I20260709 13:06:11.245846  2269 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:11.343472  2295 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:11.372952 29146 rpcz_store.cc:276] 0709 13:06:10.028713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.030932 (+  2219us) service_pool.cc:224] Handling call
0709 13:06:11.341541 (+1310609us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54001bd0d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:11.341689 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":873786112}
I20260709 13:06:11.339326  2336 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:06:11.368172  2312 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:11.362879  2304 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:11.378378  2084 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:11.401111 29185 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:11.402765 29064 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 12.
I20260709 13:06:11.347563  2354 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:06:11.396642  2342 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:11.378060  2236 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.396775  2401 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:11.434986 29067 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
W20260709 13:06:11.411268  2400 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:06:11.440618 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 409085us
I20260709 13:06:11.441987 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2738) took 1046 ms. Trace:
I20260709 13:06:11.358314  2366 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:06:11.404307  2253 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.456532 29215 rpcz_store.cc:276] 0709 13:06:10.396128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.396300 (+   172us) service_pool.cc:224] Handling call
0709 13:06:11.441691 (+1045391us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b540023c654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:11.441899 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696739200}
I20260709 13:06:11.239506  2359 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.442556  2383 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:11.173209  2147 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.465287  2364 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:11.444166  2401 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:11.426945  2370 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.401345  2196 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.406759  1948 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
W20260709 13:06:11.503561 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45790.4us
I20260709 13:06:11.517777 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:11.527825 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:11.480212  2383 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:11.529495 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:11.555388 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.530325 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 874) took 1389 ms. Trace:
W20260709 13:06:11.516665 29907 consensus_meta.cc:287] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.003s
I20260709 13:06:11.505338  2337 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.520874  2359 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.556506 29907 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:11.523345  1805 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.556222 29140 rpcz_store.cc:276] 0709 13:06:10.140820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.160524 (+ 19704us) service_pool.cc:224] Handling call
0709 13:06:11.530132 (+1369608us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400144e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:11.530266 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433836800}
W20260709 13:06:11.571439 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 466788us
I20260709 13:06:11.548394  2279 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.547593  2196 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:11.569069  2383 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:11.590813 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 171324us
I20260709 13:06:11.590150 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:11.577920  2337 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.594376 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:11.599979 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:11.548847  2370 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:06:11.614662 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:11.615796 29911 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:06:11.620671 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 92979.1us
I20260709 13:06:11.621691 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:11.617925  2213 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.788s	user 0.013s	sys 0.000s
I20260709 13:06:11.611318  2246 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.611666  2125 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:11.586671  1805 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.628366 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:11.595194  2279 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.641443  2196 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:11.654583 29065 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:11.654922  2196 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:11.656422  2347 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:11.645310 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:11.645387  2236 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.658382 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:11.659580  2236 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.660466  2125 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:11.630400 29073 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:11.661681  2125 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:11.644781  2253 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.664063  2315 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.665128  2253 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:10.996439 29068 rpcz_store.cc:276] 0709 13:06:09.968864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.510712 (+541848us) service_pool.cc:224] Handling call
0709 13:06:10.511067 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:10.511294 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:10.511320 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:10.528627 (+ 17307us) log.cc:844] Serialized 26 byte log entry
0709 13:06:10.866095 (+337468us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:10.972443 (+106348us) log.cc:844] Serialized 14 byte log entry
0709 13:06:10.972695 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.972766 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.972803 (+    37us) raft_consensus.cc:1685] finished
0709 13:06:10.972815 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.972939 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":88907,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":2203,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":15687,"thread_start_us":289853,"threads_started":1,"wal-append.queue_time_us":327282}
W20260709 13:06:11.669108  2385 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.079s	user 0.000s	sys 0.012s
W20260709 13:06:11.669715 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.649842 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:11.671568 29067 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.625761  2401 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.673542 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 2740) took 1088 ms. Trace:
I20260709 13:06:11.673914 29217 rpcz_store.cc:276] 0709 13:06:10.584871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.674018 (+ 89147us) service_pool.cc:224] Handling call
0709 13:06:11.673361 (+999343us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b540023bc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:11.673478 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261561088}
I20260709 13:06:11.664698  2315 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.652858 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:11.678164 29071 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 13: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:11.646255 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:11.629030  2417 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:11.656453  2196 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.681768  2417 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.682147  2417 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:11.682782  2303 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.931s	user 0.017s	sys 0.000s
I20260709 13:06:11.683229  2295 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.604657  2224 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:06:11.641960  1805 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:11.685755  2295 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.686542 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:11.688113  2134 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:11.687249  2347 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:11.690296  2134 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:11.690476 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:11.691594  2412 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:11.692451  2412 consensus_queue.cc:1243] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:06:11.694294  2412 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:11.694049  2269 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.694911  2412 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:11.695508  2412 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:11.696826 29875 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:11.697145  2228 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:11.697564  2228 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:11.698498  2228 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.699416  2228 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:11.700634  2083 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:11.701727  2319 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.702401  2156 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.703066  2319 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:11.704975  2057 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.692466 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:11.708149  2409 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:11.708359  2057 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:11.628728  2410 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:11.708644  2409 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:11.709591  2409 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:11.620136  2337 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.709195  2410 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:11.712046 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:11.722158 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.717964 29067 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:11.625756 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:11.696975  2269 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.699232  2084 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:11.743400 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 117712us
I20260709 13:06:11.741007 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:11.745344 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:11.751617 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:11.753860  2253 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:11.754347  2253 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:11.754768  2253 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:11.760032  2415 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:11.760689  2415 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.762907  2415 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.720047  2083 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.764048  2083 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.764509  2253 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.766187  2084 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.720295  2224 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.767012  2253 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.733569  1805 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.768924  1805 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.675184  2418 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 13:06:11.771062  1805 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.711473  2370 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:11.744047 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:11.774415 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:11.774652 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:11.773110 29140 raft_consensus.cc:2410] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:11.785149 29143 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:11.736820  2410 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.785257 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:11.767140  2224 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.786770 29147 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:11.786908 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:11.787848 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 91098.3us
W20260709 13:06:11.809455 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 117184us
I20260709 13:06:11.810712 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:11.809715 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:11.771212  2337 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:11.816751  2383 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.742158  2347 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:11.764489  2236 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:11.824198 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:11.826995 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:11.766228  2083 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:11.830822 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 53948.9us
I20260709 13:06:11.825655 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:11.773599  2125 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.774115  2370 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.832710 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:11.844398 29148 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.834195 29904 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:06:11.834831 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:11.834846 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.836202 29145 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:11.845846 29910 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:11.838876 29909 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:11.831606 29908 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:11.820737  2407 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:11.789405  2246 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:11.737587  2304 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.847385 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:11.840888 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 66312.3us
I20260709 13:06:11.860314 29147 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:11.833321  2084 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:11.852630 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:11.864830 29909 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:11.853432 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.773661  2396 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:06:11.858503 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:11.774080  2014 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:06:11.896626  2014 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:11.897025  2014 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:11.897809  2014 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.898960  2014 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:11.828377  2196 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:06:11.850064 29144 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:11.854811  2246 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.869153 29904 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:11.905119 29903 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:11.856853  2407 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.849745  2125 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.907502  2246 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:11.907851  2407 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:11.908735  2417 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.873466 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:11.853924  2059 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.973s	user 0.013s	sys 0.000s
I20260709 13:06:11.863566  2083 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.868196  2418 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.884747  2156 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:11.864696  2084 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:11.886332  2396 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:11.887987  2304 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:11.888813 29148 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:11.891933  2224 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.828523  2312 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:11.903780 29142 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:11.842334  2279 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.849057  2370 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.913192 29910 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:11.913452 29908 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:11.915871 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:11.916164  2083 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.924603 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2734) took 1911 ms. Trace:
I20260709 13:06:11.940778 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:11.928594 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:11.942356 29148 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:11.942559 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:11.931217 29143 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:11.931571 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:11.927773 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:11.904544  2383 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:11.947798 29144 raft_consensus.cc:2368] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14.
I20260709 13:06:11.947906 29139 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:11.948632 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:11.943303 29908 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:11.927428  2396 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:11.921008  2418 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.941471 29910 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.941027 29220 rpcz_store.cc:276] 0709 13:06:10.013246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.013571 (+   325us) service_pool.cc:224] Handling call
0709 13:06:10.025174 (+ 11603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:10.782308 (+757134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:11.924346 (+1142038us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b5400283cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:11.924540 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739746048}
I20260709 13:06:11.965138 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:11.926839  2084 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.936694  2319 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.955711  2394 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:11.967306  2319 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.946949  2196 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.969708  2196 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.973439  2083 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:11.951165 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:11.936666 29147 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:11.977463 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:11.951361 29146 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:11.952248 29141 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:11.953964 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:11.980243 29904 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:11.980453 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:11.980854 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.953924 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:11.981062 29146 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:11.954912  2312 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.956055  2125 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:11.951977  2279 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:11.957909  2396 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:11.960635  2418 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:11.964532 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:11.990690 29910 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:11.980269 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:11.948817  2304 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.993005 29903 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:11.998116 29146 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:12.000065 29112 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:11.996186  2084 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:11.977308  2196 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.013322 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.013887  2370 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.014698 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:11.991801  2332 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.017700  2059 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.018100  2332 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.020282  1727 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:12.021163  1727 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:12.021649  1727 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:12.018226  2059 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.024142  2279 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:12.025952 29112 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:12.025357  2409 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.026834 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:12.027247 29112 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:12.028167 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.028347  2319 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.008844 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:12.029605 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.029795 29144 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:12.029894 29145 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:12.015821 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.033252 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:12.034019 29139 raft_consensus.cc:2376] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14. Re-sending same reply.
I20260709 13:06:12.034695 29909 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:12.037096 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:12.038511 29147 raft_consensus.cc:2368] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:12.040668  2418 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:12.026033  2059 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:11.993458  2083 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:12.042922  2396 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:12.077431 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:12.077832 29145 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:11.999447  2304 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.045764 29904 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:12.047578 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:12.051174  2420 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.382s	user 0.002s	sys 0.000s
I20260709 13:06:12.053221 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:12.060101 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:12.090687 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.020325  2332 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.039501  2315 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:12.061365  2224 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:12.072561  2316 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:06:12.073025 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.041669  2276 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:12.042263  2312 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:06:12.044327 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:12.082994 29871 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:12.083876  2412 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.086442 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.084270 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:12.091817 29144 raft_consensus.cc:2368] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:06:12.044481 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:12.094048 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:12.106779 29904 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:12.107141 29145 raft_consensus.cc:2410] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:12.102619 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.081385  2418 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:12.084285  2334 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:12.108395 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.110638 29037 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:12.108675 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:12.110898 29902 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:12.089334  2265 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:12.110958 29139 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:12.114176 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:12.114780 29146 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:12.115952 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:12.116688 29139 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:12.093891  2117 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:12.120018 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 102690us
I20260709 13:06:12.120234 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:12.118993 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:12.121274 29068 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:12.082553  2246 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.105300  2276 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.122222 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:12.117777 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:12.119350 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:12.103210  2315 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:12.123234 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:12.105554  2316 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:12.117229  2430 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:06:12.106088  2332 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:12.125954 29070 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:12.126112 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.113691  2156 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:12.114111  2347 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:12.115175  2265 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.117616  2348 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:12.110608  2334 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:12.114248  2319 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:12.131317 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.132094 29139 raft_consensus.cc:2368] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:12.144192 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:12.132128 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:12.118785  2083 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:12.113901  2224 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:12.135645 29146 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:12.117805  2059 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.158181 29909 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:12.145259 29903 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:12.144891 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:12.147189 29068 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:12.125655  2276 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:12.161298 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.162453 29142 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:12.151017 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.151947 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:12.163506 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.164575 29070 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
W20260709 13:06:12.153519 29907 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 890) took 1990 ms (client timeout 2491 ms). Trace:
I20260709 13:06:12.147742  2347 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.136471  2315 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.160696 29067 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:12.124310  2117 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.137137  2409 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.309962 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.409719 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:12.337412 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:12.165411 29907 rpcz_store.cc:269] 0709 13:06:10.162670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.162818 (+   148us) service_pool.cc:224] Handling call
0709 13:06:12.153354 (+1990536us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54004e61d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:12.153467 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675965312}
I20260709 13:06:12.310029 29214 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:12.165956 29142 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15. Ops: [13.9-14.10]
I20260709 13:06:12.313436 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.190819 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:12.277210 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 111270us
I20260709 13:06:12.140332  2265 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:12.308424 29217 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:12.414669 29143 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:12.361219 29219 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:12.308507 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:12.361523 29221 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:12.291797 29905 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 13:06:12.387063 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.195007 29146 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:12.289507 29070 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:06:12.298718 29908 raft_consensus.cc:2410] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:06:12.203621 29139 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:12.348694 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:12.422488 29146 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:06:12.311421 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.320405 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:12.314013 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.424504 29220 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:12.424882 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:12.425827 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:12.378988 29903 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:12.378010 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.201048 29072 raft_consensus.cc:1217] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 7.4->[9.5-18.9]   Dedup: 9.5->[12.6-18.9]
I20260709 13:06:12.428261 29215 raft_consensus.cc:2410] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:12.429579 29142 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:12.417366 29909 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:12.338816 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:12.430469 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.415634 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:12.431283 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 92574.9us
I20260709 13:06:12.432368 29142 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:12.414726 29222 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:12.433563 29141 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:12.419872 29147 raft_consensus.cc:2410] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:12.140571  2348 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.448351 29139 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:12.427959 29072 pending_rounds.cc:85] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 5
I20260709 13:06:12.450430 29072 pending_rounds.cc:107] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:06:12.450729 29072 raft_consensus.cc:2889] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:12.350389 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:12.433563 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.451566 29222 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:12.453102 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:12.454118 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.454823 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:12.455439 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:12.456108 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.425791 29218 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:06:12.162851  2403 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.046s	user 0.006s	sys 0.009s
I20260709 13:06:12.464706 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:12.444262 29907 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:12.466471 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.444739 29142 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:12.445703 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:12.467373 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.468187 29219 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:12.145569  2156 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:12.457850 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:12.458319 29215 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:12.460453 29140 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:12.307246  2354 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.462690  2117 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:12.148341  2083 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:12.192574  2316 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.465261 29141 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:12.454699  2382 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:12.474403 29188 leader_election.cc:400] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14.
I20260709 13:06:12.454674 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:12.476390 29147 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:12.530611 29188 leader_election.cc:403] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:12.530834 29068 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:12.476897 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.532433 29214 raft_consensus.cc:2410] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:06:12.460650  2279 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:12.372025  2304 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.162298  2061 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.534201  2061 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.537158  2061 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:12.481475 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:12.481653 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.305500  2417 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.144866  2332 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.485662 29909 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:12.491112  2156 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.541885 29068 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:12.542268  2332 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:12.163678  2312 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.316s
I20260709 13:06:12.546078  2312 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:12.546617  2312 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:12.493638  2342 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:12.464679  2147 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:06:12.548291  2342 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.548790  2342 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:12.549853  2147 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:12.550459  2147 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.465288  2057 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:06:12.495235  2265 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.192416  2224 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.143855  2418 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.185001  2334 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.455331  2348 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:12.500769 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:12.501617 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.504114 29148 raft_consensus.cc:2410] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:06:12.230854  2059 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.504976  2354 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.506415  2336 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:12.148864  2410 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.465317 29065 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:06:12.506335  2315 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:12.557651 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.472565  2355 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:12.416290  2407 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.510149  2422 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:12.455878  2383 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:06:12.577695 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:12.600286 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.568619 29218 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:12.611073 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.612267 29215 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:12.627980 29221 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:12.584530 29904 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:06:12.632283 29216 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:12.632949 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 284376us
W20260709 13:06:12.634465 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 188806us
I20260709 13:06:12.636549 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.637866 29215 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:06:12.639504 29907 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:06:12.643039 29069 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:06:12.582201  2396 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.004s
I20260709 13:06:12.579728  2316 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:12.576577  2057 consensus_queue.cc:1243] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 17.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:12.583616  2334 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:12.628413  2355 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14.
I20260709 13:06:12.617590  2370 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.585703  2336 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:12.656220 29069 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:06:12.634603  1727 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.609s	user 0.013s	sys 0.001s
I20260709 13:06:12.646340 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:12.613901  2147 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:12.635993  2422 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:12.609077  2083 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:12.669307 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.673997 29910 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:12.676419 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:12.676651 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:12.677188 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 202979us
I20260709 13:06:12.678758 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:12.677228 29148 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:12.681665 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:12.682396 29069 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:06:12.686128 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.577376  2312 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.679060  2265 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:12.687362 29217 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:12.655016  2394 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:06:12.688495 29188 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:12.696441 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:12.692307 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:12.701404 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:12.703258 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:12.705730 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:12.707677 29219 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:12.708568 29069 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:06:12.708767 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:12.714114 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:12.690940  2441 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.717706 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:12.720250 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 241080us
I20260709 13:06:12.722759 29217 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:06:12.724083  2434 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:06:12.724692  2442 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:12.690826  2347 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:12.728044  2276 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.566s	user 0.008s	sys 0.004s
I20260709 13:06:12.702792  2084 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:06:12.712440  2359 raft_consensus.cc:2764] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:06:12.728304  2422 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.702543  2417 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:12.729974 29069 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:12.674557  1727 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.722456  2304 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:12.753176 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:12.720077  2401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.045s	user 0.004s	sys 0.020s
W20260709 13:06:12.732429  2437 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:06:12.736521  2265 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:12.745321  2276 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.765339  2265 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.765938  2265 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:12.766650  2407 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.768760  2410 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:12.737001  2441 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.722200  2312 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:12.771543  2441 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:12.754691  2246 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.708148  2396 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:06:12.755811 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 47106.1us
I20260709 13:06:12.776280  2396 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.776738  2396 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:12.776654 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:12.777182 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:12.777472 29908 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:12.780319 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:12.780213  2417 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:12.763362  2342 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.764179  2354 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:12.756085  2084 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:12.769488  1727 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:12.782066  2059 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.782414  2445 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:12.781658  2417 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.783243  2445 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:12.756352  2446 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:12.783814  2445 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:12.784226  2446 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:12.784296  2445 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:12.780222  2370 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:12.733247 29186 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:12.734084  2147 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:06:12.728047  2336 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.781857 29144 raft_consensus.cc:2368] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:06:12.781134 29909 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:12.785648  2446 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:12.741402 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52994.8us
I20260709 13:06:12.790226 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:12.754787  2401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.080s	user 0.005s	sys 0.021s
I20260709 13:06:12.767704  2276 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:12.805667 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:12.781735  2084 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.804842 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:12.812745 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:12.814159 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:12.800508  2434 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:12.816572 29906 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:12.819576  2057 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:12.820068  2057 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:12.821183  2057 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.835647  2134 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:12.836584  2134 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.838649  2364 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:12.839260  2134 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.842458  2134 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:12.843336  2134 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:12.846761  2134 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:12.847683  2134 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.848126  2213 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.848371  2134 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.855818  2355 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:06:12.885886  2355 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:12.887287  2355 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:12.849085  2213 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.894387  2213 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.839263  2364 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:12.896505  2364 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.897841  2364 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:12.891192  2061 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:12.861385  2134 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.900555  2061 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:12.901782  2061 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.904821  2117 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.905110  2059 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.905457  2117 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.905907  2059 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:12.908994  2117 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:12.909373  2443 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
W20260709 13:06:12.912341 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 419053us
W20260709 13:06:12.913321  2447 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.169s	user 0.004s	sys 0.001s
I20260709 13:06:12.909358  2059 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:12.924978 29903 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:12.932215 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:12.932590 29185 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:12.933943 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:12.935388 29903 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:12.946890 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:13.069518 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 122746us
I20260709 13:06:13.070434 29871 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:12.961134 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.071377 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.802423  1727 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:12.964617 29073 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:13.072850 29073 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:12.965600 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.073848 29065 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:12.822855  2057 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:12.970229 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:12.963281 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.075874 29066 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:12.973035 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:12.811846  2084 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:13.071374 29908 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
W20260709 13:06:12.974058 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 285062us
I20260709 13:06:12.816983  2383 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:12.938419 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:13.079974 29903 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:13.082307 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.084995 29871 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:13.098388 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:13.099381 29071 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:13.100915 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:13.102916 29071 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:13.036010  2345 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:06:13.107494 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:13.075829  2304 raft_consensus.cc:2764] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:06:13.072476  2347 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:13.129525 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 895) took 1522 ms. Trace:
I20260709 13:06:13.131042 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 936) took 1305 ms. Trace:
I20260709 13:06:13.034843 29909 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:12.982362 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:13.036778 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.134251 29069 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:06:12.967259  2156 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:06:13.055866 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 322691us
W20260709 13:06:13.036617  2434 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:06:13.137991 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 155833us
I20260709 13:06:13.063690 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:12.891150  2355 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:13.071897 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.074110 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.953361  2370 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.130033 29906 rpcz_store.cc:276] 0709 13:06:11.599382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.599534 (+   152us) service_pool.cc:224] Handling call
0709 13:06:11.608656 (+  9122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.954596 (+1345940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.121061 (+166465us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54007102d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:13.121292 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2777349376}
I20260709 13:06:13.131656 29905 rpcz_store.cc:276] 0709 13:06:11.823076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:11.823185 (+   109us) service_pool.cc:224] Handling call
0709 13:06:12.153805 (+330620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:12.444359 (+290554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.089625 (+645266us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54004e61d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:13.126229 (+ 36604us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54004e61d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:13.128709 (+  2480us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1076059776}
I20260709 13:06:13.081604  1727 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:13.188351 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:13.141248 29066 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
W20260709 13:06:13.153043 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 89484.2us
I20260709 13:06:13.194621 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:13.196691 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.185655 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 69354.5us
I20260709 13:06:13.166903 29186 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:13.198015 29112 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:13.204253 29039 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:13.192960 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:13.208781 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:13.207511 29221 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:13.106170  2454 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:13.106278  2246 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:13.213599 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.214561 29215 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 11.
I20260709 13:06:13.216359 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:13.219720 29141 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:06:13.236717 29139 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:06:13.238981 29146 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:13.118479  2394 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:13.229755 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:13.248111 29219 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:13.249918 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:13.250645 29143 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:13.136806  2156 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:13.128736  2407 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:13.243212 29037 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:13.117530  2383 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:13.164253  2313 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:13.152694  2410 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.207653  2441 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.164484  2147 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.166144  2334 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.186110  2440 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:06:13.253485  2345 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:13.152777  2213 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
W20260709 13:06:13.278230 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.118103  2448 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.168s
I20260709 13:06:13.247292  2409 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.235121  2451 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:06:13.313928 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 200180us
I20260709 13:06:13.215699  2370 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.253247  2460 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:13.290527 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 86389.3us
I20260709 13:06:13.263415 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:13.261222  2407 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.262856  2315 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:13.265477  2246 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:13.296512 29069 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:13.263062  2342 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:13.143740  2347 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.207499  2452 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.268477  2394 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:13.260958  2117 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:06:13.343336 29142 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:13.308599  2458 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:13.324268 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 391842us
I20260709 13:06:13.326601 29871 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:13.349952 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 86716.5us
I20260709 13:06:13.289624  2147 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.377684 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.377902 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.388319 29071 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:13.284789  2410 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.292009  2334 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.389429 29143 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:13.308244  2332 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:06:13.395411 29065 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:13.287330  2441 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.384545  2315 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:13.396486 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.397069  2315 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.397599 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:13.398413 29065 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:13.399359 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:13.400372 29065 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:13.400564  2156 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.401470  2470 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:13.401815 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.402156  2156 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:13.402434 29065 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:13.326872  2407 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:13.377933  2450 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.319s	user 0.002s	sys 0.000s
I20260709 13:06:13.293319  2457 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:06:13.411171 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2540) took 1287 ms. Trace:
W20260709 13:06:13.412326 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 245561us
I20260709 13:06:13.412644 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2543) took 1267 ms. Trace:
I20260709 13:06:13.392932 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:13.414801 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2545) took 1153 ms. Trace:
I20260709 13:06:13.414870 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.416400 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.411785 29072 rpcz_store.cc:276] 0709 13:06:12.124050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.146671 (+ 22621us) service_pool.cc:224] Handling call
0709 13:06:12.200744 (+ 54073us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 13:06:12.850394 (+649650us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:12.850440 (+    46us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:06:12.851391 (+   951us) log.cc:844] Serialized 104 byte log entry
0709 13:06:13.319403 (+468012us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:13.319439 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.319495 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:13.410922 (+ 91427us) raft_consensus.cc:1685] finished
0709 13:06:13.410943 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.411110 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":382813,"raft.queue_time_us":396912,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"thread_start_us":467540,"threads_started":1,"wal-append.queue_time_us":546474}
W20260709 13:06:13.353570  2382 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.794s	user 0.012s	sys 0.001s
I20260709 13:06:13.395610 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:13.397646  2441 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:13.418226 29065 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:13.340389  2451 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:13.342804  2313 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.455783 29144 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:13.415462 29068 rpcz_store.cc:276] 0709 13:06:12.261408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.542848 (+281440us) service_pool.cc:224] Handling call
0709 13:06:12.543479 (+   631us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:12.868879 (+325400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:12.868939 (+    60us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:12.869761 (+   822us) log.cc:844] Serialized 52 byte log entry
0709 13:06:13.318474 (+448713us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:06:13.318519 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.319172 (+   653us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:13.414501 (+ 95329us) raft_consensus.cc:1685] finished
0709 13:06:13.414535 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.414727 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":324416,"raft.queue_time_us":326988,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":146,"thread_start_us":448132,"threads_started":1,"wal-append.queue_time_us":537785}
W20260709 13:06:13.420468 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 97096.6us
I20260709 13:06:13.458070 29039 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:13.413159 29070 rpcz_store.cc:276] 0709 13:06:12.145754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.440660 (+294906us) service_pool.cc:224] Handling call
0709 13:06:12.441102 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:12.844160 (+403058us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:12.844192 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:12.845107 (+   915us) log.cc:844] Serialized 26 byte log entry
0709 13:06:13.318114 (+473007us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:13.318162 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.318232 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:13.412419 (+ 94187us) raft_consensus.cc:1685] finished
0709 13:06:13.412443 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.412581 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":402560,"raft.queue_time_us":422449,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":137,"thread_start_us":472347,"threads_started":1,"wal-append.queue_time_us":559999}
I20260709 13:06:13.435261 29071 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:13.435822 29066 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:13.450090 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.450467 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:13.460701 29069 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
W20260709 13:06:13.389626 29064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.009s	sys 0.030s
I20260709 13:06:13.322294  2061 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:13.401387  2312 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.001s
I20260709 13:06:13.322913  2409 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.338661  2246 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.333585  2460 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.314610  1727 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.403829  2470 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:13.512557 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:13.512567 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:13.513289 29069 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:13.514225 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:13.515261 29069 raft_consensus.cc:2410] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:06:13.482410 29147 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:13.413324  2134 raft_consensus.cc:2764] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:06:13.492254 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:13.493106 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 378284us
I20260709 13:06:13.475800 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.543736 29071 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:06:13.545117 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 87216.3us
I20260709 13:06:13.452263  2382 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:13.389210  2461 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:13.454586  2224 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:06:13.414157  2336 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.014s	sys 0.004s
W20260709 13:06:13.517570 29188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.009s	sys 0.013s
W20260709 13:06:13.548877 29188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.010s	sys 0.015s
I20260709 13:06:13.537535 29069 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:06:13.549564 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 735418us
I20260709 13:06:13.340967  2452 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.481415  2454 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:13.527072 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:13.379941  2458 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:13.556854 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 143656us
W20260709 13:06:13.534866  2456 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.235s	user 0.002s	sys 0.000s
W20260709 13:06:13.554066  2336 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.015s	sys 0.004s
W20260709 13:06:13.541854  2316 consensus_meta.cc:287] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.889s	user 0.014s	sys 0.001s
I20260709 13:06:13.563787 29039 leader_election.cc:400] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:06:13.565822 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:13.649461 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 83736.3us
I20260709 13:06:13.650921 29215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:13.651607 29215 raft_consensus.cc:606] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:13.651983 29215 raft_consensus.cc:614] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:13.653882 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:13.657608 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:13.554373  2461 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.472585  2313 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:13.595934 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.594275 29145 raft_consensus.cc:2410] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 13 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 13:06:13.427853  2383 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.535223  2472 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:13.675210 29219 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:13.510497  2394 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.481678  2061 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.661365 29039 leader_election.cc:403] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 13:06:13.684180 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.686687  2464 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.215s	user 0.001s	sys 0.001s
I20260709 13:06:13.688129 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2544) took 1535 ms. Trace:
I20260709 13:06:13.689205  2354 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:13.689664  2354 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:13.690586  2354 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.694213  2458 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:13.695343  2466 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:06:13.696548 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.697372 29216 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:13.700788  2410 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:13.672847 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.635116  2265 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.869s	user 0.014s	sys 0.000s
I20260709 13:06:13.701862  2265 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.668954  2083 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:13.704247 29906 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:13.704640 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2542) took 1563 ms. Trace:
I20260709 13:06:13.705339 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.667156  2485 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:13.706300  2485 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.662632  2382 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
W20260709 13:06:13.594553 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.631124  2448 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:06:13.680925 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 354380us
W20260709 13:06:13.680613 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 255553us
I20260709 13:06:13.688521 29064 rpcz_store.cc:276] 0709 13:06:12.152826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.445167 (+292341us) service_pool.cc:224] Handling call
0709 13:06:12.478299 (+ 33132us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:12.856257 (+377958us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:12.856306 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:12.857103 (+   797us) log.cc:844] Serialized 52 byte log entry
0709 13:06:13.462017 (+604914us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:13.462067 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.462137 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:13.687821 (+225684us) raft_consensus.cc:1685] finished
0709 13:06:13.687844 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.688061 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":358653,"raft.queue_time_us":367133,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":101,"thread_start_us":604430,"threads_started":1,"wal-append.queue_time_us":579172}
I20260709 13:06:13.700035  2059 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:06:13.719615 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:13.711843 29221 consensus_meta.cc:287] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.501s	user 0.006s	sys 0.002s
I20260709 13:06:13.720425 29222 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:13.721082  2147 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:13.721606  2459 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 13:06:13.722165 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.700254 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:13.723307  2467 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:06:13.723626 29909 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
W20260709 13:06:13.724741  2348 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.142s	user 0.011s	sys 0.003s
I20260709 13:06:13.666040  2451 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.725379  2454 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.707746  2422 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:13.726933  2422 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:13.712316 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:13.713768 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.713510 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.732002 29219 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:13.732147 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:13.669160 29072 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:06:13.714433 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.705056 29067 rpcz_store.cc:276] 0709 13:06:12.141298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.411633 (+270335us) service_pool.cc:224] Handling call
0709 13:06:12.412150 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:12.871491 (+459341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:12.871522 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:12.872060 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 13:06:13.319790 (+447730us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:13.319830 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.408972 (+ 89142us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54002aa654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:13.409294 (+   322us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:13.704125 (+294831us) raft_consensus.cc:1685] finished
0709 13:06:13.704152 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.704564 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":264219,"raft.queue_time_us":264398,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"spinlock_wait_cycles":447612800,"thread_start_us":447259,"threads_started":1,"wal-append.queue_time_us":583985}
I20260709 13:06:13.717195 29065 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:13.718401 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.721478 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:13.724817  2455 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.399s	user 0.002s	sys 0.000s
I20260709 13:06:13.736891 29216 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 13: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:13.683765 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 137732us
I20260709 13:06:13.720504 29221 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:13.740633 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:13.742475 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.661270  2461 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:13.679296 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 128947us
I20260709 13:06:13.747291 29221 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:13.749969 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.754772 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:13.760653 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:13.683246  2117 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:13.691901  2354 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:13.593459  2468 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.126s
I20260709 13:06:13.709597  2483 raft_consensus.cc:3055] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:06:13.774050 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:13.775156 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 62908.7us
I20260709 13:06:13.781636 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:13.782243 29218 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:13.782168 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:13.743185  2409 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:13.715010  2485 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:13.785485 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:13.772320 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:13.735011  2364 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.837s	user 0.007s	sys 0.009s
I20260709 13:06:13.787060 29065 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:13.777278 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.780229 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:13.789443 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.789844 29213 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:13.788173 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:13.799280 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:13.802223 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:13.806548  2246 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:13.807298  2246 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:13.807873  2246 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:13.808430  2474 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:06:13.809098  2246 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.812042 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:06:13.814539  2061 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:13.815156  2449 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.746s	user 0.003s	sys 0.000s
I20260709 13:06:13.777046  2458 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:06:13.799309 29188 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:13.773343  2470 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.818864 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:13.798610  2457 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:06:13.774251  2265 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:13.858536 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59328.8us
I20260709 13:06:13.859053 29216 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:13.780651  2410 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:13.890836  2410 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.891368  2410 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:13.896996  2488 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.897624  2488 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.898386  2354 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:13.898900  2354 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:13.900177  2382 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:13.904341  2471 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:13.904990  2471 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:13.905440 29140 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:13.906206  2471 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.912822 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:13.917578 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.862242 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:13.786484  2483 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:13.863114  2364 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:13.863336  2484 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:13.863876  2474 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:13.815164  2334 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:13.832482  2479 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:13.866391 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.725374  2348 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.798748  2347 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:13.786681  2228 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:13.799005  2059 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:13.876396  2418 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:13.719033  2475 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:06:13.880739 29216 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:13.922768  2156 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:06:13.882589 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 83365.6us
I20260709 13:06:13.868273  2478 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:13.710883  2472 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:13.777597  2370 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:13.961218  2315 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:13.962481  2348 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:13.926663 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:13.965188  2445 consensus_meta.cc:287] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.180s	user 0.014s	sys 0.000s
I20260709 13:06:13.965613  2445 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:13.968107 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2719) took 1849 ms. Trace:
I20260709 13:06:13.927572  2488 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:13.942345 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 80136us
I20260709 13:06:13.943547  2304 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.948050 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:13.982110 29064 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:13.947741 29069 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 13:06:13.728139  2422 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:13.951270 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 179068us
W20260709 13:06:13.880620 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 133217us
I20260709 13:06:13.921934  2364 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:13.957922 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:13.791603  2446 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.161s
I20260709 13:06:13.957288  2478 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.962108  2315 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:14.000921 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.925237  2484 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:13.939715 29213 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
W20260709 13:06:14.001658 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:13.968528 29145 rpcz_store.cc:276] 0709 13:06:12.119091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.131753 (+ 12662us) service_pool.cc:224] Handling call
0709 13:06:13.967842 (+1836089us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540029de54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:13.968036 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3328998784}
I20260709 13:06:13.975585 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:13.799064  2485 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:13.950779  2083 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:13.984844 29871 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:14.006438 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:14.006950 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:14.007645 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:14.009852 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:14.010244 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:14.010810 29903 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:14.011050 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:13.958573  2156 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.011417 29037 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:14.012104  2228 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:06:13.986155  2084 consensus_meta.cc:287] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.877s	user 0.011s	sys 0.002s
I20260709 13:06:13.931418 29218 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:13.922492  2470 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.013707  2084 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:14.014231 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:14.015354 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:14.003749 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:13.974164  2460 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:13.992826  2364 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:14.017513 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:13.986153 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:14.015794  2084 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:14.011072  2370 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:14.005151  2484 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:13.989384  2354 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:14.008607  2083 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:14.027021 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:14.017138  2409 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:06:14.037195 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.046731 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:14.007200  2478 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:14.027042 29073 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.049036 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:14.000938  2475 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:06:14.042574 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:14.037351 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:14.011803  2156 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:14.055222 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:14.120693 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:14.027755  2228 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:06:14.042344  2057 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.961s	user 0.015s	sys 0.000s
I20260709 13:06:14.065354  2491 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:06:14.067214  2412 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:14.168256 29220 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:06:14.038108  2460 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:14.048031  2396 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 1.271s	user 0.012s	sys 0.003s
I20260709 13:06:14.026854  2147 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:13.988876  2304 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.069300 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:14.005259  2485 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.008781  1727 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:13.999121  2488 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.009225  2348 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:14.171422 29070 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:14.072713 29139 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:14.072974 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:14.200229 29143 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:14.076136 29903 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:14.040439  2451 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:06:14.067519  2407 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.663s	user 0.011s	sys 0.006s
I20260709 13:06:14.036535  2492 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:14.081640 29219 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:14.057247  2440 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.193311  2485 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:14.112155 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 77022us
I20260709 13:06:14.112257 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:14.116212 29147 raft_consensus.cc:1217] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 7.4->[9.5-18.9]   Dedup: 14.7->[16.8-18.9]
W20260709 13:06:14.081223  2213 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.009s	sys 0.012s
W20260709 13:06:14.393997  2213 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.043s	user 0.009s	sys 0.013s
W20260709 13:06:14.128715 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 142668us
I20260709 13:06:14.385622  2492 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:14.129226  2409 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:14.045037  2441 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:06:14.142521 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:14.145916 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:14.396682  2409 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.397302 29148 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
W20260709 13:06:14.116130  2490 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.115s	user 0.004s	sys 0.000s
I20260709 13:06:14.206821  2488 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:14.193643  2506 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:14.083019  2083 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:14.144888 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:14.174255 29064 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:06:14.118002  2501 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:14.190932  2147 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:14.203086  2500 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:06:14.203575  2370 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.407402  2370 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:14.409464  2495 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.366s	user 0.000s	sys 0.002s
I20260709 13:06:14.200263 29073 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:14.375190 29111 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.003s (SENT) [suppressed 12 similar messages]
I20260709 13:06:14.086223 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:14.411988 29066 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:06:14.412444  2364 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:14.389319 29147 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19. Ops: [9.5-18.9]
I20260709 13:06:14.389369 29141 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:14.390739 29037 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:14.392410  2265 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:14.415712 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:14.417047 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.395538 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:14.419433 29039 leader_election.cc:400] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:14.392930 29903 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:14.392989  2480 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.395749  2492 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:14.419898 29039 leader_election.cc:403] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:14.488328 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:14.493824 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.190719  2396 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:14.204999  2347 consensus_queue.cc:1175] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e78ffbcc98242068a874a5d9e341477" responder_term: 15 status { last_received { term: 15 index: 14 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:06:14.504719 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.523218 29214 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:06:14.501896 29222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.000s	sys 0.004s
I20260709 13:06:14.385779  2470 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:14.526609 29222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.000s	sys 0.004s
W20260709 13:06:14.529603 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 71989.6us
I20260709 13:06:14.531044 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2698) took 1144 ms. Trace:
I20260709 13:06:14.467391  2506 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:14.411616  2440 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:14.537081 29064 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:06:14.524439 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:14.539337 29065 rpcz_store.cc:276] 0709 13:06:13.386455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.449817 (+ 63362us) service_pool.cc:224] Handling call
0709 13:06:14.530772 (+1080955us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400321ed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:14.530965 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590455552}
I20260709 13:06:14.533830 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 804) took 1570 ms. Trace:
I20260709 13:06:14.395077  2304 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:14.549736 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:14.570873 29905 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:14.386591  2407 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.552016 29070 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:14.522267  2441 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.520251  2474 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:14.575228 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 200083us
I20260709 13:06:14.587555 29065 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:14.576171 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51726.5us
I20260709 13:06:14.537758  1727 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.592775 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:14.591485 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:14.577805 29073 rpcz_store.cc:276] 0709 13:06:12.963482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.963862 (+   380us) service_pool.cc:224] Handling call
0709 13:06:12.964263 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:14.019289 (+1055026us) spinlock_profiling.cc:243] Waited 69 ms on lock 0x7b54001034a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a8b06f 00007f7836b03f60 00007f7836b057de 00007f7836b0fb49 00007f7836b10416 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62
0709 13:06:14.532837 (+513548us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:06:14.532912 (+    75us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:14.532998 (+    86us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:14.533107 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:14.533267 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:14.533681 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":71363,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":20331,"spinlock_wait_cycles":158753024,"thread_start_us":62674,"threads_started":1}
I20260709 13:06:14.539386  2315 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:14.494021  2265 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.494283  2213 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:06:14.540356  2370 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:14.595996 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:14.548444  2117 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
W20260709 13:06:14.640486 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 49084.3us
I20260709 13:06:14.661221 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:14.640901 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:14.498730  2480 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.642041 29065 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:14.574051  2506 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:14.548769  2348 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:14.586545  2479 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:14.623596  1727 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:14.596587  2364 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:14.670646 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:14.676479 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 80631us
I20260709 13:06:14.685843 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:14.687355 29073 raft_consensus.cc:2410] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:14.670257 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:14.593950  2441 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.683601  2460 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:14.686853 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:14.679312  2445 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:14.680860  2147 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:14.682439 29141 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:06:14.684198  2486 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.467s	user 0.001s	sys 0.001s
I20260709 13:06:14.684820  2504 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:06:14.685194  2493 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.384s	user 0.001s	sys 0.001s
I20260709 13:06:14.689937 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:14.752599 29073 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:14.753545 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:14.688762 29065 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:14.703181 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:14.735471 29148 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: [13.7-15.8]
W20260709 13:06:14.735598 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 48866.9us
I20260709 13:06:14.685068  2483 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.743438 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:14.668665  2497 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:14.735044  2451 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:06:14.749051  2515 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:06:14.761073 29143 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:14.597600  2474 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:14.758596 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:14.744895  2228 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:06:14.771988  2507 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:06:14.772486  2529 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.773706  2480 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:14.768653  2451 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:14.803408  2451 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:14.803817  2469 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.253s	user 0.005s	sys 0.009s
I20260709 13:06:14.804710  2370 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:06:14.771800  2517 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:06:14.791378 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 894) took 1138 ms. Trace:
I20260709 13:06:14.787818 29065 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:14.765296  2383 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:14.790175 29113 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:14.799073 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:14.687165  2348 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:14.789529  2059 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.838s	user 0.011s	sys 0.001s
I20260709 13:06:14.801448  2529 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:14.802320  2484 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.683s	user 0.000s	sys 0.011s
I20260709 13:06:14.808552 29906 rpcz_store.cc:276] 0709 13:06:13.653463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.653632 (+   169us) service_pool.cc:224] Handling call
0709 13:06:14.791122 (+1137490us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b5400499b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:14.791299 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113151360}
I20260709 13:06:14.862282 29065 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:06:14.793010 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 124237us
I20260709 13:06:14.864354 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2789) took 1060 ms. Trace:
I20260709 13:06:14.775275  2506 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.760087  2147 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:06:14.807252  2485 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:14.865835 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2693) took 1482 ms. Trace:
I20260709 13:06:14.801573  2528 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.879325 29071 rpcz_store.cc:276] 0709 13:06:13.383560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.545793 (+162233us) service_pool.cc:224] Handling call
0709 13:06:13.546048 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:13.546337 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:13.546366 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:13.547122 (+   756us) log.cc:844] Serialized 26 byte log entry
0709 13:06:13.741776 (+194654us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:06:13.741832 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.741920 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:14.865624 (+1123704us) raft_consensus.cc:1685] finished
0709 13:06:14.865645 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:14.865778 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":153195,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":302,"thread_start_us":129078,"threads_started":1,"wal-append.queue_time_us":383822}
I20260709 13:06:14.869836 29906 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:14.869635 29903 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:06:14.882781 29903 pending_rounds.cc:85] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 9
I20260709 13:06:14.883467 29903 pending_rounds.cc:107] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:06:14.884176 29903 raft_consensus.cc:2889] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:14.817342  2228 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:14.859392  2059 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:14.864817 29222 rpcz_store.cc:276] 0709 13:06:13.803494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.803627 (+   133us) service_pool.cc:224] Handling call
0709 13:06:13.803945 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:13.804174 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:13.984308 (+180134us) log.cc:844] Serialized 14 byte log entry
0709 13:06:14.797802 (+813494us) log.cc:844] Serialized 14 byte log entry
0709 13:06:14.863741 (+ 65939us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:14.864002 (+   261us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:14.864054 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:14.864125 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:14.864275 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":143063,"prepare.queue_time_us":255878,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":52,"raft.queue_time_us":210404,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":17074,"thread_start_us":875195,"threads_started":4,"wal-append.queue_time_us":586323}
I20260709 13:06:14.860558  2480 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.877532  2084 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:06:14.908685 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 489402us
I20260709 13:06:14.909894 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:14.886479  2461 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.106s	user 0.012s	sys 0.001s
W20260709 13:06:14.886829  2410 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.995s	user 0.013s	sys 0.000s
I20260709 13:06:14.902765 29065 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:14.903625 29067 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:14.913733 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:14.898461  2057 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:06:14.907409 29216 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.889034  2479 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:14.865937  2484 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:14.899516  2470 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.895467  2514 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:14.898655  2525 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.819439  2483 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:14.928146 29039 leader_election.cc:400] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:14.903615  2084 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:14.933548 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 134741us
I20260709 13:06:14.790130  2364 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 13:06:14.919178  2504 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:14.958585 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:14.864195  2383 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:14.933347  2083 consensus_meta.cc:287] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.005s
I20260709 13:06:14.896099  2528 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.977969  2383 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:14.978112  2083 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:14.979638  2528 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:14.980651 29144 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:14.981393 29909 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:06:14.932721  2396 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:14.955886  2084 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:14.920708  2461 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.923153  2410 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:14.972939 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:14.899012  2480 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.971355  2265 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:14.975402 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47468.5us
I20260709 13:06:14.920840  2370 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:14.995534 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:14.971225 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:14.986595 29143 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:14.971143  2518 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
I20260709 13:06:14.931139  2532 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:14.988741 29141 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:06:14.974676 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 95646.8us
I20260709 13:06:15.007844 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.008699 29142 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:14.937031  2470 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.986052  1727 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.009521 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:15.012552 29148 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:14.970148  2530 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:06:14.992061  2147 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.997840  2474 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:14.985239  2504 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:14.992141  2441 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.025727  2441 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.025880  2523 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:15.026271  2441 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:14.956631  2514 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.005203  2117 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:15.018184 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:15.015080 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:15.029516  2440 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.020352 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:14.957449  2525 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.024014 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2563) took 1103 ms. Trace:
I20260709 13:06:14.983253  2228 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.045814 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:15.039556 29072 rpcz_store.cc:276] 0709 13:06:13.920058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.533403 (+613345us) service_pool.cc:224] Handling call
0709 13:06:14.533906 (+   503us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:14.541730 (+  7824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.541777 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:14.552654 (+ 10877us) log.cc:844] Serialized 52 byte log entry
0709 13:06:14.769182 (+216528us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:14.977660 (+208478us) log.cc:844] Serialized 14 byte log entry
0709 13:06:14.985551 (+  7891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:14.985652 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:15.023705 (+ 38053us) raft_consensus.cc:1685] finished
0709 13:06:15.023727 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.023871 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14733,"prepare.queue_time_us":243918,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":1953,"raft.queue_time_us":53853,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":15798,"thread_start_us":282642,"threads_started":2,"wal-append.queue_time_us":309652}
I20260709 13:06:15.047298 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:15.049182 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:15.050359 29072 raft_consensus.cc:2368] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14.
I20260709 13:06:15.056630  2532 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:15.072652 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.073442 29140 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:15.074468 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.075207 29140 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 15: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:15.037367  2348 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.076371 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.077528 29140 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:15.077322  2451 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.078766 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.079654 29140 raft_consensus.cc:2410] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 14 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 13:06:15.080152  2542 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:15.080744  2542 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:15.081768  2542 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.076749  2504 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.084662  2529 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.081681  2523 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:06:15.085850  2484 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.086862  2523 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:15.087272  2523 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:15.088052  2364 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:15.088438  2470 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.088519  2441 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.090338  2480 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.036849  2530 consensus_queue.cc:1243] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:06:15.091717  2538 log.cc:927] Time spent T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:06:15.093266  2492 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.006s
I20260709 13:06:15.093742  2461 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.093669  2492 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.098762 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2564) took 1178 ms. Trace:
I20260709 13:06:15.099293 29070 rpcz_store.cc:276] 0709 13:06:13.920268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.574225 (+653957us) service_pool.cc:224] Handling call
0709 13:06:14.574601 (+   376us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:14.690611 (+116010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.690657 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:14.739748 (+ 49091us) log.cc:844] Serialized 52 byte log entry
0709 13:06:14.793779 (+ 54031us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:14.793835 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:14.793924 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:15.098432 (+304508us) raft_consensus.cc:1685] finished
0709 13:06:15.098472 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.098685 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4190,"raft.queue_time_us":49519,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":16493,"thread_start_us":133325,"threads_started":2,"wal-append.queue_time_us":143274}
I20260709 13:06:15.101626 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.102433 29070 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:15.103744 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:15.105590 29140 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:15.103744 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.106431 29070 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:15.055637 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:15.112722 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46734.2us
I20260709 13:06:15.113950 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:15.115792 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 60264.7us
I20260709 13:06:15.095696  2492 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.120110 29072 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:06:15.121675 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 331549us
I20260709 13:06:15.123144 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.125576 29141 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 13.
I20260709 13:06:15.124235 29039 leader_election.cc:400] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:06:15.126174 29039 leader_election.cc:403] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:15.121982  2348 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
W20260709 13:06:15.128527  2494 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.957s	user 0.000s	sys 0.006s
I20260709 13:06:15.128302  2523 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.129814  2523 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:15.131136 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2784) took 1962 ms. Trace:
I20260709 13:06:15.122352  2484 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:06:15.133139  2484 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:15.133541  2484 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:15.134657  2525 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.134441  2484 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.119827  2461 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:15.122406  2460 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:06:15.137153  2407 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.137725  2370 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:15.103835  2536 consensus_queue.cc:1175] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e78ffbcc98242068a874a5d9e341477" responder_term: 16 status { last_received { term: 15 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:15.140705 29188 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:15.141294 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:15.141820  2536 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:15.142140  2410 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.024053  2147 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.142956  2536 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:15.143232  2452 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.143375  2536 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.144300 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.145167  2147 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.145706  2364 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:15.145807  2147 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:15.147424  2147 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:15.147535 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2570) took 1108 ms. Trace:
I20260709 13:06:15.122206  2545 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:15.149256  2484 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:15.148730  2545 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.145108 29906 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:06:15.150399 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2576) took 1076 ms. Trace:
I20260709 13:06:15.131536 29217 rpcz_store.cc:276] 0709 13:06:13.169271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.169419 (+   148us) service_pool.cc:224] Handling call
0709 13:06:13.186861 (+ 17442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:13.678208 (+491347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:13.678276 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:13.679245 (+   969us) log.cc:844] Serialized 26 byte log entry
0709 13:06:13.923369 (+244124us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:13.923452 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.130736 (+1207284us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400282dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:15.130783 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:15.130829 (+    46us) raft_consensus.cc:1685] finished
0709 13:06:15.130842 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.131077 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86611,"raft.queue_time_us":191807,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":711,"spinlock_wait_cycles":620514560,"thread_start_us":358967,"threads_started":2,"wal-append.queue_time_us":348221,"wal-append.run_cpu_time_us":2697,"wal-append.run_wall_time_us":1093618}
I20260709 13:06:15.124362  2530 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:06:15.143810  2536 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:15.186057  2334 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:15.186509  2334 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:15.186697  2492 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.149741  2526 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.211s
I20260709 13:06:15.187299  2492 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.188473  2442 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:15.188858  2442 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:15.189035  2545 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:15.189703  2442 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.190804 29141 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:06:15.190909  2442 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:15.192031 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:15.192622 29141 raft_consensus.cc:2376] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15. Re-sending same reply.
W20260709 13:06:15.151954  2478 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.011s	user 0.013s	sys 0.000s
I20260709 13:06:15.111406  2485 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.148103 29145 rpcz_store.cc:276] 0709 13:06:14.038803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.038973 (+   170us) service_pool.cc:224] Handling call
0709 13:06:14.138271 (+ 99298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:14.138844 (+   573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.138875 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:14.139699 (+   824us) log.cc:844] Serialized 26 byte log entry
0709 13:06:14.420256 (+280557us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:14.420399 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:14.420599 (+   200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:14.923978 (+503379us) raft_consensus.cc:1685] finished
0709 13:06:14.924008 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:15.147269 (+223261us) spinlock_profiling.cc:243] Waited 98.9 ms on lock 0x7b5400145850. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:15.147446 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1649,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":23682,"spinlock_wait_cycles":227515648,"thread_start_us":279908,"threads_started":1,"wal-append.queue_time_us":397179}
I20260709 13:06:15.194378 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.194556  2441 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:15.195211  2492 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.195794 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.197499 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.198230 29217 raft_consensus.cc:2376] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15. Re-sending same reply.
I20260709 13:06:15.199144 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.200939 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2560) took 1289 ms. Trace:
I20260709 13:06:15.201440 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.201319 29064 rpcz_store.cc:276] 0709 13:06:13.911685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.401585 (+489900us) service_pool.cc:224] Handling call
0709 13:06:14.536799 (+135214us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:14.571641 (+ 34842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.200742 (+629101us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400321ed0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:15.200884 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309970688}
I20260709 13:06:15.195514 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.202514 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.157140 29215 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:15.157352  2525 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.156487  2461 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:15.159844 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.159852 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.081861  2514 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.162413 29221 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:06:15.184480 29142 rpcz_store.cc:276] 0709 13:06:14.074518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.074761 (+   243us) service_pool.cc:224] Handling call
0709 13:06:14.075261 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.009128 (+933867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.150164 (+141036us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b54000e9cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:15.150334 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2146268544}
I20260709 13:06:15.231458 29221 pending_rounds.cc:85] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 10
I20260709 13:06:15.231698 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.163676 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.075547  2383 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.125442  2451 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.172034  2117 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:15.240445 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.240855 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.240923 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.195226  2332 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:15.177736 29140 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:15.193545  2478 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.193521 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:15.203899 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.156109  2470 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:15.153147  2345 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:15.222103 29068 pending_rounds.cc:85] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 13
I20260709 13:06:15.250937 29068 pending_rounds.cc:107] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 14.14
I20260709 13:06:15.251242 29068 raft_consensus.cc:2889] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:15.251586 29903 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:06:15.225901  2474 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:15.211520 29216 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:15.229542 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.231979 29221 pending_rounds.cc:107] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:06:15.163041 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.232615 29214 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:15.141291  2156 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.991s	user 0.008s	sys 0.003s
I20260709 13:06:15.169034  2529 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.238561 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:15.238593 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.147845  2147 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:15.254070 29222 raft_consensus.cc:2393] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 13: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:15.242292 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 123103us
I20260709 13:06:15.171847 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:15.254940 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.255290 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:15.255448 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 83705.2us
I20260709 13:06:15.256196 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:15.248437 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.187388  2334 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.223201  2347 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:06:15.168958  2084 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.266935 29140 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:06:15.258409 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:15.252981 29221 raft_consensus.cc:2889] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:15.267748 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.256531 29909 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:15.264359 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:15.255892 29139 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:15.227190  2458 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:06:15.272524 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.273355 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:15.275627 29146 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 13:06:15.277751 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.277911 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:15.247082  2332 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:15.273734 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:15.238886  2382 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:06:15.287575 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.346487 29904 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:06:15.284596 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.343386 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.348143 29217 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:15.265753  2407 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:15.280682 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:15.234144  2359 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:06:15.345304 29909 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:15.247010  2492 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:06:15.272765  2504 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.231458  2525 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:15.349396 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 75735.7us
I20260709 13:06:15.358150 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:15.254453  2470 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:15.350759 29216 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:06:15.360036 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2788) took 1568 ms. Trace:
I20260709 13:06:15.280272  2334 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:15.361310  2549 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:06:15.363471 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.364210 29219 raft_consensus.cc:2376] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14. Re-sending same reply.
I20260709 13:06:15.364290 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.365139 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.365720 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.366763  2147 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.372184  2524 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:15.372926  2524 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.375267  2524 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.375406  2334 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:15.376509  2332 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.378059  2332 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:15.378564  2334 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:15.380079  2477 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:06:15.252930  2345 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:15.360487 29216 rpcz_store.cc:276] 0709 13:06:13.791191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.832184 (+ 40993us) service_pool.cc:224] Handling call
0709 13:06:13.858895 (+ 26711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:14.906865 (+1047970us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b4c0006a180. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7834dd9a20 00007f7834deaa29 00007f7834db0644 00007f7834dd7f2d 00007f7834dd7b93 00007f7836b0818b 00007f7836b08cb5 00007f7836b03f30 00007f7836b057de 00007f7836b0fb49 00007f7836b10416
0709 13:06:15.210778 (+303913us) spinlock_profiling.cc:243] Waited 46.9 ms on lock 0x7b58009170d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a690aa 00007f7836aeb447 00007f7836b03fe7 00007f7836b057de 00007f7836b0fb49 00007f7836b10416 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4
0709 13:06:15.359655 (+148877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.359952 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168292608}
W20260709 13:06:15.284098 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.262665  2533 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:15.353209 29903 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:15.271230  2535 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:15.279636  2458 consensus_queue.cc:1243] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:06:15.280857  2483 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:15.338845  2445 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.355559 29902 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:06:15.388301 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:15.284230 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.404618 29214 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:15.281428  2396 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.280428  2265 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:15.428133  2265 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.428632  2265 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:15.429152  2536 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.430776 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.431456 29071 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:15.342329  2528 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:15.432483 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.432598  2396 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.433077  2334 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.433548 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:15.434118  2545 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.434588 29909 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:15.383133  2470 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:15.435364  2396 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.266512  2529 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.435834  2383 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.398413  2504 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.436614  2545 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.434149 29071 raft_consensus.cc:2376] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17. Re-sending same reply.
I20260709 13:06:15.438105 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:15.438542  2535 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:15.438743 29906 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:15.438962  2528 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:15.439172  2084 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.418601 29903 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:15.422969 29908 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:15.355836  2117 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:15.426204  2348 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.426863 29902 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:06:15.427071  2461 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.453816  2461 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.456240 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2695) took 2070 ms. Trace:
I20260709 13:06:15.456910 29069 rpcz_store.cc:276] 0709 13:06:13.385563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.550182 (+164619us) service_pool.cc:224] Handling call
0709 13:06:13.565558 (+ 15376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:13.983076 (+417518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.455654 (+1472578us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b5400186d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:15.455856 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877737728}
I20260709 13:06:15.427196  2525 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.439247 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.439570  2529 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.275008  2485 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.137s
I20260709 13:06:15.442799 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.356453  2407 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.443327 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.265552  2156 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.450922  2345 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.431468  2536 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.463181 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.438066  2504 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.449185  2117 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:15.472141 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:15.500658 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.461900  2348 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.496845  2407 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:15.271080  2476 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.471513  2472 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.498239  2525 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:15.521643 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.423403  2445 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.411981  2533 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.520866 29218 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:15.511459  2485 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:15.537647 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:15.532155  2476 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.540612 29069 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:06:15.542914  2445 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.544193  2485 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.545271  2461 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.545634  2533 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.545856  2345 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:15.546839  2485 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.519375  2117 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:15.572818  2488 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.576013  2506 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:15.576669  2506 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:15.575798  2525 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.577572  2506 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.578595 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.579274 29221 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:15.579397  2347 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:06:15.581331  2528 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.583293  2552 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.583971  2552 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.586650  2552 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.555891  2478 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.528597  2482 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:15.587965  2525 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.592315 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.593394 29903 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:15.551644  2348 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.600729 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:15.553131  2156 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:15.553483  2497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.003s	sys 0.011s
I20260709 13:06:15.608578 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.612460 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:15.615686 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:15.619391 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.620191 29068 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:15.621229 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:15.621835 29068 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:15.610270  2497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.003s	sys 0.011s
I20260709 13:06:15.622864 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.636243 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:15.624221 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.608390  2478 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:06:15.619721  2488 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:15.631815  2523 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.000s
I20260709 13:06:15.596522  2492 consensus_queue.cc:1243] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:15.639537 29145 raft_consensus.cc:2410] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:06:15.635008 29871 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:15.645759 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.648295 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.623598 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:15.607261  2359 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:06:15.657271 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.627781  2497 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:15.610620  2441 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:06:15.624531  2491 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:15.659691 29221 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:15.660583 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.670800 29214 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:15.671098 29068 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:15.664182 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.642113  2535 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:15.641206  2504 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:15.668406  2492 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.658500  2475 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:06:15.678565 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.695698 29217 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:15.695887  2488 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:15.696372  2488 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.696735  2488 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:15.696645 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.680434 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.677850 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:15.676265 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.658340  2552 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:15.699565 29068 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:15.699815 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.694032 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:15.702310 29216 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
W20260709 13:06:15.694694 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 71264.2us
I20260709 13:06:15.703266 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.703542 29185 leader_election.cc:400] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14.
I20260709 13:06:15.690986 29064 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:15.686836 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:15.703838 29185 leader_election.cc:403] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:06:15.685721 29214 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:15.684397 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.703840 29216 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
W20260709 13:06:15.711226 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:15.686872  2484 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.000s
I20260709 13:06:15.678246  2504 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.678980  2492 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.714692 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:15.708016  2552 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.730228 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:15.686533  2497 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.730424  2552 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:15.709677  2475 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14.
I20260709 13:06:15.736696 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:15.737480 29216 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:15.720394  2470 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.741641  2492 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.709910  2477 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.729902  2504 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:15.721359 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:15.743621 29217 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:15.744169 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:15.724195  2535 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.747251 29902 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:15.727056 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:15.728227 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.715466 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.680832  2117 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.741799  2470 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:15.722561 29145 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:15.751734 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.742633  2477 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.769906 29069 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:15.779489 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.779758 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.781700 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:15.782437 29069 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:06:15.792447 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2582) took 1113 ms. Trace:
I20260709 13:06:15.753296  2562 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:15.792877 29146 rpcz_store.cc:276] 0709 13:06:14.679042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.684055 (+  5013us) service_pool.cc:224] Handling call
0709 13:06:14.684422 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:15.276135 (+591713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.792195 (+516060us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b54003a4dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:15.792385 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1358863104}
I20260709 13:06:15.793643  2562 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:15.794574  2562 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.767524  2561 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:15.796294  2561 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.796700  2561 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:15.797667  2117 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.800011  2477 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.767691  2265 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:15.800633 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.802033  2265 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.804557 29910 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:15.804973 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.806865  2383 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:15.806978 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:15.807418  2383 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:15.808308 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:15.808943 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.809335 29065 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:15.814642 29221 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:15.815995 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:15.753590  2567 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:15.819077 29145 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:15.768713  2484 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.823411 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:15.824123 29065 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:15.826824 29221 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
W20260709 13:06:15.838943 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.823262  2567 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.849692 29911 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:06:15.852276 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:15.854197 29141 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:15.860860 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:15.868896 29145 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
W20260709 13:06:15.870460 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.520263 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:15.876127 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:15.876786 29908 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:15.826790  2484 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.873389 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:15.859601  2567 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:15.886682 29186 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:15.887328 29112 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:15.898563 29069 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:15.900802 29142 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: [13.7-15.8]
I20260709 13:06:15.870139  2441 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.823087  2383 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.901667 29141 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:15.908878  2407 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.910247  2488 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.894874  2470 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:15.900887  2479 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:15.934494  2479 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.934998  2479 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:15.930392  2442 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.900712  2336 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:15.935662  2442 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:15.936226  2336 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.936684  2336 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:15.938060  2442 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.940366  2484 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.941581 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:15.943298  2513 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.169s	user 0.003s	sys 0.001s
I20260709 13:06:15.944157  2470 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:15.933861  2441 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.945266  2470 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:15.946798  2441 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:15.950016  2407 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.950181  2313 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:15.951355 29904 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:06:15.933334 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:15.957603 29069 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:06:15.958830 29069 pending_rounds.cc:85] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 11
I20260709 13:06:15.960321 29069 pending_rounds.cc:107] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:06:15.964041 29069 raft_consensus.cc:2889] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:16.001865 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 198021us
I20260709 13:06:16.007030 29142 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:16.012370 29141 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:16.015156 29216 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:16.002770  2441 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.010537  2535 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:16.024072 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:16.009665  2471 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.031360 29142 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:15.956169  2504 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.047691 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:16.049211 29141 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:16.005035  2492 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:06:16.073954  2061 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:16.087440  2061 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:16.089219  2061 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:16.090715  2579 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:06:16.091009  2061 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:16.093313  2552 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.095563  2552 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:16.096891 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:16.107352  2580 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:06:16.108513 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.109638  2488 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:16.111445  2504 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:16.029489  2567 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.112543  2560 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:16.113200  2560 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:16.114322  2560 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.115223  2567 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:16.115702  2560 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:16.116039  2562 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:16.117069  2562 consensus_queue.cc:1243] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:06:16.117717  2562 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:16.118158  2562 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:16.046277  2535 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:16.123881 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:16.125430 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.134243 29142 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:16.136713 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:16.142428 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.145287  2567 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:16.172989  2567 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:16.174247  2567 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.047971  2471 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.175566  2336 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.033747  2441 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.183934 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:16.186828  2471 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:16.190840 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:16.191557 29142 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:16.192675 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:16.032392  2407 consensus_queue.cc:1059] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:06:16.141804  2478 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.142274  2442 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.119246  2562 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.192619  2535 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.244565 29065 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:16.264470 29221 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:16.268419 29221 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:16.233798  2441 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:16.242352  2336 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:16.276507  2561 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:16.279066 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.243605  2083 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:16.224251  2478 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:16.277283 29142 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:16.233435  2442 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:16.269958  2562 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:16.246742  2479 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:16.297099 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 365305us
W20260709 13:06:16.302736 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.282658  2561 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:16.318392  2479 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:16.323879 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2607) took 1089 ms. Trace:
I20260709 13:06:16.324473 29145 rpcz_store.cc:276] 0709 13:06:15.234125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.883703 (+649578us) service_pool.cc:224] Handling call
0709 13:06:15.902371 (+ 18668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:15.902723 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:15.902749 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:15.903214 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:06:16.031139 (+127925us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:16.031177 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.031241 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:16.322557 (+291316us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b240009a178. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78338a69c4 00007f78338a4acb 00007f7833a5fdc9 00007f7833a5fa55 00007f7833a5efb0 00007f78366d5488 00007f78366d5be8 00007f78366ad398 00007f78366a876a 00007f78366a7bb6
0709 13:06:16.323457 (+   900us) raft_consensus.cc:1685] finished
0709 13:06:16.323486 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.323789 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1213,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":2870,"spinlock_wait_cycles":31051520,"thread_start_us":127595,"threads_started":1,"wal-append.queue_time_us":279324}
I20260709 13:06:16.333091 29217 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:06:16.349677  2504 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.350409  2504 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.353046 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2573) took 1423 ms. Trace:
I20260709 13:06:16.353698 29064 rpcz_store.cc:276] 0709 13:06:14.930092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.202286 (+272194us) service_pool.cc:224] Handling call
0709 13:06:15.202578 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.704460 (+501882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.352683 (+648223us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54002085d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:16.352949 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1122337664}
I20260709 13:06:16.355494  2504 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:16.358839 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2718) took 1123 ms. Trace:
I20260709 13:06:16.359254 29071 rpcz_store.cc:276] 0709 13:06:15.235140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.497819 (+262679us) service_pool.cc:224] Handling call
0709 13:06:15.517335 (+ 19516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.580221 (+ 62886us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:15.580258 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:15.580303 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:15.580342 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.358589 (+778247us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b54002ac954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:16.358636 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.358777 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144121600}
I20260709 13:06:16.368040 29064 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:16.372162 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:16.306111  2336 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.376075 29071 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:06:16.379014 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:16.387795  2504 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:16.391520 29113 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:16.380437  2228 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:06:16.425983 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.384950  2336 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.396404  2083 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:16.420886  2228 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:16.428051  2478 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:16.473595 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2713) took 2739 ms. Trace:
I20260709 13:06:16.474050 29215 rpcz_store.cc:276] 0709 13:06:13.734494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.763252 (+ 28758us) service_pool.cc:224] Handling call
0709 13:06:13.797276 (+ 34024us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.222586 (+1425310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.473318 (+1250732us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002841d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:16.473528 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3126899072}
I20260709 13:06:16.393961  2504 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:16.475075 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2704) took 1886 ms. Trace:
I20260709 13:06:16.478237 29073 rpcz_store.cc:276] 0709 13:06:14.588975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.791569 (+202594us) service_pool.cc:224] Handling call
0709 13:06:14.795299 (+  3730us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:14.795530 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:14.957009 (+161479us) log.cc:844] Serialized 14 byte log entry
0709 13:06:16.474492 (+1517483us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b18003aaf20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a
0709 13:06:16.474728 (+   236us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:16.474796 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:16.474841 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.474911 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.475025 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45544,"prepare.queue_time_us":224200,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":1931,"spinlock_wait_cycles":69445504,"thread_start_us":160722,"threads_started":2,"wal-append.queue_time_us":93351,"wal-append.run_cpu_time_us":1183,"wal-append.run_wall_time_us":1003906}
W20260709 13:06:16.477371 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 58652.7us
I20260709 13:06:16.477625  2442 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:16.494556 29220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 938) took 2741 ms (client timeout 2821 ms). Trace:
W20260709 13:06:16.495169 29220 rpcz_store.cc:269] 0709 13:06:13.749549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.749668 (+   119us) service_pool.cc:224] Handling call
0709 13:06:16.490733 (+2741065us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b540026b6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:16.491034 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":959484800}
I20260709 13:06:16.474424  2083 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:06:16.504889 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 895) took 2844 ms. Trace:
I20260709 13:06:16.505544 29905 rpcz_store.cc:276] 0709 13:06:13.655702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.655884 (+   182us) service_pool.cc:224] Handling call
0709 13:06:13.656220 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:14.572688 (+916468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.499353 (+1926665us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b54002a08d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:16.499685 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2061568896}
I20260709 13:06:16.477370  2478 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:06:16.512394 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:16.512274  2505 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:06:16.510267  2442 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
W20260709 13:06:16.514345 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.529402 29905 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:16.534552 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2573) took 2470 ms. Trace:
I20260709 13:06:16.538918 29147 rpcz_store.cc:276] 0709 13:06:14.064277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.064435 (+   158us) service_pool.cc:224] Handling call
0709 13:06:14.066056 (+  1621us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 13:06:14.414427 (+348371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.534167 (+2119740us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b540027ced4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:16.534472 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114338560}
I20260709 13:06:16.543009 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:16.544961 29905 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:16.548823  2606 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:06:16.526912  2589 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:06:16.524670  2593 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:06:16.515755  2505 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:16.524232  2313 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.549391  2606 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:16.562491  2591 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:06:16.563233 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:16.569893  2505 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:16.655187  2592 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.151s	user 0.006s	sys 0.003s
W20260709 13:06:16.667757 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2558) took 2756 ms (client timeout 2010 ms). Trace:
W20260709 13:06:16.668366 29067 rpcz_store.cc:269] 0709 13:06:13.910934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:13.926328 (+ 15394us) service_pool.cc:224] Handling call
0709 13:06:16.667515 (+2741187us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b54003edfd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:16.667662 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2151419264}
I20260709 13:06:16.670843  2504 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:16.671437  2504 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:16.673575  2504 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:16.675901  2504 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:16.676663  2505 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.677028  2606 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:16.678088  2505 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:16.678157 29905 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:16.679232 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:16.678433  2606 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:16.682353  2474 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.682997  2474 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:16.684419 29904 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:06:16.686983 29904 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:06:16.691260  2476 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:16.700276 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:16.703431 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.517403  2442 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:06:16.656993 29908 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:06:16.658043  2336 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:16.533882  2228 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:06:16.556684  2454 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.705439  2454 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:16.658543  2061 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.567s	user 0.020s	sys 0.001s
W20260709 13:06:16.661002 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:16.714409 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.714285 29186 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:16.717584 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 228035us
I20260709 13:06:16.515492  2604 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:06:16.666582 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2585) took 1986 ms. Trace:
W20260709 13:06:16.694454  2521 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.890s	user 0.004s	sys 0.001s
W20260709 13:06:16.671777  2573 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.741s	user 0.000s	sys 0.002s
I20260709 13:06:16.656032  2313 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.718881 29144 rpcz_store.cc:276] 0709 13:06:14.701754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.702003 (+   249us) service_pool.cc:224] Handling call
0709 13:06:14.702474 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:14.981070 (+278596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.666249 (+1685179us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400145354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:16.666461 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639292160}
I20260709 13:06:16.734845 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:16.736213 29216 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:06:16.737387 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2595) took 1018 ms. Trace:
I20260709 13:06:16.739310 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:16.742024 29220 raft_consensus.cc:2410] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:06:16.747037 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.747340 29221 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:16.741344 29071 rpcz_store.cc:276] 0709 13:06:15.719107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.378974 (+659867us) service_pool.cc:224] Handling call
0709 13:06:16.387989 (+  9015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.388182 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:16.392772 (+  4590us) log.cc:844] Serialized 14 byte log entry
0709 13:06:16.726015 (+333243us) log.cc:844] Serialized 14 byte log entry
0709 13:06:16.736886 (+ 10871us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:16.736966 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:16.737043 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.737141 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.737310 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7331,"prepare.queue_time_us":6326,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":100,"raft.queue_time_us":134175,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":332440,"threads_started":2,"wal-append.queue_time_us":165045}
W20260709 13:06:16.756043 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.725208  2561 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:16.759907  2441 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:16.769083 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2561) took 2849 ms. Trace:
I20260709 13:06:16.785789 29908 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:16.786880  2609 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.787904 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2572) took 1862 ms. Trace:
I20260709 13:06:16.751195  2084 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.788318 29072 rpcz_store.cc:276] 0709 13:06:14.926299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.108740 (+182441us) service_pool.cc:224] Handling call
0709 13:06:15.109033 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.121268 (+ 12235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.787659 (+1666391us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b54002aba54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:16.787850 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14525696}
I20260709 13:06:16.789183  2084 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:16.790625 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:16.769644  2454 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:16.787420  2609 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:16.791299 29904 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:16.773401  2313 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:16.778411  2382 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:06:16.793429  2147 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:16.794071  2609 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:16.785590 29066 rpcz_store.cc:276] 0709 13:06:13.919463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.412671 (+493208us) service_pool.cc:224] Handling call
0709 13:06:14.413015 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:14.413365 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:14.413392 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:14.413882 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:06:15.218014 (+804132us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b18004b3520. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1
0709 13:06:15.218261 (+   247us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:15.218315 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:15.218407 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:16.693828 (+1475421us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:16.768447 (+ 74619us) raft_consensus.cc:1685] finished
0709 13:06:16.768468 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.769005 (+   537us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25032,"raft.queue_time_us":3049,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":70637,"spinlock_wait_cycles":60854784,"thread_start_us":113434,"threads_started":1,"wal-append.queue_time_us":375174}
I20260709 13:06:16.727562  2476 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:06:16.757659  2604 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:06:16.795528 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:16.803120 29905 raft_consensus.cc:2410] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
W20260709 13:06:16.797227 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:16.809412 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.809871 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:16.812618 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:16.791767  2084 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:16.816066 29215 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:16.812723  2476 consensus_queue.cc:1243] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:06:16.812216  2561 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:16.840636  2482 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:16.840754  2561 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.815557  2336 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:06:16.841799 29911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 961) took 1557 ms. Trace:
I20260709 13:06:16.842227 29911 rpcz_store.cc:276] 0709 13:06:15.284414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.284729 (+   315us) service_pool.cc:224] Handling call
0709 13:06:15.356374 (+ 71645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:15.853493 (+497119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.841524 (+988031us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54005a32d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:16.841734 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1133794048}
I20260709 13:06:16.841361  2482 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:16.846720 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.847971 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2582) took 1222 ms. Trace:
I20260709 13:06:16.849781 29065 rpcz_store.cc:276] 0709 13:06:15.626009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.256340 (+630331us) service_pool.cc:224] Handling call
0709 13:06:16.275648 (+ 19308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:16.275860 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:06:16.464798 (+188938us) log.cc:844] Serialized 14 byte log entry
0709 13:06:16.668478 (+203680us) log.cc:844] Serialized 14 byte log entry
0709 13:06:16.847587 (+179109us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:16.847642 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:16.847690 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.847757 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:16.847890 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28053,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":126973,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":551319,"threads_started":4,"wal-append.queue_time_us":94412}
I20260709 13:06:16.852141  2604 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:16.872844  2336 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:16.906436  2474 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.908146  2604 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.910621  2604 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:16.920388 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:16.926736 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.885073  2336 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:16.949435 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 965) took 1653 ms. Trace:
I20260709 13:06:16.950013 29909 rpcz_store.cc:276] 0709 13:06:15.296076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.365366 (+ 69290us) service_pool.cc:224] Handling call
0709 13:06:16.949194 (+1583828us) spinlock_profiling.cc:243] Waited 68.2 ms on lock 0x7b54006ffc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:16.949355 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156857088}
I20260709 13:06:16.964870 29904 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:06:16.973838 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:16.976249 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 913) took 1415 ms. Trace:
W20260709 13:06:16.980551  2587 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.563s	user 0.000s	sys 0.003s
I20260709 13:06:16.981460  2454 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:16.981882  2454 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:16.982095  2606 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:16.982463  2606 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:16.982816  2454 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.971442  2336 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:16.976814 29902 rpcz_store.cc:276] 0709 13:06:15.560873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.569211 (+  8338us) service_pool.cc:224] Handling call
0709 13:06:15.584375 (+ 15164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:15.748000 (+163625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.975814 (+1227814us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54006c02d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:16.976163 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365290752}
I20260709 13:06:16.999084 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:17.002803 29066 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:06:17.005210 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:17.005973 29070 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:17.007823 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:17.015123 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:17.018366 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:17.018801 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:17.019836 29069 raft_consensus.cc:2410] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:06:17.021063 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:17.021513 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:16.983258  2606 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.024355  2605 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.028679  2304 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:17.029217 29065 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:17.030068 29217 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:06:17.030447 29217 pending_rounds.cc:85] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 9
I20260709 13:06:17.030656 29217 pending_rounds.cc:107] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:06:17.030872 29217 raft_consensus.cc:2889] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:17.029134  2605 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.032161  2482 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.034902  2482 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:17.034796 29143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2592) took 2063 ms (client timeout 2101 ms). Trace:
W20260709 13:06:17.035620 29143 rpcz_store.cc:269] 0709 13:06:14.971590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.007097 (+ 35507us) service_pool.cc:224] Handling call
0709 13:06:17.034599 (+2027502us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b5400191c54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:17.034732 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756772608}
I20260709 13:06:17.036487 29068 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:17.037489 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:17.038335 29217 raft_consensus.cc:2410] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:06:17.041067  2605 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:17.043529 29218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 939) took 3222 ms (client timeout 2031 ms). Trace:
W20260709 13:06:17.044005 29218 rpcz_store.cc:269] 0709 13:06:13.821155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.015117 (+193962us) service_pool.cc:224] Handling call
0709 13:06:17.043290 (+3028173us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54003a98d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:17.043440 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3460670208}
I20260709 13:06:17.045728 29215 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:06:17.048493  2446 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:17.048905  2446 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.049275  2446 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:17.051115  2492 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:17.054636  2526 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:17.055158  2382 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:17.055595  2334 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:17.056087  2526 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:17.056529  2526 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:17.023587 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:17.026396 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.026921 29221 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:17.083138 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:17.089224 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:17.096153 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.100742 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:17.103423 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:17.119117 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:17.072050  2454 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:17.124475 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:17.072427  2084 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:17.128993 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:17.128973 29910 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 959) took 1845 ms (client timeout 1655 ms). Trace:
W20260709 13:06:17.137184 29910 rpcz_store.cc:269] 0709 13:06:15.283840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.283985 (+   145us) service_pool.cc:224] Handling call
0709 13:06:17.128758 (+1844773us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54003fdc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:17.128900 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":905362816}
I20260709 13:06:17.130609 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:17.140313 29064 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:06:17.089478  2606 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:17.153671 29221 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:17.153734 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 916) took 1539 ms. Trace:
I20260709 13:06:17.158568 29906 rpcz_store.cc:276] 0709 13:06:15.614300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.616809 (+  2509us) service_pool.cc:224] Handling call
0709 13:06:15.873942 (+257133us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:15.874313 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:15.874344 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:15.893443 (+ 19099us) log.cc:844] Serialized 26 byte log entry
0709 13:06:16.138029 (+244586us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:16.138090 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:16.138190 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:16.979778 (+841588us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:16.987158 (+  7380us) raft_consensus.cc:1685] finished
0709 13:06:16.987181 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.153550 (+166369us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54004e61d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:17.153674 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":487,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":13282,"spinlock_wait_cycles":597575296,"thread_start_us":208029,"threads_started":1,"wal-append.queue_time_us":477186}
I20260709 13:06:17.162108 29070 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:17.162941 29068 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:17.120787  2396 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:17.051816  2492 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.117843  2532 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:17.178488 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:17.181471 29219 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:17.182140 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:17.140395  2562 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:17.234112 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.154544  2604 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.151171  2482 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:17.123584  2468 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:06:17.156126  2471 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:17.298657 29213 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:06:17.150888  2059 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:17.311805 29066 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:17.118906  2567 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:06:17.150744  2613 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:17.284554 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:17.154036  2084 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.236333  2396 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.177325  2526 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.338431 29072 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:17.338399 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 418811us
W20260709 13:06:17.311594  2479 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.008s	sys 0.028s
I20260709 13:06:17.185292  2228 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.166445  2611 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.297777  2608 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:17.364459 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:17.366708 29066 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:17.175712  2412 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:17.377151 29064 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:06:17.338523  2609 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.298601  2471 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:17.355748  2479 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.009s	sys 0.029s
I20260709 13:06:17.341667  2478 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.289741  2604 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.356200  2313 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:17.386765 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.337046  2059 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.370000  2608 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:17.383855  2526 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:17.387687  2471 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:17.300824  2482 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.237365  2532 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.349277  2396 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.355829  2613 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.371417  2228 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.375976  2611 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.392827  2609 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.399930  2478 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.408890  2313 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:17.479696  2634 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 13:06:17.481818  2630 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:06:17.438035  2635 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.445437  2383 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:17.499500  2628 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
W20260709 13:06:17.506206  2627 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:06:17.484625  2492 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:17.499269  2629 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
W20260709 13:06:17.503291  2633 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:06:17.509330  2611 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:17.517835  2631 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:06:17.518307  2478 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:17.518843  2622 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 13:06:17.521106  2623 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:06:17.522555  2604 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.523408  2613 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:17.500425  2635 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.524348  2532 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:17.525883  2635 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.577925 29065 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:06:17.578322 29064 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:06:17.581453 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:17.581461 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.583235 29071 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:17.369199 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:17.586697 29073 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:17.587273 29071 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:17.587796 29071 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:17.598584 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 968) took 1903 ms. Trace:
I20260709 13:06:17.600993 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:17.602450 29143 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:17.602166 29071 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.604941 29071 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:17.612618 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.630097  2313 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.632892 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:17.634177 29148 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:17.636512  2083 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.637566  2083 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.638248  2482 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:17.640151 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.641605 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:17.642097  2441 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:17.643082  2441 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.644402  2442 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:17.645324  2442 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:17.650352 29219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 954) took 2187 ms (client timeout 1664 ms). Trace:
W20260709 13:06:17.651005 29219 rpcz_store.cc:269] 0709 13:06:15.462297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.462827 (+   530us) service_pool.cc:224] Handling call
0709 13:06:17.649704 (+2186877us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54002832d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:17.649981 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3834757504}
I20260709 13:06:17.655500  2482 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:17.656796  2482 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.657630 29147 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:17.604997 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:17.661115 29144 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:17.661221 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.662600 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:17.662848 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:17.663594 29145 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:17.663810 29142 raft_consensus.cc:2410] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:06:17.610116  2084 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.664171  2478 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.664734  2084 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:17.639827 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:17.613931  2396 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:17.668911 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.670120 29140 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:06:17.666705  2084 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.668589  2604 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:17.672848  2604 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:17.674233  2604 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.676841  2355 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:17.678642 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:17.679620 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.680949 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:17.693459 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:17.694430 29141 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:17.694558  2552 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:17.695103  2552 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:17.695503 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:17.684643 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:17.686018 29142 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:17.734575 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:17.734699 29145 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:17.736042  2608 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.737875 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:17.737705  2608 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:17.687263  2336 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:17.745110 29142 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
W20260709 13:06:17.753585 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.758426 29143 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:17.690541  2482 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.672632 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:17.696751  2552 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.668684 29139 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:17.612952  2228 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.665125  2478 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.702487  2492 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.607169  2059 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.599330 29903 rpcz_store.cc:276] 0709 13:06:15.694765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:15.703403 (+  8638us) service_pool.cc:224] Handling call
0709 13:06:15.703818 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:15.704265 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:15.704306 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:17.056222 (+1351916us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b2c00012928. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:06:17.056640 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:06:17.173592 (+116952us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:17.173636 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.173716 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:17.597650 (+423934us) raft_consensus.cc:1685] finished
0709 13:06:17.597692 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:17.598259 (+   567us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3854,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":3180,"spinlock_wait_cycles":13011712,"thread_start_us":116560,"threads_started":1,"wal-append.queue_time_us":285030}
I20260709 13:06:17.704479 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:17.790043 29140 raft_consensus.cc:2410] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:06:17.705487 29072 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:17.713063  2442 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:17.713639  2474 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:17.717935  2458 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.718305 29902 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:17.718570  2057 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.720438  2604 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:17.720996 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:17.722450 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:17.724746 29213 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:17.674711  2084 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.725832  2477 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:06:17.664935 29038 connection.cc:256] RPC call timeout handler was delayed by -1.1056807229997503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2081s for a response
W20260709 13:06:17.808394 29038 connection.cc:256] RPC call timeout handler was delayed by -1.0527776630005974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2827s for a response
I20260709 13:06:17.733196 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:17.784979 29071 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:17.799376 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:17.729980 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:17.823318 29874 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.012s	sys 0.012s
W20260709 13:06:17.823395 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 75565.3us
W20260709 13:06:17.824120 29874 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.013s	sys 0.012s
W20260709 13:06:17.825623 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 701190us
I20260709 13:06:17.826552  2609 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.828028 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:17.828428 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 131223us
I20260709 13:06:17.832350 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:17.832269 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:17.833395 29218 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:17.834687 29112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.003s	sys 0.012s
W20260709 13:06:17.835103 29112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.003s	sys 0.012s
W20260709 13:06:17.836071 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:17.836542 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:17.836973 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.837280  2601 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:17.837704  2059 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.838593  2059 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:17.839272 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:17.839948 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:17.840359 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.840479  2407 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:17.841362  2482 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.842027  2482 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.843638 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:17.844094  2458 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.845675  2396 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:17.840368 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:17.848325 29219 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:17.853022 29066 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:17.856402  2530 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.857304  2530 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.860985  2059 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:17.861923  2059 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.862754  2059 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.866470  2441 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.869223  2083 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:17.870091 29144 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
W20260709 13:06:17.870744  2446 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.821s	user 0.016s	sys 0.006s
I20260709 13:06:17.871268  2446 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.872699 29220 raft_consensus.cc:2368] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:06:17.875563 29904 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:17.876140  2057 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.823154  2569 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.877903  2383 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:17.878401  2569 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.878696  2383 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:17.880084  2396 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:17.881709  2471 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.881405  2304 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:06:17.807763  2477 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:17.885354 29148 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
W20260709 13:06:17.885429 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59108.3us
W20260709 13:06:17.887358 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 49249.1us
I20260709 13:06:17.899176 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:17.916013 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:17.914355 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:17.920396 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:17.933432  2569 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:17.923373  2454 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.731s	user 0.012s	sys 0.003s
I20260709 13:06:17.923797  2396 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:06:17.936878 29148 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:17.945034 29904 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:17.937554  2458 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:17.952310  2482 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.935945  2383 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:17.957428 29145 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
W20260709 13:06:17.967459 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 51262.8us
I20260709 13:06:17.979249 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:17.896557  2492 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:17.919342  2304 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:06:18.010571  2530 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:18.014086  2383 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:18.063799 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.064218 29148 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:18.018992  2478 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:17.938127  2313 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:18.021514 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51627.7us
I20260709 13:06:18.022856 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:17.946815  2477 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:18.029688 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:18.032027  2471 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:18.032573  2552 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:18.069842 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:18.034842  2441 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:18.035898  2059 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:18.040709  2446 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:18.075619 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 145778us
W20260709 13:06:18.076431 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 103865us
I20260709 13:06:18.075376 29038 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:18.070873 29213 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:18.070911  2313 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.084336  2383 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.081421 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:18.067198  2083 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:18.110222 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.099236 29141 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:18.120651 29142 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:06:18.121059 29142 pending_rounds.cc:85] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 10
I20260709 13:06:18.118486 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:18.122998 29142 pending_rounds.cc:107] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:06:18.123337 29142 raft_consensus.cc:2889] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:18.071324  2458 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:06:18.116878  2083 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:06:18.112665  2059 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:18.128237  2059 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:18.128697  2059 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:18.130242 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.136528 29213 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:18.136950  2441 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:18.125106 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:18.143527 29145 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 12: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:18.144903 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:18.145355  2604 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:06:18.146275 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 223169us
I20260709 13:06:18.146736 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:18.147504  2532 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:18.147857 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:18.148746 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 97695.5us
I20260709 13:06:18.121598  2383 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:18.148144  2313 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:18.152976 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.147506 29907 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:18.148059  2613 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:18.115226  2530 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.165468 29904 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:18.167644 29146 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:06:18.168213  2084 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:18.178386 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:18.173368 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 654576us
I20260709 13:06:18.183763 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:18.170282  2613 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:18.192386 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 111091us
W20260709 13:06:18.220329 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.218464 29909 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 12.
I20260709 13:06:18.178179  2084 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:18.193399 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:18.230722 29145 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:06:18.191778  2477 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:18.232251 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.232851 29907 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:18.238605 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:18.198184  2613 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:18.241902  2643 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:18.248111  2636 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:06:18.248847 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 81996.3us
I20260709 13:06:18.250772 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:18.198220  2057 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:06:18.252445 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:18.178437  2532 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:18.253760 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:18.225235  2530 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.238386  2313 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:18.314163  2471 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.314419 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:18.319433 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 63947.7us
I20260709 13:06:18.335816 29219 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:06:18.336100 29213 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:18.336680 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.309618  2383 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.310668  2636 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:18.318279  2604 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:18.344828 29217 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
W20260709 13:06:18.346359 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.336503 29218 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:18.349824 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:18.353001 29143 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.005s	sys 0.017s
I20260709 13:06:18.319531  2643 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:18.366263 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:18.367491 29143 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.005s	sys 0.017s
I20260709 13:06:18.365681 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:18.340086  2454 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.336164  2569 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.366989 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:18.383594 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 198780us
I20260709 13:06:18.387130 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:18.388640 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44362.9us
W20260709 13:06:18.391263  2646 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.152s	user 0.000s	sys 0.003s
I20260709 13:06:18.351275  2609 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:06:18.392601  2471 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.393973  2609 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:18.394447  2609 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:18.394894  2609 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:18.395318  2471 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:18.381290  2313 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:18.396615 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.398551 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:18.398835  2313 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:18.399544  2336 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:18.358795  2636 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:18.400465 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:18.411558 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:18.412115 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.403935 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:18.415980 29216 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:18.404066 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:18.417469 29214 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:18.400413 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:18.428349 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:18.401675  2532 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:18.396622  2645 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.375s	user 0.004s	sys 0.003s
I20260709 13:06:18.395494  2059 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.360175  2383 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:18.446321 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 204945us
I20260709 13:06:18.400604  2336 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.403764  2441 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:18.403415  2530 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:18.454321 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.457491 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2824) took 1370 ms. Trace:
I20260709 13:06:18.458107  2336 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.457937 29215 rpcz_store.cc:276] 0709 13:06:17.060399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:17.061305 (+   906us) service_pool.cc:224] Handling call
0709 13:06:17.061892 (+   587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:17.089859 (+ 27967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:17.089895 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:17.090472 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:06:17.645343 (+554871us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b1800474a60. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:06:17.927474 (+282131us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:17.927519 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:17.927628 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:18.408500 (+480872us) raft_consensus.cc:1685] finished
0709 13:06:18.408520 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:18.430996 (+ 22476us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27469,"raft.queue_time_us":27746,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":7571,"spinlock_wait_cycles":69612800,"thread_start_us":258332,"threads_started":1,"wal-append.queue_time_us":1025055}
I20260709 13:06:18.460995 29910 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:06:18.461297 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:18.462117  2608 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:18.463037 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:18.463443  2608 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.464535  2608 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:18.457950 29214 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:06:18.466599 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:18.467108  2608 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:18.468747 29218 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:06:18.469218  2059 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:18.469514  2478 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.469949  2059 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:18.470428  2059 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:18.470889  2059 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:18.471412  2313 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:18.469249 29218 pending_rounds.cc:85] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 13
I20260709 13:06:18.472880 29218 pending_rounds.cc:107] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:06:18.473201 29218 raft_consensus.cc:2889] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:18.470263  2478 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:18.476356 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 131710us
I20260709 13:06:18.476730  2478 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:18.477830 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:18.480413  2552 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:18.483050  2643 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:18.486805 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59189.2us
I20260709 13:06:18.399853  2084 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.406313  2569 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:18.498447 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:18.479580 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:18.500068 29213 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:06:18.500473 29213 pending_rounds.cc:85] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 12
W20260709 13:06:18.500519 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48391.9us
I20260709 13:06:18.500849 29213 pending_rounds.cc:107] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:06:18.501122 29213 raft_consensus.cc:2889] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:18.406078  2454 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.428200  2609 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.512477  2441 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:18.513420  2441 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:06:18.514243  2651 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:06:18.514386 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:18.515748 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 52839.3us
I20260709 13:06:18.516938 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:18.518072 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:18.522022  2609 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:18.522771  2383 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:18.523361  2482 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:18.524217  2482 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:06:18.524309 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 124008us
I20260709 13:06:18.525890 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:18.525765  2482 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:18.482287  2635 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:18.528286 29221 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
W20260709 13:06:18.533385 29188 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.591s (SENT)
I20260709 13:06:18.535693  2313 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:18.568470 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:18.507217  2530 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.052s
W20260709 13:06:18.576061 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64513.2us
I20260709 13:06:18.576529  2635 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:18.576954  2084 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:18.577557  2313 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:18.594972 29188 leader_election.cc:336] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.611s (SENT)
I20260709 13:06:18.598204 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:18.596026  2635 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:18.600940  2083 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:18.607774  2454 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:18.608615  2477 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:18.572494  2471 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:06:18.598613  2313 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:18.617111 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:18.601382  2552 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:18.616665  2635 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:18.628732  2083 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:18.658773 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55644.9us
I20260709 13:06:18.660588  2613 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:18.661089  2662 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:06:18.662036  2532 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:18.662968  2613 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:18.605116  2383 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:06:18.626127  2454 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:18.632012  2552 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:18.682917 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 161439us
I20260709 13:06:18.632043  2057 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:06:18.700708 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 186408us
W20260709 13:06:18.681934 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 339182us
I20260709 13:06:18.670662  2552 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:18.718107 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:18.682715  2569 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:18.734138 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:18.734751 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.683703  2478 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:18.729400  2636 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:18.740330  2663 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:06:18.741693  2636 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:18.742648 29038 leader_election.cc:400] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:06:18.743070 29038 leader_election.cc:403] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:06:18.743902 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:18.745990 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 43695.7us
W20260709 13:06:18.748306  2654 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.344s	user 0.003s	sys 0.000s
W20260709 13:06:18.741740  2659 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.351s	user 0.004s	sys 0.000s
I20260709 13:06:18.750816  2675 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:18.751336  2675 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:18.753208  2604 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:18.753734  2604 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:18.756040 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.726017  2532 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:06:18.765611  2532 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:18.767023 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:18.768504  2665 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:06:18.768821 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:18.751700  2675 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:18.770292 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:18.771905 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:18.772608 29905 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:18.774714 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:18.710234  2454 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:18.776332 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:18.777063  2454 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:18.728839  2492 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:18.777782  2643 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:18.778609  2643 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:18.768309 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:18.780315 29910 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:18.780568  2471 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:06:18.782974 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 51657.5us
I20260709 13:06:18.789657 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:18.790488 29908 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:18.701424  2613 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:06:18.795491 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:18.797310  2477 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:18.738428  2609 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:18.807726 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 280233us
I20260709 13:06:18.805784 29186 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:18.816545  2383 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:18.805249  2569 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:18.834501 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 48328.7us
I20260709 13:06:18.774741  2059 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.807305  2482 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:18.859455 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 111467us
I20260709 13:06:18.864398  2668 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:18.865089  2668 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:18.865602  2668 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:18.868083 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:18.860230  2477 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:18.872473  2477 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:18.872925  2477 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:18.876509  2336 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:18.877684  2482 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.809041  2675 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.881705  2675 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:18.877264  2552 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.871255 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:18.888096 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:18.933552 29140 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:18.833025  2569 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:18.936739 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 46182.7us
W20260709 13:06:18.942348 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:18.945256 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.852131  2609 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:18.839288  2383 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:18.953863  2671 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:06:18.955179 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.961855 29908 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
W20260709 13:06:18.965780 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 145760us
W20260709 13:06:18.967451 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 223626us
I20260709 13:06:18.966292 29140 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:06:18.968629 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:18.950344  2492 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:18.967352  2530 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:18.956377  2482 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:19.017594 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:18.940488  2635 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:19.025585 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 60597.9us
I20260709 13:06:19.029546 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:18.990351  2492 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.033787 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:19.039350 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 151347us
I20260709 13:06:19.046257 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:19.053293 29071 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:19.019157  2530 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.051753 29140 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:06:19.060848 29185 leader_election.cc:336] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 2.010s (SENT)
I20260709 13:06:19.065114 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:19.086378 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:19.065325 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:19.066237 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:19.088714 29221 raft_consensus.cc:2410] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:06:19.070135  2635 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:19.072160  2608 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:19.072444 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.075403 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.077054 29065 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:19.078135 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.080467 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:19.121872 29068 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:19.036405  2482 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:19.085592  2530 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:19.036693  2609 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:19.126350 29039 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.532s (SENT)
I20260709 13:06:19.047649  2492 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:19.060730  2673 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 13:06:19.104861  2626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.006s	sys 0.020s
I20260709 13:06:19.122609 29072 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:19.122942 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 125076us
I20260709 13:06:19.123973 29073 raft_consensus.cc:2410] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 13:06:19.126907 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.126194 29070 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:19.129748 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.130121  2552 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:19.130522  2679 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:19.139346 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:19.140044 29069 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:19.120121  2478 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.140293 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:19.141489 29142 raft_consensus.cc:2368] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16.
I20260709 13:06:19.141214  2478 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:19.119685  2608 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:19.137230 29064 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:19.144711 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.138077 29067 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:19.145452 29065 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:19.145612 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.131325 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:19.138324 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 90780.8us
I20260709 13:06:19.150828 29140 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:19.147025 29064 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
W20260709 13:06:19.131455  2680 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:06:19.158041 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:19.152792 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:19.166400 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:19.136552  2626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.007s	sys 0.024s
I20260709 13:06:19.157358 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.170046 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.175071 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.177399 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:19.179683 29218 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:19.179145 29065 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:19.132992  2643 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:19.138119  2683 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.192832 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:19.185530 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:19.133872  2482 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:19.145289  2608 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:19.212077 29139 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:19.212924  2608 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:19.217721 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:19.218801 29215 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:19.232477  2478 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:19.205588 29069 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:19.208408 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.236486 29064 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:19.236868 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.188541  2059 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:19.237715  2686 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:06:19.219712 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.220556 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:19.225167 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 103695us
I20260709 13:06:19.227555 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:19.236555  2569 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:19.237622 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.240679 29064 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:19.209578 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.219149 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:19.241335 29141 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:19.237756 29069 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:19.242014 29214 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:19.128768  2477 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:19.207348 29073 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:19.249214 29072 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
W20260709 13:06:19.239487 29039 leader_election.cc:336] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.551s (SENT)
I20260709 13:06:19.259217 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.259943 29072 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:19.263770 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.205381  2635 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.273551 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.273557 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:19.274974 29064 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:19.276947 29069 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:06:19.272251 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:19.277679 29221 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
W20260709 13:06:19.279214  2084 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.682s	user 0.002s	sys 0.011s
I20260709 13:06:19.283576  2471 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:19.285421 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.299679 29065 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:19.285768 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:19.287006 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:19.287566 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:19.303596 29071 raft_consensus.cc:2376] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20. Re-sending same reply.
I20260709 13:06:19.278834  2478 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:19.300870  2672 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.333s	user 0.000s	sys 0.002s
W20260709 13:06:19.312700  2692 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:06:19.206518  2683 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:19.313972  2313 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.002s
W20260709 13:06:19.336257 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:19.314217  2694 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:06:19.311596 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.300717  2471 consensus_queue.cc:1243] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:06:19.308820  2688 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:06:19.315045 29072 raft_consensus.cc:2376] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14. Re-sending same reply.
W20260709 13:06:19.371218 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 131769us
W20260709 13:06:19.346280 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 179635us
W20260709 13:06:19.360025 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 132575us
I20260709 13:06:19.284036  2482 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:19.310599  2676 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:19.276052  2685 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:06:19.286561  2635 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:19.394901 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 115852us
I20260709 13:06:19.390450  2676 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:19.391398 29039 leader_election.cc:336] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.573s (SENT)
I20260709 13:06:19.388034 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:19.397719 29073 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:19.306020  2083 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:06:19.387854  2532 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.622s	user 0.004s	sys 0.006s
I20260709 13:06:19.400372 29871 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:19.405365 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:19.405982 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.359644  2313 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:19.399643  2676 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:19.342168  2478 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:19.408075  2532 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:06:19.445631 29073 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:06:19.447156 29066 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:19.448115 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 968) took 1028 ms. Trace:
I20260709 13:06:19.452729  2689 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:06:19.453115 29216 rpcz_store.cc:276] 0709 13:06:18.419589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.421593 (+  2004us) service_pool.cc:224] Handling call
0709 13:06:18.444163 (+ 22570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:18.454753 (+ 10590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:18.454785 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:18.464262 (+  9477us) log.cc:844] Serialized 26 byte log entry
0709 13:06:18.670431 (+206169us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:18.670466 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:18.670531 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:19.447818 (+777287us) raft_consensus.cc:1685] finished
0709 13:06:19.447852 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:19.448037 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1239,"raft.queue_time_us":5665,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":205735,"threads_started":1,"wal-append.queue_time_us":387225}
I20260709 13:06:19.440244  2679 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:19.471493 29214 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
W20260709 13:06:19.476647 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 104508us
I20260709 13:06:19.445308  2383 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:19.489423 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 101626us
W20260709 13:06:19.485304 29039 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.852s (SENT) [suppressed 58 similar messages]
I20260709 13:06:19.454970  2601 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:06:19.491096 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 969) took 1071 ms. Trace:
W20260709 13:06:19.476890  2604 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.723s	user 0.014s	sys 0.001s
I20260709 13:06:19.465161  2471 raft_consensus.cc:2764] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:06:19.507565 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:19.495026 29219 rpcz_store.cc:276] 0709 13:06:18.419976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:18.420123 (+   147us) service_pool.cc:224] Handling call
0709 13:06:18.433342 (+ 13219us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:18.451216 (+ 17874us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:18.864408 (+413192us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b1800403840. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f783a5053c7 00007f7836b179a7 00007f7836b270bf 00007f7836b27038
0709 13:06:18.864697 (+   289us) log.cc:844] Serialized 14 byte log entry
0709 13:06:19.061644 (+196947us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:19.115613 (+ 53969us) log.cc:844] Serialized 26 byte log entry
0709 13:06:19.118352 (+  2739us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:19.118387 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:19.118452 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:19.478847 (+360395us) raft_consensus.cc:1685] finished
0709 13:06:19.490823 (+ 11976us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:19.491017 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19138,"prepare.queue_time_us":426355,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":96312,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":3861,"spinlock_wait_cycles":5325568,"thread_start_us":185707,"threads_started":2,"wal-append.queue_time_us":269243}
W20260709 13:06:19.506079  2674 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.059s	user 0.004s	sys 0.005s
W20260709 13:06:19.559738 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.609603  2441 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:19.616271  2707 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:06:19.504815  2604 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:19.692826 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 652469us
W20260709 13:06:19.690487 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.691970 29218 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:19.499135  2492 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:19.695022 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 209736us
I20260709 13:06:19.610411  2477 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:19.698330  2613 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:19.707835  2613 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:19.708761  2683 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:19.709052  2613 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:19.710517  2613 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:19.719079 29039 leader_election.cc:336] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.569s (SENT)
W20260709 13:06:19.725167 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 220578us
W20260709 13:06:19.730077 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:19.743227 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 192273us
W20260709 13:06:19.745777 29068 consensus_meta.cc:287] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.613s	user 0.010s	sys 0.000s
I20260709 13:06:19.746201 29068 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:19.748461 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:19.761355 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:19.762351  2441 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:19.765300  2441 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:19.765926  2441 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:19.774153 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55215.3us
W20260709 13:06:19.775169 29039 leader_election.cc:336] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.700s (SENT)
I20260709 13:06:19.782238 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:19.784664 29068 raft_consensus.cc:2376] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20. Re-sending same reply.
I20260709 13:06:19.775799 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:19.789165 29875 leader_election.cc:400] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16.
I20260709 13:06:19.789490 29875 leader_election.cc:403] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:06:19.790571  2478 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:19.791671  2635 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:19.774698  2626 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.803826 29070 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:19.807703 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:19.779933  2313 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:19.808885 29068 raft_consensus.cc:2376] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18. Re-sending same reply.
I20260709 13:06:19.815034 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:19.815443  2715 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:19.815095  2706 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.344s	user 0.001s	sys 0.000s
I20260709 13:06:19.816301  2715 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:19.817965  2690 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:19.819012  2704 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.349s	user 0.001s	sys 0.000s
I20260709 13:06:19.767530  2441 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:19.822623 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.809689  2304 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:19.823427 29068 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
W20260709 13:06:19.819854  2708 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
W20260709 13:06:19.819669  2710 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 13:06:19.820029  2717 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:06:19.828455 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:19.833271 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 72075.3us
I20260709 13:06:19.833345 29070 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:19.828181 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:19.835232 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.837291 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:19.840340 29070 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:19.844549 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:19.845289  2687 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:19.845739  2687 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.846125  2687 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:19.846534 29905 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:19.846897 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:19.847878 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:19.850042 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:19.850660 29905 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:19.850737 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:19.851985 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:19.852802 29909 raft_consensus.cc:2410] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:06:19.853704 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:19.854341 29907 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:19.854672  2723 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:06:19.861047 29143 pending_rounds.cc:85] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 8
I20260709 13:06:19.861653 29143 pending_rounds.cc:107] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:06:19.862020 29143 raft_consensus.cc:2889] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:19.864856 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 249106us
I20260709 13:06:19.865083  2697 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:19.866106 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:19.867003 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:19.867169  2697 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:19.868117  2446 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:19.868341 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.868537  2446 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.868971  2446 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:19.869136 29142 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:19.870146  2383 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:19.870565  2383 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.870888  2383 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:19.871248  2608 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.657s	user 0.010s	sys 0.003s
I20260709 13:06:19.874146 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.875032 29140 raft_consensus.cc:2410] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:06:19.876268 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:19.876971  2683 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.880132  2336 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:19.880209  2683 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:19.881649 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:19.877542 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 102423us
I20260709 13:06:19.882772 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:19.882958 29039 leader_election.cc:336] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.658s (SENT)
I20260709 13:06:19.883409 29142 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
W20260709 13:06:19.884240 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.884883  2604 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:19.884271  2477 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:19.886447 29039 leader_election.cc:336] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.741s (SENT)
W20260709 13:06:19.887399 29039 leader_election.cc:336] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.783s (SENT)
I20260709 13:06:19.877887  2684 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:19.878544  2703 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.868562 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:19.878544 29188 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.101s (SENT)
I20260709 13:06:19.880630  2336 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:19.826975  2715 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:19.827157  2492 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:19.808259  2626 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:19.858776 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:19.877637 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.888827 29144 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:19.895483 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.900367 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.897276 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:19.943608 29142 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 13:06:19.900331 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.896974 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.944837 29145 raft_consensus.cc:2410] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:19.902632 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.939275  2626 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:19.946985 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.947736 29144 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
W20260709 13:06:19.887900 29039 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.801s (SENT)
I20260709 13:06:19.954748 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:19.897670 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.897329 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:19.903048 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:19.891119 29188 leader_election.cc:336] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.568s (SENT)
I20260709 13:06:19.890971 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:19.957557 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:19.961530 29140 raft_consensus.cc:2368] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
W20260709 13:06:19.963900 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 72795.8us
W20260709 13:06:19.964773 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 73864.6us
I20260709 13:06:19.970541 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:19.965703 29907 raft_consensus.cc:2410] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 13:06:19.970556 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:19.972481 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:19.972792 29147 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:19.965091 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 67847.4us
I20260709 13:06:19.978866 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:19.905627  2668 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.039s	user 0.002s	sys 0.012s
I20260709 13:06:19.906661  2718 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:19.984015 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:19.990192 29144 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:19.988711 29140 raft_consensus.cc:1217] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 13.10->[14.11-14.11]   Dedup: 14.11->[]
I20260709 13:06:19.984732 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:19.899791  2690 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:19.986810 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:19.997496 29213 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 12.
I20260709 13:06:20.001482 29907 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:20.001922 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:19.903508  2383 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:20.005811 29910 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:19.994586 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 594376us
I20260709 13:06:19.995303 29140 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:06:20.006079 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:20.006145 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:19.904366  2604 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:19.982337  2626 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:19.984107  2492 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:20.014055 29144 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:20.014501 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:20.007881 29185 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:20.007328 29908 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
W20260709 13:06:20.010613 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 122724us
I20260709 13:06:20.011004 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:20.016101 29905 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:20.013274 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:19.889112  2336 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:20.014263  2492 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:19.999040  2690 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:20.022665 29065 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.031s	sys 0.052s
I20260709 13:06:19.906746  2596 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 13:06:20.024313 29220 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:06:20.036139 29220 pending_rounds.cc:85] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 9
I20260709 13:06:20.036451 29220 pending_rounds.cc:107] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:06:20.036712 29220 raft_consensus.cc:2889] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:20.018615 29039 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.081s (SENT)
I20260709 13:06:20.037457 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:20.025122 29141 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:20.030299 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:20.002803  2608 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:20.038682 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:20.025309 29148 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:20.034871 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:20.005450  2057 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:20.031872 29065 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.031s	sys 0.052s
I20260709 13:06:20.040346 29140 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:20.014919  2697 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:19.888571  2703 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:20.042833 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:20.043114 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.044870 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.048125 29144 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:20.014263  2613 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:20.050029 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:19.996599  2668 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:20.034178  2492 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:20.030814  2626 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:20.053690 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:20.037053  2690 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:20.031838  2458 raft_consensus.cc:2764] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:06:20.068746 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:20.018927  2604 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:20.078209 29188 leader_election.cc:336] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.876s (SENT)
I20260709 13:06:20.050539  2697 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.090112 29188 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:20.055605  2530 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.879s	user 0.009s	sys 0.004s
I20260709 13:06:20.090389 29144 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:20.092859 29067 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:20.054689  2716 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:20.053656  2608 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:20.093999 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:20.094853 28969 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 241) took 24854 ms (client timeout 15000 ms). Trace:
W20260709 13:06:20.099323 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 80757.1us
I20260709 13:06:20.100148 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:20.104971 29219 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:20.112047 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.113365 29147 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:06:20.114336 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.114620  2691 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:20.115149 29146 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:20.115229  2691 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:20.094059  2697 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:20.116096 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.125308 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.097757  2477 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:20.132560 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.135283 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:20.115885  2691 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:20.143438 29220 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:20.165699 29146 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:20.171103 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:20.171878 29146 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:20.166363 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.130375 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.117274  2304 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16.
W20260709 13:06:20.174978 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 96833.9us
W20260709 13:06:20.192284 29188 leader_election.cc:336] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.903s (SENT)
I20260709 13:06:20.115294  2703 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:20.176867  2715 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:20.100099  2530 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:20.128655  2059 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:20.192580 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:20.133975  2702 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:20.211711 29141 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:20.217411 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:20.212145 29039 leader_election.cc:336] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.547s (SENT)
I20260709 13:06:20.219328 29039 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:20.170428  2482 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:20.220849 29144 raft_consensus.cc:2410] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:06:20.225180 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:20.221382 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:20.225235 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.226760 29220 raft_consensus.cc:2368] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:06:20.102823  2716 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:20.223673  2608 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:20.222431 29906 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
W20260709 13:06:20.228644 29188 leader_election.cc:336] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.047s (SENT)
I20260709 13:06:20.231453 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:20.234095 29222 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:20.235128 29213 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:20.237082 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:20.240990 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.206775  2715 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:20.246212 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:20.246277 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:20.247320 29217 raft_consensus.cc:2368] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:20.190128  2601 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:06:20.250504 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:20.251442 29220 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:20.251601 29039 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.548s (SENT)
I20260709 13:06:20.252151 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:20.250684 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:20.253715 29144 raft_consensus.cc:2376] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16. Re-sending same reply.
W20260709 13:06:20.257958 29188 leader_election.cc:336] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.117s (SENT)
I20260709 13:06:20.258607 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:20.267148 29905 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:20.268676 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.270232 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:20.241320  2709 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:20.276618 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:20.280618 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 221472us
I20260709 13:06:20.243687  2715 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:20.282361 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 211033us
W20260709 13:06:20.270094  2729 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
W20260709 13:06:20.289117 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 254313us
I20260709 13:06:20.292109 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:20.281775  2726 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:06:20.294977 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:20.276024  2703 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:20.296363 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:20.281474  2684 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:20.241791  2482 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:20.227789  2626 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:20.334312  2530 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:20.335121  2530 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:20.335166 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:20.336341  2057 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:20.336347 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:20.337070  2057 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:20.337790 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:20.338008  2057 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:20.338805  2702 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:20.339331  2702 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:20.098433 28969 rpcz_store.cc:269] 0709 13:05:55.082074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.087097 (+  5023us) service_pool.cc:224] Handling call
0709 13:06:19.906017 (+24818920us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:09.461575 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:09.506394 (+ 44819us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:09.506457 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:06:09.507027 (+   570us) tablet.cc:662] Decoding operations
0709 13:06:09.950490 (+443463us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:09.974453 (+ 23963us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:09.979265 (+  4812us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:06:10.280000 (+300735us) tablet.cc:701] Row locks acquired
0709 13:06:10.280049 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:06:10.280246 (+   197us) write_op.cc:270] Start()
0709 13:06:10.280394 (+   148us) write_op.cc:276] Timestamp: P: 1783602370280171 usec, L: 0
0709 13:06:10.280436 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:06:10.281152 (+   716us) log.cc:844] Serialized 9096 byte log entry
0709 13:06:10.958215 (+677063us) op_driver.cc:464] REPLICATION: finished
0709 13:06:11.173546 (+215331us) write_op.cc:301] APPLY: starting
0709 13:06:11.173647 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:11.885904 (+712257us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:11.897242 (+ 11338us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:12.174777 (+277535us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:12.175407 (+   630us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:06:12.175476 (+    69us) write_op.cc:312] APPLY: finished
0709 13:06:12.288460 (+112984us) log.cc:844] Serialized 204 byte log entry
0709 13:06:12.577965 (+289505us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:12.604608 (+ 26643us) write_op.cc:454] Released schema lock
0709 13:06:12.627438 (+ 22830us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:12.653353 (+ 25915us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:12.653393 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":496647,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":210473,"apply.run_cpu_time_us":73413,"apply.run_wall_time_us":1479769,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100267,"num_ops":23,"prepare.queue_time_us":536408,"prepare.run_cpu_time_us":63670,"prepare.run_wall_time_us":1100830,"raft.queue_time_us":255722,"raft.run_cpu_time_us":22749,"raft.run_wall_time_us":130622,"replication_time_us":677590,"thread_start_us":1061913,"threads_started":5,"wal-append.queue_time_us":586853,"wal-append.run_cpu_time_us":98488,"wal-append.run_wall_time_us":2581526}]]}
I20260709 13:06:20.340214  2702 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:20.342995  2696 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:20.343466  2696 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:20.330026 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:20.344570 29217 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:06:20.345155 29215 raft_consensus.cc:2410] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:06:20.320499 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:20.286734  2383 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:20.331667 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:20.349007 29144 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:20.335656  2530 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:20.352093 29146 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:20.286666  2458 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:06:20.357788 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.359632 29142 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:20.362905 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.363902 29146 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:20.364234 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.365033 29142 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 13:06:20.385938 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:20.354084  2721 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.497s	user 0.004s	sys 0.000s
I20260709 13:06:20.388530 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.388820 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.394467 29215 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:20.394797 29146 raft_consensus.cc:2368] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:06:20.348903  2668 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:20.395567 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.386690 29144 raft_consensus.cc:2376] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13. Re-sending same reply.
W20260709 13:06:20.397365 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59662.5us
I20260709 13:06:20.397449  2454 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:20.412906 29185 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:20.387351  2676 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.931s	user 0.008s	sys 0.005s
I20260709 13:06:20.355362  2530 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:20.356474  2604 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:20.442242 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.447863 29909 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 12.
I20260709 13:06:20.453783 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.457073 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:20.458668  2761 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:20.459504  2761 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:20.460515  2738 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:06:20.460724  2761 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:20.461917  2736 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 13:06:20.459632  2731 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
W20260709 13:06:20.480089 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 128892us
I20260709 13:06:20.484680 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:20.467927  2383 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:20.492234 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:20.500140 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:20.572049 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 251483us
I20260709 13:06:20.573519 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.574417 29144 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:20.579720 29186 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:20.580278 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:20.581382 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:20.582353 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 671867us
I20260709 13:06:20.585391 29875 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:20.591678 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:20.596604 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.597462 29144 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:20.598861 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 186163us
I20260709 13:06:20.456060  2676 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:20.602914 29215 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:20.505625  2383 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.602100 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:20.572492  2668 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:20.487392  2746 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.109s
I20260709 13:06:20.616580 29144 pending_rounds.cc:85] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 10
I20260709 13:06:20.621441 29144 pending_rounds.cc:107] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:06:20.621726 29144 raft_consensus.cc:2889] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:20.586427  2684 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:20.625294 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:20.607975 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 107837us
I20260709 13:06:20.596076  2626 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.629588  2626 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:20.628489 29148 raft_consensus.cc:2368] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 17. Current term is 18.
I20260709 13:06:20.636031 29038 leader_election.cc:400] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:06:20.636089 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.636488 29038 leader_election.cc:403] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:20.638218 29188 leader_election.cc:336] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.784s (SENT)
I20260709 13:06:20.604780  2713 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:20.638976 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:20.640413 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 155802us
I20260709 13:06:20.615078  2383 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:20.641355 29871 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:20.642980 29141 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:20.647018 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.647956 29139 raft_consensus.cc:2410] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:06:20.615756  2668 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:20.619179  2716 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:20.642719  2733 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
I20260709 13:06:20.656481 29038 leader_election.cc:400] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:20.659116 29038 leader_election.cc:403] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:20.661111 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2626) took 1424 ms. Trace:
I20260709 13:06:20.661245 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:20.661692 29072 rpcz_store.cc:276] 0709 13:06:19.230753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.381137 (+150384us) service_pool.cc:224] Handling call
0709 13:06:19.403687 (+ 22550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:19.444835 (+ 41148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:19.444864 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:19.462325 (+ 17461us) log.cc:844] Serialized 26 byte log entry
0709 13:06:19.770231 (+307906us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:19.778995 (+  8764us) log.cc:844] Serialized 14 byte log entry
0709 13:06:19.875815 (+ 96820us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b18001e54a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f783a9e158f 00007f7833a20571 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd
0709 13:06:19.876741 (+   926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:19.876816 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.221088 (+344272us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:20.655324 (+434236us) raft_consensus.cc:1685] finished
0709 13:06:20.655344 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.655481 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98389,"prepare.queue_time_us":5767,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":135,"raft.queue_time_us":133333,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":19174,"spinlock_wait_cycles":57206528,"thread_start_us":264386,"threads_started":1,"wal-append.queue_time_us":359839}
I20260709 13:06:20.647281  2636 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:06:20.660220  2613 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.597s	user 0.008s	sys 0.004s
I20260709 13:06:20.657418  2668 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:20.653513  2626 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:20.675686 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 73758.1us
I20260709 13:06:20.654863  2713 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:06:20.679186 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 99829.8us
I20260709 13:06:20.680820 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:20.668037  2609 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:20.673019  2482 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:20.671218  2636 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.659152  2757 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:20.695531 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:20.690186 29139 raft_consensus.cc:2376] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15. Re-sending same reply.
I20260709 13:06:20.680132  2613 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:20.707665 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:20.710776  2446 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.841s	user 0.013s	sys 0.000s
I20260709 13:06:20.710633  2683 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:20.711324  2446 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:20.712355  2768 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:20.713371  2336 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.673s	user 0.008s	sys 0.003s
I20260709 13:06:20.713615  2446 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:20.713795  2336 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:20.711830  2683 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:20.690860  2762 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:20.721396 29145 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:20.699276  2761 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:20.720728 29220 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:20.695775  2458 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:20.700050  2636 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:20.700636  2757 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:20.704489  2482 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:20.679320  2756 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:06:20.726356 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.708029  2746 raft_consensus.cc:2764] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:06:20.736336 29139 raft_consensus.cc:2410] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:06:20.748451 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.765592 29145 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:20.768963 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.785575 29216 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
W20260709 13:06:20.790289 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 94846.8us
W20260709 13:06:20.752605  2752 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:06:20.732277  2757 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:20.797689 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 156434us
I20260709 13:06:20.799095 29871 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:20.800074 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 106403us
W20260709 13:06:20.802019 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 176803us
I20260709 13:06:20.804765 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:20.801561 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 149774us
I20260709 13:06:20.805315 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:20.792049  2753 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 13:06:20.793062  2691 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.002s
I20260709 13:06:20.814378 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:20.814786 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:20.816404 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.817129 29145 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:20.821573 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:20.791728  2458 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:20.793165  2761 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:20.822175 29216 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
W20260709 13:06:20.803177  2735 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.342s	user 0.000s	sys 0.001s
I20260709 13:06:20.801733  2676 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:20.824139 29072 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
W20260709 13:06:20.824911 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 186718us
W20260709 13:06:20.827433 29188 leader_election.cc:336] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.218s (SENT)
I20260709 13:06:20.825464  2765 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:20.827885 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:20.820524  2746 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:20.836253 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:20.823309  2441 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:20.854492 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.855425 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.856221 29146 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:20.844797 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:20.846414 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:20.846848 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:20.848495  2783 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:06:20.850596  2771 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:20.851701 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:20.836184  2697 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.003s
I20260709 13:06:20.811523  2482 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:06:20.832026  2765 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:20.878069  2765 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:20.880957  2761 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:20.842542  2336 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:20.858985  2754 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.182s	user 0.000s	sys 0.002s
I20260709 13:06:20.876477  2697 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:20.887255 29214 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:20.888154  2749 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:06:20.890053  2783 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:20.890484  2783 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:20.876614 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.892025 29145 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:20.892658  2781 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:20.892941 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.893543  2772 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:20.894239 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:20.895190 29145 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:20.888231  2697 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:20.845719  2613 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:20.897897 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 835) took 1414 ms. Trace:
I20260709 13:06:20.870105 29139 raft_consensus.cc:2410] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:20.882983  2668 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:20.895884 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.821404  2691 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:20.900825 29144 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:20.904763 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:20.903155 29069 rpcz_store.cc:276] 0709 13:06:19.483094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.842111 (+359017us) service_pool.cc:224] Handling call
0709 13:06:19.842427 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:19.900686 (+ 58259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:19.900720 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:19.901283 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.171429 (+270146us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:20.171471 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.171669 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:20.897690 (+726021us) raft_consensus.cc:1685] finished
0709 13:06:20.897710 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:20.897839 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57864,"raft.queue_time_us":66634,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":23037,"thread_start_us":269753,"threads_started":1,"wal-append.queue_time_us":354124}
I20260709 13:06:20.864719  2782 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:20.868112  2609 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:20.921146 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:20.894307  2772 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:20.859653  2676 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 13:06:20.930817 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:20.931569 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.933138 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:20.940582 29145 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:20.919165 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:20.936599 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:20.916961  2745 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.258s	user 0.001s	sys 0.001s
I20260709 13:06:20.952390 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:20.953559 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2618) took 1727 ms (client timeout 1791 ms). Trace:
I20260709 13:06:20.902835  2613 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:20.923585  2750 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:06:20.954397 29219 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:20.941414 29144 raft_consensus.cc:2376] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14. Re-sending same reply.
W20260709 13:06:20.972505 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 145104us
I20260709 13:06:20.915771  2746 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:20.971189 29067 rpcz_store.cc:269] 0709 13:06:19.202538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.209286 (+  6748us) service_pool.cc:224] Handling call
0709 13:06:20.929677 (+1720391us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b5400186854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:20.929835 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957303680}
I20260709 13:06:20.913961  2744 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:20.925040  2687 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 1.078s	user 0.014s	sys 0.000s
W20260709 13:06:20.998703 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 91511.3us
I20260709 13:06:20.974300  2613 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:21.006677 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 183502us
I20260709 13:06:20.971647  2718 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:06:21.021230 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.022698 29147 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:20.668213  2684 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 13. Current term is 14.
I20260709 13:06:20.954322  2458 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.002533  2687 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.050693  2687 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:21.052529  2779 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 13:06:21.059363  2530 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.579s	user 0.001s	sys 0.008s
I20260709 13:06:20.466816  2696 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.062575  2696 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:21.062907  2492 consensus_meta.cc:287] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.958s	user 0.013s	sys 0.000s
I20260709 13:06:21.067854  2383 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.068881  2772 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.070277  2383 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:21.076635  2760 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:21.077211  2760 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:21.078125  2760 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.079524  2760 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:21.063606  2492 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.081802 29213 raft_consensus.cc:2368] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
W20260709 13:06:21.083036  2734 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.694s	user 0.001s	sys 0.002s
I20260709 13:06:21.084250  2492 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
W20260709 13:06:21.084817 29188 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.160s (SENT)
W20260709 13:06:21.086511  2715 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.792s	user 0.013s	sys 0.000s
W20260709 13:06:21.089337 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:21.089808 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.087136  2715 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.003155  2746 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.094331 29910 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:21.094981  2746 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:21.096379 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.097008 29071 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:21.059520  2636 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.099454 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 977) took 1255 ms. Trace:
I20260709 13:06:21.099852 29906 rpcz_store.cc:276] 0709 13:06:19.843778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.846642 (+  2864us) service_pool.cc:224] Handling call
0709 13:06:21.099207 (+1252565us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b5400023d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:21.099321 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862156160}
I20260709 13:06:21.101253 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.101956 29069 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:21.102599  2715 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:21.106963 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.104281 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.108690 29064 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:21.105829 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.110468 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.111855 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:21.112063  2636 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:21.113116  2083 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:21.113335 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.113626  2083 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:21.114094 29069 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:21.105329 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:21.115607 29216 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:21.117209 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2624) took 1887 ms. Trace:
I20260709 13:06:21.117606 29065 rpcz_store.cc:276] 0709 13:06:19.230167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.309120 (+ 78953us) service_pool.cc:224] Handling call
0709 13:06:19.337630 (+ 28510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:19.365891 (+ 28261us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:06:20.041451 (+675560us) log.cc:844] Serialized 14 byte log entry
0709 13:06:20.286231 (+244780us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:20.295769 (+  9538us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.295916 (+   147us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:20.304631 (+  8715us) log.cc:844] Serialized 14 byte log entry
0709 13:06:20.328210 (+ 23579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.328282 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:21.116998 (+788716us) raft_consensus.cc:1685] finished
0709 13:06:21.117017 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:21.117147 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23222,"prepare.queue_time_us":390361,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":1146,"raft.queue_time_us":62074,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":14133,"thread_start_us":835643,"threads_started":2,"wal-append.queue_time_us":292392}
I20260709 13:06:21.002980  2744 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.121701  2744 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:20.793758  2713 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.123075  2608 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:20.763068  2762 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:20.891393  2783 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.123787  2608 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.128162 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:21.130059  2690 consensus_meta.cc:287] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.033s	user 0.009s	sys 0.004s
I20260709 13:06:21.130137  2608 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:21.130574  2690 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.124502 29218 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
W20260709 13:06:21.036139  2766 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:06:21.118758 29037 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:21.122560  2636 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.126030  2775 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.125579  2441 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.131728 29142 raft_consensus.cc:2368] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16.
I20260709 13:06:21.132815  2775 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.133203  2441 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.138176  2441 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:21.139307  2683 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:06:21.139460 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 867565us
I20260709 13:06:21.135782  2775 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:21.144376 29039 leader_election.cc:336] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.229s (SENT)
I20260709 13:06:21.114552  2083 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.146440 29216 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:21.155285 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.156625 29068 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:21.102811 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.157842 29141 raft_consensus.cc:2410] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:06:21.156374 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 143456us
W20260709 13:06:21.158679 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 72751.6us
I20260709 13:06:21.161293 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:21.158381 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:21.092952  2668 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:21.170967 29039 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.236s (SENT)
I20260709 13:06:21.174662 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.175228 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:21.179212 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.180445 29068 raft_consensus.cc:2410] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:06:21.181717 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.182374 29066 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:21.183799 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:21.184345 29065 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:21.185878 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.112077  2604 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:21.125725  2626 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:21.194360 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.125674  2762 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:21.189477 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:21.171154  2709 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.888s	user 0.007s	sys 0.006s
I20260709 13:06:21.200323  2709 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:21.192800  2604 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:21.202239  2604 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.208133  2709 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:21.209160  2709 consensus_queue.cc:1243] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 12.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:06:21.213258 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 944) took 1353 ms. Trace:
I20260709 13:06:21.216802 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2747) took 1198 ms. Trace:
I20260709 13:06:21.217831 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.227214 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:21.229712  2730 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.959s	user 0.002s	sys 0.009s
I20260709 13:06:21.233074 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.059900  2530 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.234747 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.213855 29143 rpcz_store.cc:276] 0709 13:06:19.860012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.860181 (+   169us) service_pool.cc:224] Handling call
0709 13:06:19.860695 (+   514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:19.862771 (+  2076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:19.862809 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:19.885129 (+ 22320us) log.cc:844] Serialized 52 byte log entry
0709 13:06:20.164435 (+279306us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:20.211934 (+ 47499us) log.cc:844] Serialized 14 byte log entry
0709 13:06:20.212256 (+   322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.212367 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:21.082419 (+870052us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:21.212824 (+130405us) raft_consensus.cc:1685] finished
0709 13:06:21.212945 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:21.213147 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5047,"prepare.queue_time_us":47923,"raft.queue_time_us":9771,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":3668,"spinlock_wait_cycles":85712000,"thread_start_us":278790,"threads_started":1,"wal-append.queue_time_us":433794}
I20260709 13:06:21.219509 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:21.237903 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:21.253445 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.244722 29142 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:21.254245 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 227530us
W20260709 13:06:21.223789 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.260572 29139 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:21.217383 29140 rpcz_store.cc:276] 0709 13:06:20.018122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.024291 (+  6169us) service_pool.cc:224] Handling call
0709 13:06:20.040020 (+ 15729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:20.040956 (+   936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:21.216579 (+1175623us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b5400192150. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:21.216735 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35092352}
I20260709 13:06:21.262815 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.263789 29140 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:21.235872 29146 raft_consensus.cc:2410] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
W20260709 13:06:21.266076 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 113286us
I20260709 13:06:21.269538 29146 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:21.132790  2636 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.209863  2709 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:21.195495  2083 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:21.275559 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:21.276299 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:21.277136 29902 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:06:21.279978 29909 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:21.170234  2782 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 13:06:21.247987  2604 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:21.236755  2668 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:21.309285 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.309957 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.310287 29071 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:21.310593 29064 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:21.285091 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.273135 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.286384 29067 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:21.287751 29904 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:21.288568 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:21.289647 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.292687 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:21.313990 29139 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 14: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:21.304738  2530 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:21.305361  2757 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.283941  2709 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:21.267566  2626 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:21.311937 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52110.6us
I20260709 13:06:21.315841 29142 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:21.334193 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.337719 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.338608 29145 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:21.334266 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.340799 29142 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
W20260709 13:06:21.348699 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 223629us
I20260709 13:06:21.354032 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.355197 29071 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:21.355948 29904 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 13.
I20260709 13:06:21.356904 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.360790 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.366353 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.367345 29140 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:21.369129 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.369918 29140 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:21.374971 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.375778 29065 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:21.384791 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.385532 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.385842 29140 raft_consensus.cc:2376] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14. Re-sending same reply.
I20260709 13:06:21.307770  2765 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.389412 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.387151 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.393883 29143 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:21.347644  2441 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.394951 29140 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
W20260709 13:06:21.392130 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 233822us
I20260709 13:06:21.398387 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.395305 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.401234 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:21.401566 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.405207 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.406214 29140 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:21.410157 29068 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:21.404100 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.390370 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:21.402246 29146 raft_consensus.cc:2376] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19. Re-sending same reply.
I20260709 13:06:21.413563 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.382853  2775 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.411657 29067 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:21.416339 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.422273 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.425038 29139 raft_consensus.cc:2376] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14. Re-sending same reply.
I20260709 13:06:21.380553  2687 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:21.423170 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.426246 29146 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:21.432124  2482 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:21.371217  2530 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:06:21.433619 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:21.436492 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.436342 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.450160 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.481253 29069 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:06:21.435707 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 45383.5us
W20260709 13:06:21.471729 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 134708us
I20260709 13:06:21.436231 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.476162 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:21.411561  2601 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:21.469852 29145 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:21.482798 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:21.484477 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:21.481336 29067 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:21.481698 29146 raft_consensus.cc:2376] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20. Re-sending same reply.
I20260709 13:06:21.426100  2668 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:21.501349  2304 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:21.411878  2441 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.469208  2676 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:21.380155  2709 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.509963 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.512702 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.437911  2608 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.518508 29146 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:21.492776  2806 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:21.434226  2775 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.525424 29064 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:21.472721  2782 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:06:21.531199 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.532138 29067 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:21.436712  2687 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.534463 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:21.538658  2626 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:21.538868  2797 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:06:21.542543  2765 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:21.502681  2809 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:21.545406 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2749) took 1323 ms. Trace:
I20260709 13:06:21.545828 29214 rpcz_store.cc:276] 0709 13:06:20.221742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.221904 (+   162us) service_pool.cc:224] Handling call
0709 13:06:21.545162 (+1323258us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b54003cd2d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:21.545304 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1492584704}
I20260709 13:06:21.551512 29065 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
W20260709 13:06:21.552507 29188 leader_election.cc:336] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.396s (SENT)
I20260709 13:06:21.523166  2806 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.567710 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.568409 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:21.573410 29067 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:21.570791 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.563787 29140 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
W20260709 13:06:21.576828 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48913.4us
I20260709 13:06:21.577832 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:21.498119  2601 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.582141 29146 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:21.580721 29905 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:21.585970 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.586966 29146 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:21.588869 29905 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:21.594998 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:21.560714  2804 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:21.567831  2808 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:21.628990  2441 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:21.526647  2608 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.584291 29069 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:21.631855  2687 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:21.602566  2613 consensus_meta.cc:287] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.001s
I20260709 13:06:21.632884  2613 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.633808  2608 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:21.634734 29071 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:21.635051  2613 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:21.592212  2482 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:21.566546  2636 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:21.582073  2691 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:21.637543 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.640264 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:21.640751 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.641461 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.642055 29145 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:21.642829 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.643579 29145 raft_consensus.cc:2410] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:06:21.644253 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.644968 29139 raft_consensus.cc:2368] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16.
I20260709 13:06:21.645686 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.646188 29139 raft_consensus.cc:2376] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16. Re-sending same reply.
I20260709 13:06:21.646988 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.647526 29139 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:21.592151  2668 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.574431  2806 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:21.653183  2454 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:21.653834 29140 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:21.654937 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.655534 29140 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:21.656409 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.656911 29140 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:21.658003  2601 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:21.585135  2757 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:21.598467  2530 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:21.659337  2530 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:21.611469 29148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2749) took 1568 ms (client timeout 1801 ms). Trace:
I20260709 13:06:21.609997  2775 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:21.660162 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.666054 29142 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:21.667309 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.668432 29146 raft_consensus.cc:2368] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13.
I20260709 13:06:21.669371 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.669991 29146 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
W20260709 13:06:21.614459  2737 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 1.265s	user 0.008s	sys 0.003s
I20260709 13:06:21.670653 29140 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:21.653784  2454 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:21.664839 29148 rpcz_store.cc:269] 0709 13:06:20.042687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.042835 (+   148us) service_pool.cc:224] Handling call
0709 13:06:21.611198 (+1568363us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b5400041854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:21.611356 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1331873024}
I20260709 13:06:21.677603 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:21.680203 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.680792 29140 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:21.682524 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:21.670816 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.560173  2800 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:06:21.688827  2530 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:21.679847  2808 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:21.679210  2636 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:06:21.681900  2608 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:06:21.735149 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:21.734700 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:21.693637 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.741722 29908 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
W20260709 13:06:21.726684 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.742043 29148 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:21.685936  2441 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:21.744212 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:21.693029 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:21.702611 29188 leader_election.cc:336] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.514s (SENT)
W20260709 13:06:21.748260 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45708us
I20260709 13:06:21.727833 29141 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:21.749368 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.734514 29142 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:21.750545 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.686515  2482 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:21.684528  2714 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.754402 29910 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:21.756192 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.758060 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.758648 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.761265 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:21.761557 29142 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:21.762521  2636 consensus_queue.cc:1243] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:06:21.763091 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.768918 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.770012 29069 raft_consensus.cc:2410] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:06:21.772399 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.764454 29140 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:21.722359  2775 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:06:21.719444  2816 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:06:21.792359 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.773020 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:21.762425  2441 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:21.801275 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.802285 29067 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:06:21.801728  2441 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.804366  2747 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:21.776451 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.777331 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.774675  2813 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:21.778551  2822 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:21.778834 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.774312 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.804551  2304 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:21.808594 29142 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:21.796396  2668 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:21.809387  2668 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:06:21.789583 29188 leader_election.cc:336] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.718s (SENT)
I20260709 13:06:21.808411  2747 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:21.812269  2747 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.813674  2747 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:21.816084 29146 raft_consensus.cc:2376] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14. Re-sending same reply.
W20260709 13:06:21.817237  2732 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.541s	user 0.003s	sys 0.009s
I20260709 13:06:21.817905 29140 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:21.810678  2668 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:21.820333 29148 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:21.820551  2668 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:21.821724 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.822422 29148 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:21.823457 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:21.824221  2813 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.763723  2714 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.824651  2813 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:21.828737 29188 leader_election.cc:400] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16.
I20260709 13:06:21.829137 29188 leader_election.cc:403] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:06:21.829763 29141 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:06:21.830415 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:21.831285 29141 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:21.832633  2530 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:21.817202 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:21.836473 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:21.837162 29146 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:21.838673 29148 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:06:21.846711 29871 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:21.787533  2454 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.849818 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.846444 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:21.853951 29067 raft_consensus.cc:2376] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16. Re-sending same reply.
I20260709 13:06:21.851238 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:21.865026 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 92121.4us
I20260709 13:06:21.866220 29875 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:21.868831 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:21.870800 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:21.869838 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:21.871498 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.872159 29067 raft_consensus.cc:2376] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15. Re-sending same reply.
I20260709 13:06:21.872395 29140 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:21.872488 29142 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:21.875957 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.880312 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.880908 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:21.848119  2690 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:21.852520  2336 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:21.852494  2477 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:21.884557 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:21.886328 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:21.887240 29065 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:21.869024  2796 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:21.892273 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.893540 29064 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:21.897598 29146 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
W20260709 13:06:21.902385 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 307451us
I20260709 13:06:21.882050  2755 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:21.877679  2691 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:21.877547  2747 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:21.881428  2765 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:21.929541 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.906630 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:21.912140 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.888648  2816 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:21.883431  2759 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:21.859987  2806 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.916177  2796 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:21.932163 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:21.932503 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50451.1us
I20260709 13:06:21.884286  2810 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:21.889279  2477 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:21.934644 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:21.941583 29066 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:21.905264  2690 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:21.954521 29185 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.791s (SENT)
I20260709 13:06:21.920472  2691 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:21.960719 29065 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:21.956486 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:21.928905  2747 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:21.959745 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:21.958702 29068 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:21.963459 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.934453  2775 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:21.961179 29039 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:21.939051  2336 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:21.967963 29188 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.077s (SENT)
I20260709 13:06:21.967720 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:21.941964  2477 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:21.996238 29066 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:06:21.939754  2765 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:21.960454  2820 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:21.999578 29065 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:21.946616  2816 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:21.942901  2714 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:22.007890 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:21.968822  2747 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:21.999071 29064 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:21.965801  2759 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:22.018544 29068 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:22.020437 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:22.040759 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 79688us
W20260709 13:06:22.060532 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.061088 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:22.005916  2482 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:22.041345  2820 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:22.044929  2759 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:22.048237 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 109985us
I20260709 13:06:22.112085 29139 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:06:21.997691  2458 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:22.048290  2758 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.583s	user 0.006s	sys 0.000s
I20260709 13:06:22.063830 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:22.120252 29066 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:06:22.120858 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 834) took 2637 ms. Trace:
I20260709 13:06:22.121232 29070 rpcz_store.cc:276] 0709 13:06:19.482944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.841262 (+358318us) service_pool.cc:224] Handling call
0709 13:06:19.841688 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:19.910115 (+ 68427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:19.910145 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:19.962372 (+ 52227us) log.cc:844] Serialized 26 byte log entry
0709 13:06:20.103978 (+141606us) spinlock_profiling.cc:243] Waited 9.87 ms on lock 0x7b480027c258. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783ba36aa1 00007f783ba2db37 00007f783ba2d9ca 00007f783bb2526f 00007f783bb24f37 00007f783bb3955c 00007f783bb39508 00007f783bb39488 00007f783bb39443 00007f783bb386ec 00007f783aa62a96
0709 13:06:20.410176 (+306198us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:20.410287 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:20.410469 (+   182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:22.043527 (+1633058us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b2400404c78. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78338a69c4 00007f78338a4acb 00007f7833a5fdc9 00007f7833a5fa55 00007f7833a5f09d 00007f783669110e 00007f78366a888c 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221
0709 13:06:22.120606 (+ 77079us) raft_consensus.cc:1685] finished
0709 13:06:22.120631 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:22.120772 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67946,"raft.queue_time_us":68088,"raft.run_cpu_time_us":2932,"raft.run_wall_time_us":205883,"spinlock_wait_cycles":79980288,"thread_start_us":404592,"threads_started":1,"wal-append.queue_time_us":459992}
I20260709 13:06:22.123323 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:22.124066 29070 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:22.107673 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:22.106694 29065 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:22.111883 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.127032 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.005868  2626 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:22.112496 29139 pending_rounds.cc:85] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 14
W20260709 13:06:22.114566 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 171907us
I20260709 13:06:22.128568 29139 pending_rounds.cc:107] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:06:22.128887 29139 raft_consensus.cc:2889] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:22.115382  2454 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:22.125599 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:22.131042  2441 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:22.130513  2690 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:22.132331 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.133631  2626 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:22.135571 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.137080  2626 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:22.131073 29070 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:22.144802  2770 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:22.145323  2770 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:22.145458  2713 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:22.145713  2812 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:22.146095  2713 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.146237  2812 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.147073 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:22.148406  2713 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:22.149281 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:22.113581  2714 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:22.011286  2336 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:22.152281 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 91379.6us
I20260709 13:06:22.157348 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:22.158753  2765 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:22.159375 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.159734  2757 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.003s
W20260709 13:06:22.164335  2829 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:06:22.151876 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:22.170545 29148 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:22.171450  2477 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:22.174135  2477 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:22.174743 29065 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:22.175441 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:22.175472 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.176904  2604 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:22.177886  2757 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:22.178295  2604 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:22.178925  2712 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:22.179030  2806 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:22.179354  2712 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:22.179814  2712 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:22.066346  2796 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.180346 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.180490  2668 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:22.182533  2668 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:22.182937  2668 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:22.180917  2792 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.828s	user 0.002s	sys 0.001s
W20260709 13:06:22.184230 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.186103  2446 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:22.186668  2446 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:22.181932 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:22.180931 29066 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:22.146261  2770 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.190474 29064 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:22.189082 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:22.195093 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.197800 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.199683 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.205876 29070 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:22.209852 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:22.210872 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:22.210647 29065 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:22.217784 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:22.184545  2691 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:22.109170  2813 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.230507 29070 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:22.194638  2814 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:22.238618 29220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2755) took 1983 ms (client timeout 1696 ms). Trace:
I20260709 13:06:22.240846 29221 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:22.243355 29064 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:22.236089 29066 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
W20260709 13:06:22.239035 29220 rpcz_store.cc:269] 0709 13:06:20.255000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.255154 (+   154us) service_pool.cc:224] Handling call
0709 13:06:22.238384 (+1983230us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b540016c954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:22.238548 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1015499648}
I20260709 13:06:22.207429  2810 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:22.211438  2441 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:22.252941  2441 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.253394  2441 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:22.207969  2809 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:22.207687  2604 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:22.224247  2816 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.260360 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:22.261438  2691 raft_consensus.cc:2764] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:06:22.216130  2770 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:22.261866 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.264688  2530 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16.
I20260709 13:06:22.265465  2530 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:22.212689  2636 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:22.252218 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55844.5us
I20260709 13:06:22.211277  2755 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:22.269342  2755 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.270004 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:22.270615  2813 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:22.271343  2813 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.271590  2755 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:22.274354 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:22.280288 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:22.275368 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:22.228628  2806 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:22.279552 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:22.230407  2458 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:22.278550 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 101023us
I20260709 13:06:22.282284 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2743) took 2179 ms. Trace:
I20260709 13:06:22.283574 29147 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:22.232558  2812 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:22.293954 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:22.291842 29222 rpcz_store.cc:276] 0709 13:06:20.098237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.098399 (+   162us) service_pool.cc:224] Handling call
0709 13:06:20.117573 (+ 19174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:20.238023 (+120450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.277202 (+2039179us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54003a9dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:22.277368 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267330816}
I20260709 13:06:22.302278 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:22.304280 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:22.306140 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:22.271835  2813 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.313705 29185 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:22.303859 29071 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:22.321241 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:22.323005 29073 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:22.283818  2059 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:22.301225  2691 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:22.325582  2691 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.325940  2691 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:22.312178  2684 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:22.329052  2477 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:22.311533  2755 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:06:22.329661  2477 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:22.330574  2477 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:22.332402  2813 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:22.297219  2683 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:22.355998 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 71031.3us
I20260709 13:06:22.361361 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:22.365003 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:22.366118 29141 raft_consensus.cc:2410] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:06:22.351462  2770 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:22.303241  2530 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:06:22.301725  2747 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:22.350462  2613 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:22.297696  2458 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.272051  2820 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:22.351046  2059 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:22.449137 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:22.449977 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:22.450635 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:22.370450  2770 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:22.454095 29220 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:06:22.452127 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:22.360193  2684 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:06:22.459753 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:22.457772 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 100131us
I20260709 13:06:22.460546 29213 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:22.460485 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:22.458969 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 249146us
W20260709 13:06:22.462616 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 149818us
I20260709 13:06:22.481417 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:22.461978 29147 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:22.462446 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:22.449168  2755 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:22.487529 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.529264 29218 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:22.530943  2814 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:22.531436  2814 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:22.527330 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:22.534247 29871 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.476300 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:22.468497 29219 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:22.460490  2446 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:22.442047  2683 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:22.540130 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:22.540896 29213 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:22.507073 29073 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:22.448097  2530 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:22.529367  2836 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:06:22.461472 29216 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:22.556685 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:22.557560 29216 raft_consensus.cc:2410] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:06:22.559108  2059 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:22.561118  2845 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:22.561952  2845 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:06:22.566164 29071 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:22.566323 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:22.567646  2458 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:22.569180  2691 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:22.572538 29214 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:06:22.574365  2691 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:22.456640  2770 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:22.579855 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.474220  2806 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.462605  2336 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:22.536732  2846 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:22.539899 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:22.540882  2844 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:22.466542 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:22.545593 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.547150 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:22.546159 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:22.549739 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:22.695605 29213 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:22.554586  2684 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:22.537765 29222 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 13: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:22.487310  2712 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:22.707026 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 233811us
I20260709 13:06:22.712726 29068 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:22.535658  2755 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.588124  2782 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.732961 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:22.735975 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:22.743002 29067 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:22.747109 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 52995.4us
I20260709 13:06:22.601411  2747 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:22.748955 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:22.750852 29219 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:22.760988 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.580577  2530 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:22.706856  2336 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:22.753154 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:22.755944  2838 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.282s	user 0.001s	sys 0.001s
I20260709 13:06:22.714883  2846 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:22.781298 29067 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:22.647140  2806 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:22.784440 29067 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:22.752349  2683 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:22.782958 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:22.769114  2809 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:22.789026 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:22.796177  2846 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:22.798645  2765 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:22.799398  2765 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:22.800557  2765 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:22.809901 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:22.811743 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 987) took 2165 ms. Trace:
I20260709 13:06:22.812201 29221 rpcz_store.cc:276] 0709 13:06:20.646632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.646776 (+   144us) service_pool.cc:224] Handling call
0709 13:06:22.240668 (+1593892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.241255 (+   587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.811535 (+570280us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b540016c950. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:22.811681 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3634224896}
I20260709 13:06:22.813398 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:22.814004 29221 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:22.817855 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:22.818833 29218 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:22.821230  2083 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:22.821837  2083 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:22.822897  2083 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.824518  2454 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:06:22.825402 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 363434us
I20260709 13:06:22.826689  2843 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:22.827953 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 106274us
I20260709 13:06:22.828120  2794 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:22.830704 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:22.832551 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.833010 29220 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:22.728607  2684 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:22.819849 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:22.821399 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:22.840394 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 249678us
W20260709 13:06:22.889585 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 57107.1us
I20260709 13:06:22.857904 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:22.859064 29217 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:22.890340 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:22.884378 29039 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.741s (SENT)
I20260709 13:06:22.891146 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:22.794040  2336 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:22.757604  2782 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:22.947572 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 57794.2us
I20260709 13:06:22.953795 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:22.853003  2804 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:23.003029 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 112758us
I20260709 13:06:22.854637  2626 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:22.786623  2702 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 13:06:22.852869  2454 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:22.894574  2083 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:22.794533  2683 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:23.013234 29219 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:22.943159  2845 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:23.008754  2850 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:22.884393  2822 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.127s
I20260709 13:06:23.022279  2609 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:23.027778 29218 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:06:23.008937  2336 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:23.054481 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.013772  2794 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:06:23.119442 29216 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.006s	sys 0.015s
W20260709 13:06:23.016016  2668 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.833s	user 0.012s	sys 0.000s
I20260709 13:06:22.856227  2809 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:23.142632 29904 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: [18.14-20.15]
W20260709 13:06:23.145951 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 315273us
W20260709 13:06:23.097309 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:23.125139 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 271279us
I20260709 13:06:22.944432  2458 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:06:23.150976 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:23.014732  2698 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:23.017834  2845 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:22.888051  2843 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:23.130169 29216 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.006s	sys 0.016s
I20260709 13:06:23.028473  2712 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:23.209546 29218 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
W20260709 13:06:23.246033 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.171361  2636 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:23.146111  2822 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:23.189175  2845 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:23.148108  2794 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:23.025187  2812 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.188s
I20260709 13:06:23.279971 29221 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:23.281464 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:23.012339  2804 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:23.240470  2458 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.172734  2668 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:23.275326  2755 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:23.269767  2796 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.186831  2853 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.121s
W20260709 13:06:23.330961 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.269889  2822 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:23.283152  2794 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:23.274782  2676 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:06:23.285869  2840 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:23.332785 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 181942us
I20260709 13:06:23.291410  2530 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:23.402810 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 545006us
W20260709 13:06:23.335764 29213 consensus_meta.cc:287] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.000s
I20260709 13:06:23.285535  2847 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:23.353005  2855 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:06:23.360702  2852 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:06:23.270435  2781 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:06:23.420279 29221 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:23.394081 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:23.421618 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:23.407063 29213 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
W20260709 13:06:23.415333 29114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.009s	sys 0.018s
I20260709 13:06:23.306483  2636 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:23.422771 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:23.309562  2477 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.300401  2336 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:23.437191 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:23.438134 29221 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:23.300381  2458 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:23.393765 29070 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:23.395561  2683 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:23.448477 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:23.394603 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 246716us
W20260709 13:06:23.534476 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 112931us
I20260709 13:06:23.535576 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:23.538667  2676 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:23.539196  2676 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:23.540091  2676 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:23.549664 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:23.371279  2796 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:23.437623  2477 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:23.551064  2822 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:23.533627  2689 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:23.556375  2783 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:23.590991  2689 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:23.592267 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:23.592934 29147 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:23.595521  2690 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:23.557046  2862 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:06:23.426582  2847 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:23.558220  2804 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:23.559789 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 999) took 1508 ms. Trace:
I20260709 13:06:23.563077  2843 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:23.497542  2782 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:23.609313 29904 rpcz_store.cc:276] 0709 13:06:22.051146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.051300 (+   154us) service_pool.cc:224] Handling call
0709 13:06:22.051651 (+   351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:23.143235 (+1091584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.559496 (+416261us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54004a5bd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:23.559719 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2508325248}
I20260709 13:06:23.564356  2668 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:23.566238  2441 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.312s	user 0.010s	sys 0.002s
I20260709 13:06:23.421949  2840 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:23.493026  2806 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:23.572525  2809 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:23.573509  2872 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:23.576198 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2666) took 1031 ms. Trace:
I20260709 13:06:23.579468 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2667) took 1034 ms. Trace:
I20260709 13:06:23.497087  2684 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:23.556433  2718 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.590366  2871 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:23.545228  2608 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:23.608109 29147 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:23.614347 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 999) took 1109 ms. Trace:
I20260709 13:06:23.671072 29216 rpcz_store.cc:276] 0709 13:06:22.504767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.559799 (+ 55032us) service_pool.cc:224] Handling call
0709 13:06:22.574909 (+ 15110us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.575095 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:06:23.193131 (+618036us) log.cc:844] Serialized 14 byte log entry
0709 13:06:23.613986 (+420855us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:23.614038 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:23.614081 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.614142 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.614261 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":731787,"prepare.run_cpu_time_us":2700,"prepare.run_wall_time_us":28669,"raft.queue_time_us":2223,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3872,"thread_start_us":1037775,"threads_started":2}
I20260709 13:06:23.613356  2690 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:23.655405 29065 rpcz_store.cc:276] 0709 13:06:22.544683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.546224 (+  1541us) service_pool.cc:224] Handling call
0709 13:06:22.546514 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.546738 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.546763 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.578445 (+ 31682us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.882217 (+303772us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:22.882257 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.882327 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:23.575928 (+693601us) raft_consensus.cc:1685] finished
0709 13:06:23.575958 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.576125 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":146433,"threads_started":1,"wal-append.queue_time_us":333628}
I20260709 13:06:23.657505 29072 rpcz_store.cc:276] 0709 13:06:22.544963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.547495 (+  2532us) service_pool.cc:224] Handling call
0709 13:06:22.560315 (+ 12820us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.560709 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:06:22.560753 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.561229 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:06:22.853053 (+291824us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:06:22.853103 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.853174 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:23.579226 (+726052us) raft_consensus.cc:1685] finished
0709 13:06:23.579254 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.579396 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":709,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":94,"spinlock_wait_cycles":203136,"thread_start_us":291319,"threads_started":1,"wal-append.queue_time_us":559991}
I20260709 13:06:23.591903  2689 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:23.620750  2709 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:23.622545  2770 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.975s	user 0.012s	sys 0.000s
W20260709 13:06:23.657612 29068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.010s	sys 0.005s
W20260709 13:06:23.614871 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:23.764729 29068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.011s	sys 0.005s
W20260709 13:06:23.658311  2814 consensus_meta.cc:287] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.123s	user 0.012s	sys 0.001s
I20260709 13:06:23.658393  2872 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:23.780716  2872 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:23.617940  2843 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:23.774902 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2674) took 1053 ms. Trace:
I20260709 13:06:23.833043 29065 rpcz_store.cc:276] 0709 13:06:22.721146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.773744 (+1052598us) service_pool.cc:224] Handling call
0709 13:06:23.774191 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.774443 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:23.774482 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:23.774538 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:23.774587 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.774657 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.774822 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:23.845043 29039 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.004s	sys 0.012s
I20260709 13:06:23.668923  2608 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:23.845320 29186 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.009s	sys 0.004s
W20260709 13:06:23.848016 29188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.022s	sys 0.071s
W20260709 13:06:23.866941 29039 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.004s	sys 0.013s
W20260709 13:06:23.867377 29188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.025s	sys 0.071s
W20260709 13:06:23.868393 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 983961us
W20260709 13:06:23.867202 29186 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.009s	sys 0.005s
W20260709 13:06:23.845271 29875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.003s	sys 0.008s
W20260709 13:06:23.871621 29875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.004s	sys 0.008s
I20260709 13:06:23.874164 29904 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:23.844430 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 863) took 1134 ms. Trace:
I20260709 13:06:23.893692 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:23.621887  2441 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:23.880400 29068 rpcz_store.cc:276] 0709 13:06:22.710341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.713713 (+  3372us) service_pool.cc:224] Handling call
0709 13:06:22.714014 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.714243 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:22.983213 (+268970us) log.cc:844] Serialized 14 byte log entry
0709 13:06:23.843896 (+860683us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:23.843992 (+    96us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:23.844059 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.844155 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.844331 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89721,"prepare.queue_time_us":426549,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":64,"raft.queue_time_us":86532,"thread_start_us":1001004,"threads_started":2}
I20260709 13:06:23.902338 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:23.775563  2713 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:23.955935 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:23.622836  2845 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:23.776515  2059 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:23.941002 29067 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.000s	sys 0.024s
W20260709 13:06:23.969125 29067 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.000s	sys 0.024s
I20260709 13:06:23.967718  2636 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:23.967514  2613 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:23.947772  2719 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:23.955374  2304 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:06:23.969627 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:23.968288 29038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.011s	sys 0.026s
I20260709 13:06:23.970548  2804 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:24.000036 29038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.012s	sys 0.026s
I20260709 13:06:23.961433  2684 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.012903 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.013831 29216 raft_consensus.cc:2410] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:06:24.014074 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.014878 29214 raft_consensus.cc:2410] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:06:23.976115 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2672) took 1264 ms. Trace:
I20260709 13:06:24.016136 29067 rpcz_store.cc:276] 0709 13:06:22.711613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.847033 (+135420us) service_pool.cc:224] Handling call
0709 13:06:22.915285 (+ 68252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.915536 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:23.230669 (+315133us) log.cc:844] Serialized 14 byte log entry
0709 13:06:23.238256 (+  7587us) log.cc:844] Serialized 14 byte log entry
0709 13:06:23.975714 (+737458us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:23.975774 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:23.975825 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.975894 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.976052 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40665,"prepare.queue_time_us":373682,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":363976,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":3834,"spinlock_wait_cycles":109507200,"thread_start_us":959709,"threads_started":2}
W20260709 13:06:23.949968  2083 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.935s	user 0.006s	sys 0.006s
I20260709 13:06:23.775943  2770 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:23.924744  2782 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:23.894438  2786 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:06:23.996624 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 461169us
I20260709 13:06:23.849880  2814 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:23.678382  2718 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:23.980877 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 861) took 1271 ms. Trace:
I20260709 13:06:24.010926 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:24.072710 29905 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.005s	sys 0.014s
I20260709 13:06:23.965070  2847 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:24.074889 29114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.019s	sys 0.024s
W20260709 13:06:24.083837 29114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.019s	sys 0.025s
I20260709 13:06:23.969854  2867 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:24.077497 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.020s	sys 0.025s
W20260709 13:06:24.095026 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.020s	sys 0.025s
I20260709 13:06:23.980244  2809 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.077790 29072 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:24.079977 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:24.080773 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:24.083719 29220 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
W20260709 13:06:24.084378 29905 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.005s	sys 0.014s
I20260709 13:06:24.082000 29071 rpcz_store.cc:276] 0709 13:06:22.709759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.712003 (+  2244us) service_pool.cc:224] Handling call
0709 13:06:22.753249 (+ 41246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.753518 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:06:23.215462 (+461944us) log.cc:844] Serialized 14 byte log entry
0709 13:06:23.980514 (+765052us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:23.980581 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:23.980628 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.980699 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:23.980820 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9341,"prepare.queue_time_us":527820,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"raft.queue_time_us":68213,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":1195703,"threads_started":3,"wal-append.queue_time_us":560813}
W20260709 13:06:24.095446 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 598294us
I20260709 13:06:24.098646 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:24.084882 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 72618.4us
W20260709 13:06:24.086825 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 66135.6us
I20260709 13:06:24.101926 29067 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:24.101302 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:24.088160 29039 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.180s (SENT)
I20260709 13:06:24.102838 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:23.891258  2843 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.109203 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:23.961110  2713 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.105777 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:24.108222 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:24.110826 29139 raft_consensus.cc:2410] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:06:24.111255 29148 raft_consensus.cc:2368] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:06:24.111512 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:23.997521  2613 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.079499  2786 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:24.112640 29072 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:06:23.987012  2059 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.128484 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:24.086426  2816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.007s	sys 0.014s
I20260709 13:06:24.112623 29146 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:24.134102 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:24.159562 29141 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:24.104236  2458 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:24.140467  2863 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.522s	user 0.000s	sys 0.000s
W20260709 13:06:24.141384  2874 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.247s	user 0.000s	sys 0.000s
W20260709 13:06:24.144150 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 274124us
W20260709 13:06:24.177737 29188 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.731s (SENT)
W20260709 13:06:24.144544  2873 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
I20260709 13:06:24.145217  2885 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:24.080410  2608 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.074481  2684 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.146447  2796 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.073283  2804 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.105397  2809 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.106338  2806 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.147802  2786 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.148933  2840 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.088846  2782 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.246417  2684 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.246803  2843 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.247126  2684 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:24.247566  2782 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:24.152793  2879 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:24.248494  2879 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:24.249485  2879 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.153191  2814 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.153364  2770 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:24.106339  2690 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:24.154495 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 224151us
I20260709 13:06:24.155889 29144 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:24.253842 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:24.254524 29144 raft_consensus.cc:2376] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18. Re-sending same reply.
I20260709 13:06:24.255388 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:24.109999 29145 raft_consensus.cc:2368] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16.
I20260709 13:06:24.129060 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:24.256662 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:24.256875 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:24.117340  2743 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:24.257284 29145 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:06:24.258456 29145 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
W20260709 13:06:24.119150 29039 leader_election.cc:336] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.222s (SENT)
I20260709 13:06:24.121964 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 943) took 2416 ms. Trace:
I20260709 13:06:24.259940 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:24.261173 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 142051us
I20260709 13:06:24.260089 29905 rpcz_store.cc:276] 0709 13:06:21.697080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.722929 (+ 25849us) service_pool.cc:224] Handling call
0709 13:06:21.752272 (+ 29343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:21.752520 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:21.754161 (+  1641us) log.cc:844] Serialized 14 byte log entry
0709 13:06:23.255562 (+1501401us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b1800356c20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a 00007f7836b17a2a
0709 13:06:24.113408 (+857846us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.113467 (+    59us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:24.113516 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.113584 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.113696 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33314,"prepare.queue_time_us":1658,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":34769,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"spinlock_wait_cycles":3241344,"thread_start_us":842121,"threads_started":1,"wal-append.queue_time_us":2445867}
I20260709 13:06:24.262683 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.078249  2875 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:06:24.263587 29220 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:24.113709  2768 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:24.264257  2768 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.264925 29144 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:24.247711  2684 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.166682 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:24.268452 29139 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:24.168181 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:24.269832 29148 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:24.169875 29142 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:06:24.180214 29188 leader_election.cc:336] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.160s (SENT)
I20260709 13:06:24.271698 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:24.272833 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 92728.7us
W20260709 13:06:24.274796 29188 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.321s (SENT)
I20260709 13:06:24.111979  2871 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:24.109733  2876 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.081s
W20260709 13:06:24.182587  2609 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.014s	user 0.003s	sys 0.009s
I20260709 13:06:24.103971  2867 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.296201  2609 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:24.297952  2867 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.184311  2718 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:24.186389  2059 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.117606  2477 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:24.298628 29147 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:24.299134 29068 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:24.295616  2684 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.193382  2713 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.194522  2083 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:24.195657  2845 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:24.198742  2783 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.608s	user 0.009s	sys 0.002s
I20260709 13:06:24.132403  2636 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.116191  2868 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.065s
W20260709 13:06:24.161098  2816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.008s	sys 0.014s
I20260709 13:06:24.157485 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:24.209278 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 234851us
I20260709 13:06:24.332675 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:24.211791  2794 consensus_meta.cc:287] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.867s	user 0.009s	sys 0.002s
W20260709 13:06:24.333156 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 175766us
I20260709 13:06:24.239367 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:24.144222  2890 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:06:24.240449  2884 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:06:24.243189  2613 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.244747  2806 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.245780  2786 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:24.247632  2668 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.164321 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:24.345098 29146 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:24.277133  2770 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:24.277769  2900 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:24.278026  2785 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.279152 29213 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:24.369207 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:24.280377 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.280800 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.279081  2885 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:06:24.281602 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 150347us
I20260709 13:06:24.339043  2871 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:24.283267 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:24.239403  2804 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.245823  2809 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:24.285415  2860 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.624s	user 0.000s	sys 0.002s
I20260709 13:06:24.291265  2875 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:24.393581 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:24.367063  2814 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:24.293121  2709 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:24.398522  2709 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:06:24.293977  2846 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.495s	user 0.010s	sys 0.014s
I20260709 13:06:24.399452  2709 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:24.400151  2846 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.601s	user 0.011s	sys 0.015s
I20260709 13:06:24.190438  2886 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:24.400563  2709 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:24.303447 29188 leader_election.cc:336] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.951s (SENT)
I20260709 13:06:24.227833 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:24.353982 29905 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:24.357880  2820 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.404230 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:24.405217 29902 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:24.361249 29067 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:24.407233 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:24.408082 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 104704us
I20260709 13:06:24.362023  2684 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.362630 29071 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:24.363282  2876 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.364594 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2874) took 1909 ms. Trace:
I20260709 13:06:24.415881 29215 rpcz_store.cc:276] 0709 13:06:22.455424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.466755 (+ 11331us) service_pool.cc:224] Handling call
0709 13:06:22.559992 (+ 93237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:22.560159 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.560183 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:22.560222 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:22.560250 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:22.560298 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.364341 (+1804043us) spinlock_profiling.cc:243] Waited 92.9 ms on lock 0x7b5400132ed0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:24.364512 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213620480}
I20260709 13:06:24.418188 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:24.332842  2757 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:24.335945  2676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.016s	sys 0.007s
I20260709 13:06:24.332754  2304 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 18 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:24.377759  2868 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.281718 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.432114 29221 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:24.338025  2794 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:24.374289 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.376111 29220 raft_consensus.cc:2376] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18. Re-sending same reply.
I20260709 13:06:24.377022 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:24.320771  2083 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.375869 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:24.366029  2816 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:24.381064 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.392787 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:24.380390 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 856) took 2155 ms. Trace:
W20260709 13:06:24.486901 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 111176us
I20260709 13:06:24.487169 29070 rpcz_store.cc:276] 0709 13:06:22.224890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.249664 (+ 24774us) service_pool.cc:224] Handling call
0709 13:06:22.305662 (+ 55998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.493837 (+1188175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.380166 (+886329us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002ab554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:24.380336 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2453454720}
I20260709 13:06:24.488595 29070 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:24.489820 29070 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15. Ops: [12.8-14.9]
W20260709 13:06:24.490062 29038 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.793s (SENT) [suppressed 40 similar messages]
W20260709 13:06:24.433408  2676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.017s	sys 0.007s
I20260709 13:06:24.491521 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:24.492995 29067 raft_consensus.cc:2410] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:24.359578 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:24.411177 29213 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:24.494117 29072 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:24.494586 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.414557 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:24.375671  2871 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:24.427314 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.497980 29904 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
W20260709 13:06:24.498634 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 84165.1us
I20260709 13:06:24.370441 29214 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:24.500367 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.501137 29214 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:24.502100 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.503026 29214 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:24.438261 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:24.378697  2843 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.366202  2668 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.469530 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:24.505527 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 67336us
I20260709 13:06:24.507385 29221 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:24.440653 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.380743 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:24.509004 29908 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:24.267688  2768 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:24.491626 29038 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 2.018s (SENT)
I20260709 13:06:24.404129  2820 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.512157 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:24.513013 29902 raft_consensus.cc:2410] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:06:24.493521 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:24.511029 29038 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:24.516279 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:24.511760  2809 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.518631 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:24.526345 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:24.527949 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.529595 29214 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:24.439836  2814 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:24.373601  2785 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.520448  2458 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:24.581315 29069 raft_consensus.cc:2410] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 15 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 13:06:24.426575  2684 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.411770  2875 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.428455  2770 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 13:06:24.468173  2845 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.100s
I20260709 13:06:24.591481 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 862) took 1881 ms. Trace:
I20260709 13:06:24.591813 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:24.520709  2804 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:24.608193 29072 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:24.594620 29908 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
W20260709 13:06:24.584920 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.569725  2713 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:24.441633  2757 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.598399  2458 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:24.601145  2770 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.467131  2747 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.140s
I20260709 13:06:24.531816  2746 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:24.570865  2059 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:24.517036  2900 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:24.604013 29064 rpcz_store.cc:276] 0709 13:06:22.710016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.712102 (+  2086us) service_pool.cc:224] Handling call
0709 13:06:22.753972 (+ 41870us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:22.754362 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:22.754400 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:22.766672 (+ 12272us) log.cc:844] Serialized 26 byte log entry
0709 13:06:23.283530 (+516858us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:23.283572 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:23.283643 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:24.243754 (+960111us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:24.584555 (+340801us) raft_consensus.cc:1685] finished
0709 13:06:24.584578 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.591408 (+  6830us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":545,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":10608,"spinlock_wait_cycles":26319232,"thread_start_us":487758,"threads_started":1,"wal-append.queue_time_us":827086}
I20260709 13:06:24.586026  2884 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.437316  2876 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.584336  2676 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.470461  2868 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.624087 29908 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:24.630609 29907 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
W20260709 13:06:24.646064 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 146497us
I20260709 13:06:24.648066 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:24.522219  2816 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:24.617141  2879 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.662830 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:24.570724  2809 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.652993 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:24.624464  2713 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:24.667057 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:24.667277 29140 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:24.631156  2059 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.626639  2746 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:24.663406  2906 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:06:24.665141  2908 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:06:24.666333 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:24.700546 29146 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:24.671250 29147 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:24.617904  2304 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.690344  2746 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:24.624532  2770 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:24.666529  2910 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 13:06:24.667273 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 79533.3us
I20260709 13:06:24.668756  2890 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:06:24.714602 29215 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:24.715026 29215 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:24.665757  2458 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.674620  2059 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:24.532724  2843 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.654278  2783 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:24.660544  2747 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:06:24.698865  2889 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:24.663995  2857 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:06:24.661916  2806 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.645159  2884 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.646452  2814 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.617246  2871 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.654423  2676 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.653053  2900 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.703513  2906 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:24.715786 29215 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.671100  2879 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:24.732728  2915 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 13:06:24.727826  2918 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.004s
I20260709 13:06:24.736251  2809 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.737509  2713 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.744050  2868 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.707993  2742 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:24.745268  2875 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.726140  2083 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.746945  2684 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.748596  2785 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.750234  2919 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.721841  2604 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:24.752317  2816 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.757587  2770 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:24.762995 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.764076  2783 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.769236  2709 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:24.781076 29222 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2767) took 2180 ms (client timeout 2018 ms). Trace:
W20260709 13:06:24.873869 29222 rpcz_store.cc:269] 0709 13:06:22.600665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.719783 (+119118us) service_pool.cc:224] Handling call
0709 13:06:24.780809 (+2061026us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54000799d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:24.780984 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3758654720}
I20260709 13:06:24.791255  2689 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:24.817617  2676 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.819543  2900 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.824124 29143 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:24.823264 29218 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:24.800899  2857 consensus_queue.cc:1243] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:06:24.850672 29910 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:24.803167  2884 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.794442  2820 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.814339  2906 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:24.797856  2876 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.814950  2810 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:06:24.800451  2889 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.878966 29218 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:24.868606  2919 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:24.879853  2742 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:24.898952 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 180146us
I20260709 13:06:24.902706 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.913277 29214 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:06:24.879686  2781 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.385s	user 0.010s	sys 0.016s
I20260709 13:06:24.914489 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:24.918045 29039 leader_election.cc:336] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.445s (SENT)
I20260709 13:06:24.918613 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:24.912240 29218 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.898118  2876 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:24.920096 29218 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:24.921723 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:24.878881  2709 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.898480  2820 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:24.900163  2742 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:24.916935  2781 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.474s	user 0.010s	sys 0.016s
W20260709 13:06:24.930953 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:24.916749  2879 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.949863 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 1005) took 1072 ms. Trace:
I20260709 13:06:24.950356 29904 rpcz_store.cc:276] 0709 13:06:23.877425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.877561 (+   136us) service_pool.cc:224] Handling call
0709 13:06:23.877905 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.498373 (+620468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.949555 (+451182us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54005bd1d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:24.949800 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425590144}
W20260709 13:06:24.954919 29039 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 3.747s (SENT)
I20260709 13:06:24.955492 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:24.957432 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1007) took 1726 ms. Trace:
W20260709 13:06:24.960071  2755 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:06:24.960819  2927 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:24.961452  2927 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.961709  2059 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:24.964056  2059 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.964344  2879 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:24.957940 29218 rpcz_store.cc:276] 0709 13:06:23.231352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:23.294426 (+ 63074us) service_pool.cc:224] Handling call
0709 13:06:23.330489 (+ 36063us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.823139 (+1492650us) spinlock_profiling.cc:243] Waited 69.4 ms on lock 0x7b54000f9ea0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a84dc0 00007f7836b0f1c0 00007f7836b10231 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c
0709 13:06:24.957070 (+133931us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:24.957104 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:24.957139 (+    35us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:24.957166 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.957235 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:24.957372 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159549696}
W20260709 13:06:24.966758  2864 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 1.432s	user 0.014s	sys 0.002s
W20260709 13:06:24.970394  2786 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.623s	user 0.010s	sys 0.000s
I20260709 13:06:24.921371  2785 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:06:24.931396  2884 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:24.997007  2709 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:24.998600  2879 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:24.999212  2879 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.000674  2626 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:25.001351  2626 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:25.002130  2690 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:25.002658  2690 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:25.005036  2843 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:25.006989 29069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2766) took 3215 ms (client timeout 1724 ms). Trace:
W20260709 13:06:25.007392 29069 rpcz_store.cc:269] 0709 13:06:21.791183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:21.792048 (+   865us) service_pool.cc:224] Handling call
0709 13:06:25.006733 (+3214685us) spinlock_profiling.cc:243] Waited 2.79 s on lock 0x7b5400186d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:25.006895 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6411070848}
I20260709 13:06:25.013118  2783 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:25.014221  2921 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.014775  2921 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.017378  2921 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:25.025079 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 70198.2us
I20260709 13:06:24.926213  2876 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.037634 29906 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
W20260709 13:06:25.037434  2923 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:06:25.019729  2927 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:25.044474 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.052598  2794 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:24.934914  2820 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.084722  2458 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:25.088620 29908 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:25.087251  2626 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:25.091807  2915 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:06:25.038802  2746 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.003727  2690 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.073336  2668 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:25.110474  2919 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:25.117799  2879 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:25.134172  2626 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:25.137194 29220 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:06:25.138558 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.139297 29217 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:25.142858  2742 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.132145  2865 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:25.149075 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.149446  2865 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:25.098390  2913 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.580s	user 0.003s	sys 0.000s
I20260709 13:06:25.150869 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.145406 29220 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:06:25.130618  2876 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:25.148878  2690 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:25.144568 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.154588 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.151712 29222 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:25.157322 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.156831 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.160032 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.162113 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.163498 29070 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:25.163017 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.161528 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.169020  2454 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.170538 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:25.170696 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.171129 29220 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:06:25.171238 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.171424 29071 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:25.172039 29067 raft_consensus.cc:2410] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:25.145864  2889 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:25.096604  2458 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.175683 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.177532 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:25.178186 29070 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:25.179131 29902 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:25.150411  2865 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.170248  2454 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:25.187357 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 860) took 2477 ms. Trace:
I20260709 13:06:25.188061 29067 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:06:25.190310 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.192610  2746 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:25.193761  2865 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:25.194564  2458 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:25.194909 29219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2768) took 2452 ms (client timeout 1793 ms). Trace:
W20260709 13:06:25.195266 29219 rpcz_store.cc:269] 0709 13:06:22.742282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.773061 (+ 30779us) service_pool.cc:224] Handling call
0709 13:06:25.194730 (+2421669us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54000bd954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:25.194836 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502718208}
I20260709 13:06:25.187764 29066 rpcz_store.cc:276] 0709 13:06:22.709497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:22.709613 (+   116us) service_pool.cc:224] Handling call
0709 13:06:22.709907 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:23.004829 (+294922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:23.004988 (+   159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:23.005065 (+    77us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:23.005120 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.187118 (+2181998us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54002d3a54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:25.187171 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:25.187299 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677674368}
I20260709 13:06:25.200634  2820 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:25.203835 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.204649 29073 raft_consensus.cc:2368] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:06:25.206905 29070 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:25.200706  2441 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:25.223500 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 864) took 1216 ms. Trace:
I20260709 13:06:25.224000 29068 rpcz_store.cc:276] 0709 13:06:24.007363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.007760 (+   397us) service_pool.cc:224] Handling call
0709 13:06:24.008053 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.299528 (+291475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.223228 (+923700us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b5400129654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:25.223428 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669027072}
I20260709 13:06:25.227139 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.230332 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:25.231093 29219 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:25.232380 29068 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:06:25.209735  2906 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.232091  2877 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.232161  2458 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:25.471777 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:25.436064 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.570570 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:25.361829 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.568574 29220 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:25.278131 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 865) took 1270 ms. Trace:
W20260709 13:06:25.571717 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:25.332427 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.262538 29072 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:25.247822 29216 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:06:25.572166 29065 rpcz_store.cc:276] 0709 13:06:24.007656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.008302 (+   646us) service_pool.cc:224] Handling call
0709 13:06:24.008568 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.154907 (+146339us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:06:24.999567 (+844660us) log.cc:844] Serialized 14 byte log entry
0709 13:06:25.039181 (+ 39614us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:25.039240 (+    59us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:25.039286 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.277860 (+238574us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400185954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:25.277914 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:25.278066 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":844276,"prepare.queue_time_us":844888,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":1477,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":3758,"spinlock_wait_cycles":335914368,"thread_start_us":39265,"threads_started":1,"wal-append.queue_time_us":97860}
I20260709 13:06:25.244377 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.579457 29214 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:06:25.585441 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:25.325675  2917 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.627s	user 0.000s	sys 0.003s
W20260709 13:06:25.277345  2907 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.882s	user 0.009s	sys 0.000s
W20260709 13:06:25.284647  2935 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:06:25.589977 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.232546  2441 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:25.596684 29073 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:25.277369  2906 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:25.453604  2820 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.610817 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 957) took 1042 ms. Trace:
I20260709 13:06:25.611505 29910 rpcz_store.cc:276] 0709 13:06:24.568747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.568949 (+   202us) service_pool.cc:224] Handling call
0709 13:06:25.610415 (+1041466us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400768c54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:25.610694 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563146496}
I20260709 13:06:25.250494  2877 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.618113 29038 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:25.625994 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.626829 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.627538 29065 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:25.629097 29073 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:25.580821  2458 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.631652 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:25.634785 29066 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:25.633996 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 953) took 1252 ms. Trace:
I20260709 13:06:25.643760 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:25.635208 29907 rpcz_store.cc:276] 0709 13:06:24.381517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.381663 (+   146us) service_pool.cc:224] Handling call
0709 13:06:25.633837 (+1252174us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54005e92d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:25.633933 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330030848}
I20260709 13:06:25.695267 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2784) took 1205 ms. Trace:
I20260709 13:06:25.696694 29218 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:25.698601 29185 leader_election.cc:400] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:06:25.699117 29185 leader_election.cc:403] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:25.698926 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.604662  2441 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:20.951782 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:25.627962  2876 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:25.695714 29213 rpcz_store.cc:276] 0709 13:06:24.489836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.513645 (+ 23809us) service_pool.cc:224] Handling call
0709 13:06:24.513885 (+   240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:24.514085 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:24.514108 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:24.514574 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:06:24.570771 (+ 56197us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:24.570810 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:24.570874 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:25.279802 (+708928us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:25.633327 (+353525us) raft_consensus.cc:1685] finished
0709 13:06:25.633351 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:25.695194 (+ 61843us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2420992,"thread_start_us":55862,"threads_started":1,"wal-append.queue_time_us":125896}
I20260709 13:06:25.640256  2691 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.608196  2820 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.712553 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:25.702087  2898 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:25.694331  2843 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:25.624782  2683 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 13:06:25.620409  2906 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:06:25.745942 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.702370  2857 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:06:25.734004  2712 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:25.733502  2876 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.734884  2454 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:06:25.751055 29068 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:25.745360  2921 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:25.743970  2898 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.754398 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:25.756325 29067 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:25.752548  2876 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:25.742101  2691 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.752813  2865 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.748343  2843 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.750773  2336 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:06:25.755065  2795 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:25.758026  2898 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:25.751526  2712 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:25.760164  2921 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:25.768000  2938 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:06:25.765360  2865 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.783772  2865 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:25.785175 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 983) took 1133 ms. Trace:
I20260709 13:06:25.785368  2441 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:25.785642 29147 rpcz_store.cc:276] 0709 13:06:24.651359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.652708 (+  1349us) service_pool.cc:224] Handling call
0709 13:06:24.668284 (+ 15576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.701412 (+ 33128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.784850 (+1083438us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b5400192154. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:25.785102 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6016256}
I20260709 13:06:25.768220  2795 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.786352  2921 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.794094  2820 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:25.794756  2820 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:25.795267  2820 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.798063 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:25.798961  2683 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:25.799445  2683 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:25.800360 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.800513  2683 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.801416  2795 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:25.805941 29188 leader_election.cc:336] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.503s (SENT)
I20260709 13:06:25.808307  2820 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:25.816552 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.817890  2876 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.818472 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:25.819291 29213 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:25.820389  2876 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:25.822244  2898 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.825572  2898 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:25.829411  2458 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:25.832342  2871 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:25.835004  2608 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:25.842437 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.845407 29907 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:25.846169 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.848601  2691 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:25.850106 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 958) took 1253 ms. Trace:
I20260709 13:06:25.850558 29902 rpcz_store.cc:276] 0709 13:06:24.596362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.596602 (+   240us) service_pool.cc:224] Handling call
0709 13:06:24.622049 (+ 25447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.182415 (+560366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:25.849818 (+667403us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b54005bc554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:25.850031 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1277820672}
I20260709 13:06:25.852715 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.843789 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.857589 29221 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:25.859076 29221 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:06:25.865815 29113 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:25.887046 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.888098 29904 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:06:25.902376 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.904258 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.905521 29905 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:25.906697 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:25.907828 29910 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:25.893805  2889 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:25.909049  2889 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.909407  2889 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:25.911685 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.913192 29213 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:25.915715 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:25.912392 29902 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:25.934198 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:25.935292 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.900410  2872 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:06:25.934872 29902 raft_consensus.cc:2376] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21. Re-sending same reply.
I20260709 13:06:25.938254 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:25.921466  2865 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.939096 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.939100 29213 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:25.923906  2441 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:25.921247  2939 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:25.926764  2613 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:25.942333 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:25.943897 29902 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:25.950354 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.954515 29902 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:25.896073  2712 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:25.939808  2843 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:25.964263 29909 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:25.966588 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.966879 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.966609 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.967800 29903 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:25.968251 29909 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:25.968935 29875 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:25.969818  2816 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:25.970211 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:25.970517 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:25.971386  2816 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:25.971715 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.972679 29909 raft_consensus.cc:2410] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:06:25.973237 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:25.975622 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.976925 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:25.971760  2816 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:25.980083  2816 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:25.978081  2921 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:06:25.974803 29186 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:25.944294  2613 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:06:25.997329 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.998526  2879 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:25.999328  2879 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.003716  2879 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:26.009188  2879 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.011436  2879 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.960778  2939 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:26.019281  2939 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.020560  2877 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:26.024050 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.026180 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:26.027254 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 52532.5us
I20260709 13:06:26.027784 29910 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:26.028551 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:25.944072  2865 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:26.029824 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 269820us
W20260709 13:06:26.033334 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:25.948493  2441 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:25.989439  2691 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:25.990229  2843 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:26.039130 29186 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:26.038388 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:26.031354  2921 consensus_queue.cc:1243] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 13:06:26.047479 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.030740  2761 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:26.060669 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.061475 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.062312 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.039393  2336 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:26.111984 29909 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
W20260709 13:06:26.078893 29188 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.776s (SENT)
W20260709 13:06:26.125378 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46567.6us
W20260709 13:06:26.088343 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.045415  2844 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:06:26.123836 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 183447us
I20260709 13:06:26.131089 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.058781  2857 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:26.062965  2761 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:26.039724  2755 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:06:26.048048  2877 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:26.123111  2921 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:26.138482 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 163165us
I20260709 13:06:26.124400  2879 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:26.058589  2613 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.046384  2843 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.129117  2872 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:26.129738  2336 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:26.153295  2921 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:26.153805  2941 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:06:26.154366 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.155458  2850 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:26.156826 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.139896  2761 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:26.167688 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.168957 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:26.169682 29910 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:26.140286  2844 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:26.168570 29072 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:26.139281  2742 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:26.171234 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:26.172616 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.172027 29072 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:26.136183  2857 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.169926  2951 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.172082  2949 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.151134  2877 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.211023 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.213178 29072 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:26.181253  2857 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:26.218663 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.216375 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.220804  2441 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:26.184258  2844 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.160195  2872 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.210220  2613 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:26.223887 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.226030 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.225620 29140 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
W20260709 13:06:26.228154 29222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.004s	sys 0.004s
W20260709 13:06:26.229677 29222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.005s	sys 0.005s
I20260709 13:06:26.231935 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.232712 29910 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:26.232052 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.236140 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.237020 29142 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:26.238632 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.171065  2850 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.242153 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.240571 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.246296 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.255752  2967 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:26.222187  2477 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.257004 29141 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:06:26.257366 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.257622 29140 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:06:26.258131 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.259856 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.211977  2454 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 13:06:26.266274 29140 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:06:26.267266 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.267827 29140 raft_consensus.cc:2376] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19. Re-sending same reply.
I20260709 13:06:26.268703 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.271240 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.271884 29146 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:26.272953 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.274741 29141 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:26.275483 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.279950 29141 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:26.280153 29144 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:26.282986 29215 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:26.285175  2795 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:26.285820  2795 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.288092  2454 consensus_queue.cc:1243] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:06:26.289062  2889 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.290328 29148 raft_consensus.cc:2368] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20.
I20260709 13:06:26.291334 29146 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:26.291405  2889 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:26.216265  2949 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.304549 29142 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:26.304734  2921 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:26.305778  2949 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:26.292474 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:26.296919  2954 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:06:26.297724  2956 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:06:26.310822 29146 raft_consensus.cc:2376] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17. Re-sending same reply.
I20260709 13:06:26.311975 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.298844 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.314697 29139 raft_consensus.cc:2410] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:06:26.315487 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:26.315688 29908 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 960) took 1668 ms (client timeout 2081 ms). Trace:
I20260709 13:06:26.316054 29139 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
W20260709 13:06:26.316108 29908 rpcz_store.cc:269] 0709 13:06:24.647525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.647669 (+   144us) service_pool.cc:224] Handling call
0709 13:06:26.315538 (+1667869us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b540061fdd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:26.315634 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1011239168}
I20260709 13:06:26.316965 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.317945 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.318465 29139 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:26.318677  2757 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:26.319231 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.302376  2795 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:26.319795 29139 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:26.321708 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.324059 29139 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:06:26.313740 29145 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:26.320870 29146 raft_consensus.cc:2376] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18. Re-sending same reply.
W20260709 13:06:26.334199 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.343904 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2895) took 1117 ms. Trace:
I20260709 13:06:26.344481  2872 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:26.240132  2943 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.120s
I20260709 13:06:26.353539 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2898) took 1125 ms. Trace:
I20260709 13:06:26.361925  2877 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:26.363934  2850 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:26.368743  2843 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:26.259337  2865 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:26.213563  2951 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.227054  2336 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.277516  2477 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.377256 28964 catalog_manager.cc:5697] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:26.380766 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.381047 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:26.381708 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.384387 29147 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:26.386682  2889 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:26.298336  2950 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:06:26.387609  2898 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:26.390061 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:26.390232 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 66055.4us
I20260709 13:06:26.391947 29222 rpcz_store.cc:276] 0709 13:06:25.226967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.228422 (+  1455us) service_pool.cc:224] Handling call
0709 13:06:25.228827 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:25.569918 (+341091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:25.569955 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:25.581542 (+ 11587us) log.cc:844] Serialized 26 byte log entry
0709 13:06:26.231311 (+649769us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:26.231348 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.231413 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:26.343679 (+112266us) raft_consensus.cc:1685] finished
0709 13:06:26.343698 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.343827 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6072,"raft.queue_time_us":345809,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":3413,"thread_start_us":641323,"threads_started":1,"wal-append.queue_time_us":751447}
I20260709 13:06:26.406546 29219 rpcz_store.cc:276] 0709 13:06:25.228071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.236849 (+  8778us) service_pool.cc:224] Handling call
0709 13:06:25.237227 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.712724 (+475497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:25.712771 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:25.712834 (+    63us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:25.712892 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.353215 (+640323us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b5400132ed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:26.353264 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.353403 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1092908800}
W20260709 13:06:26.410180 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.404302  2690 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.425419  2948 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:26.418085  2871 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:26.426074 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.429189 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:26.437474 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.467983 29144 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:26.440057 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.469148 29910 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:26.471388 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:26.472167 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.481036 29188 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.179s (SENT)
I20260709 13:06:26.425046  2949 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
W20260709 13:06:26.509764 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.489374 29188 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.187s (SENT)
I20260709 13:06:26.541685 29188 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:26.542227 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:26.542550 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:26.543066 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 53740.4us
I20260709 13:06:26.422051  2872 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:26.457540 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 231378us
W20260709 13:06:26.461534 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44349.2us
I20260709 13:06:26.543183 29222 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:26.545023 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2688) took 1372 ms. Trace:
I20260709 13:06:26.442701  2898 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:26.430857  2742 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:06:26.485762 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 2794) took 1339 ms. Trace:
I20260709 13:06:26.549422 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:26.564360 29219 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:26.553252 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.545506 29071 rpcz_store.cc:276] 0709 13:06:25.173127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.173254 (+   127us) service_pool.cc:224] Handling call
0709 13:06:25.182896 (+  9642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:25.183075 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:25.189500 (+  6425us) log.cc:844] Serialized 14 byte log entry
0709 13:06:26.326138 (+1136638us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b18002798e0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a 00007f7836b17a2a
0709 13:06:26.544665 (+218527us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.544727 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:26.544774 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.544841 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:26.544965 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6262,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":87,"spinlock_wait_cycles":26735488,"thread_start_us":218241,"threads_started":1}
I20260709 13:06:26.566286 29218 rpcz_store.cc:276] 0709 13:06:25.146011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.146139 (+   128us) service_pool.cc:224] Handling call
0709 13:06:26.485525 (+1339386us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b54002837d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:26.485683 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1232669440}
I20260709 13:06:26.484725 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 979) took 1836 ms. Trace:
I20260709 13:06:26.569336 29143 rpcz_store.cc:276] 0709 13:06:24.648260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.648409 (+   149us) service_pool.cc:224] Handling call
0709 13:06:24.648673 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:24.878474 (+229801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.484419 (+1605945us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54003b38d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:26.484650 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401084800}
I20260709 13:06:26.570892 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:26.573766 29039 leader_election.cc:336] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.994s (SENT)
I20260709 13:06:26.434490  2865 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:06:26.587697 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:26.589143 29220 raft_consensus.cc:2410] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 19 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:06:26.456561  2843 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.587750 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:26.589498 29904 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:26.589885 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.593713 29073 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:06:26.442313  2816 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:26.613289 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.613282 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:26.614758 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:26.614966 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:26.615440 29220 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:26.610355 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.443212  2889 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.619520 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:26.463990  2690 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.545841  2877 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:26.577131  2844 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:06:26.679019 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:26.650115 28970 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 273) took 67050 ms (client timeout 15000 ms). Trace:
I20260709 13:06:26.475528  2871 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.692365 29903 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:06:26.714660 28970 rpcz_store.cc:269] 0709 13:05:19.377409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.377650 (+   241us) service_pool.cc:224] Handling call
0709 13:06:26.422780 (+67045130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4575295,"reported_tablets":50}
I20260709 13:06:26.471549  2948 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.493052  2785 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:26.698933  2963 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:06:26.697976 28964 catalog_manager.cc:5697] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 9 to 11, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 11 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:26.700407  2973 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:06:26.740557 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.741233 29146 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:26.742093 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.591907  2765 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 13:06:26.707254  2477 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:26.696825 29039 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.568s (SENT)
I20260709 13:06:26.571795 29143 raft_consensus.cc:2376] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16. Re-sending same reply.
I20260709 13:06:26.716741  2691 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:06:26.712275 29188 leader_election.cc:336] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.679s (SENT)
I20260709 13:06:26.752315 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:26.753170 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:26.753875 29188 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.713s (SENT)
I20260709 13:06:26.623116  2454 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:26.756064  2454 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:26.756574  2454 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:26.757431 28964 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 13 to 15, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 15 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:26.735761  2898 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.718292  2877 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.763223 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:26.764578  2958 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:06:26.766640 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:26.767239 29069 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:26.768013 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.622259  2843 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.722689 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:26.726837 29067 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:26.728382 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.764604 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.781070 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.781750 29068 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:26.782749 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.783412 29068 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:26.730391 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.784531 29065 raft_consensus.cc:2410] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:06:26.731673 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.732090 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.732868 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.786566 29064 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:06:26.582738  2446 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 13:06:26.561164  2872 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.743078 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:26.746197 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:26.747473 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:26.805893 29071 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:26.748250 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 51462.1us
I20260709 13:06:26.748538 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.808579 29143 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:26.809504 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.810015 29143 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:26.810606 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.446425 29140 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
W20260709 13:06:26.750808 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 206784us
I20260709 13:06:26.812189 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.812628 29140 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:26.813165 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.814821 29140 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:26.815711 29140 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:26.818475 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63641.8us
I20260709 13:06:26.821141 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.811225 29143 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:26.824528 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.826169 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:26.791399 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:26.829180 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.798718 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:26.824956 29140 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:06:26.820176 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.801909 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:26.837882 29146 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:26.842428 29069 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:26.847996 29146 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:06:26.853459 29147 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:06:26.856114 29147 pending_rounds.cc:85] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 9
I20260709 13:06:26.856451 29147 pending_rounds.cc:107] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:06:26.856858 29147 raft_consensus.cc:2889] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:26.853469 29140 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:26.856498 29146 pending_rounds.cc:85] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 14
I20260709 13:06:26.858994 29146 pending_rounds.cc:107] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:06:26.859422 29146 raft_consensus.cc:2889] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:26.845217 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:26.872951 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:26.873776 29072 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:06:26.875180 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 63151.2us
I20260709 13:06:26.759373 28964 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 11 to 13, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 13 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:26.754431  2690 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:26.877372 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:26.877578 29143 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:26.878229 29148 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:26.878636 29146 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:26.806841  2816 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:26.886793 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:26.874257  2613 consensus_meta.cc:287] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.643s	user 0.010s	sys 0.001s
I20260709 13:06:26.835961  2879 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:26.834287  2477 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:26.895632  2877 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:26.898881  2843 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:26.899340  2970 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:06:26.900596  2871 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:26.903728  2872 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:26.876647  2604 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:26.908700 29065 raft_consensus.cc:2410] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:06:26.884331  2865 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:26.916015 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:26.916996 29216 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:26.918524  2761 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.742s	user 0.009s	sys 0.000s
I20260709 13:06:26.918989  2761 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:26.852761  2668 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
W20260709 13:06:26.929847 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 110762us
I20260709 13:06:26.884613  2768 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 13:06:26.905848  2951 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:26.913365  2865 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:26.945382 29215 consensus_meta.cc:287] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.662s	user 0.009s	sys 0.000s
I20260709 13:06:26.947785 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:26.955260 29215 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:06:26.955662 29215 pending_rounds.cc:85] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 10
I20260709 13:06:26.955890 29215 pending_rounds.cc:107] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:06:26.960155 29215 raft_consensus.cc:2889] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:26.966739 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:26.983642 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 2886) took 2270 ms. Trace:
I20260709 13:06:26.974984 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:26.957873 29222 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:26.907815  2879 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.910640  2972 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
W20260709 13:06:26.959650 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 83370.5us
I20260709 13:06:26.984234 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:26.985633  2975 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:06:26.984942 28964 catalog_manager.cc:5697] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 14 to 16, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 16 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:26.987457  2949 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:27.011373  2949 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:26.988265 29039 leader_election.cc:336] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.793s (SENT)
I20260709 13:06:26.905707  2965 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
W20260709 13:06:26.960613 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 243334us
I20260709 13:06:26.984189 29215 rpcz_store.cc:276] 0709 13:06:24.713604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:24.714172 (+   568us) service_pool.cc:224] Handling call
0709 13:06:24.714481 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.282678 (+1568197us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b58009644d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a690aa 00007f7836aeb447 00007f7836b03fe7 00007f7836b057de 00007f7836b0fb49 00007f7836b10416 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4
0709 13:06:26.983368 (+700690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:26.983580 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629399936}
I20260709 13:06:26.991830 29188 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:26.966497 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:27.041085 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:27.041962 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:27.042299 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:27.042755 29214 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:26.993875 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:26.994127 29146 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
W20260709 13:06:26.994455 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 158439us
I20260709 13:06:27.005518 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:26.936743  2850 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:06:26.988173  2458 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.012416  2889 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:27.014478 28964 catalog_manager.cc:5697] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.016132  2477 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:27.305231 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 267707us
I20260709 13:06:27.022431  2946 raft_consensus.cc:2764] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:06:26.989356  2454 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:26.952638  2865 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:26.958232  2822 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:27.030814 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:27.321102 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 332856us
W20260709 13:06:27.322781 29039 leader_election.cc:336] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.037s (SENT)
I20260709 13:06:27.033691 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:27.043845 29215 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:27.045074 29216 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:27.290150 29066 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:06:27.303653 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 88452.4us
I20260709 13:06:27.326028  2668 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:27.299248 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 265611us
I20260709 13:06:27.020406  2948 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:27.310395 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 274965us
I20260709 13:06:27.328615 29871 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:26.958793  2336 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.348s
I20260709 13:06:26.937168  2765 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 4, Time since last communication: 0.366s
W20260709 13:06:27.287106  2857 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.054s	user 0.012s	sys 0.000s
I20260709 13:06:27.329876 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:27.319092  2761 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:27.344867 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:27.352288 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:27.354432 29113 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:27.358042 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48366.1us
I20260709 13:06:27.341986  2336 consensus_queue.cc:1243] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:06:27.343653  2765 consensus_queue.cc:1243] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 17.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:06:27.346794  2608 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:27.385538 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 57008.7us
I20260709 13:06:27.348503  2972 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.313197  2458 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.357462  2948 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:06:27.389617 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 397850us
I20260709 13:06:27.375947  2889 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:27.405845 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:27.427583  2979 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:06:27.387595  2978 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:06:27.435527  2994 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:27.436954  2454 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:27.388526  2746 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:27.388988  2965 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:27.347607  2857 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:27.462285 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.398783  2948 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:27.468734 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 60097.7us
I20260709 13:06:27.475046  2746 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:27.475898  2746 consensus_queue.cc:1243] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:06:27.480080  2943 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:27.483104  2943 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:06:27.484058 28964 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 12 to 14, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 14 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.466481  2965 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:27.487593  2965 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:27.484102  2987 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:27.488523  2987 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:27.484967 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:27.489383  2987 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.490307 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:27.490635  2987 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:27.491592  2943 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:27.346393  2949 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.492631  2943 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:27.469466  2889 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:27.493860  2889 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:27.493943  2943 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.496256  2879 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:27.498983 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 57926.4us
W20260709 13:06:27.500013 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.502666  2336 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:06:27.505988 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.506974 29902 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:27.516108 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.473754  2948 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:27.435410  2865 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.410032  2972 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.525157 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:27.525368 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.526507 29910 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:27.487309 28964 catalog_manager.cc:5697] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.530489 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.529783 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:27.533653 29039 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.113s (SENT)
I20260709 13:06:27.535059 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:27.536072 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.536752 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.537429 29902 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:27.508533  2782 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
W20260709 13:06:27.541855 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.543052 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.547955 29902 raft_consensus.cc:2410] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:06:27.544096 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.544261 29909 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:27.543826 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.550743 29143 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:06:27.555430 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 962) took 1339 ms. Trace:
I20260709 13:06:27.556206 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.557204 29906 raft_consensus.cc:2410] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:06:27.570950  2865 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:27.555902 29903 rpcz_store.cc:276] 0709 13:06:26.215935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.216068 (+   133us) service_pool.cc:224] Handling call
0709 13:06:27.555203 (+1339135us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b54004e61d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:27.555362 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1084175616}
I20260709 13:06:27.575886  2921 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:27.576421 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:27.576943  2857 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:27.577229 29215 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:06:27.576473  2921 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.578723  2972 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:27.579826  2921 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:27.581457 29070 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:06:27.583163 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2698) took 1998 ms. Trace:
I20260709 13:06:27.583518 29070 rpcz_store.cc:276] 0709 13:06:25.585082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.585190 (+   108us) service_pool.cc:224] Handling call
0709 13:06:27.582970 (+1997780us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b5400187c54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:27.583108 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4587848448}
I20260709 13:06:27.585997 29215 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
W20260709 13:06:27.622014 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 819456us
W20260709 13:06:27.622987 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:27.623462 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:27.623961 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.636371 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.641096 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:27.642066 29903 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:27.616191  2804 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.660207  2985 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:27.660437  2804 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.661150  2985 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:27.628697  2690 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.648355 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:27.662312  2985 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:27.663269 29111 connection.cc:256] RPC call timeout handler was delayed by -1.0028953520004507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
I20260709 13:06:27.663857 29906 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:27.575555  2965 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.662871  2690 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:27.679944 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:27.664378  2691 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.006s	sys 0.010s
I20260709 13:06:27.677052 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:27.665194  2865 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:27.683543 29037 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:27.675486 29906 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:27.683765 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.687152 29903 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:27.687731 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:27.681828  2691 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.006s	sys 0.010s
I20260709 13:06:27.688805 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.689532 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:27.693463 29188 leader_election.cc:336] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.894s (SENT)
I20260709 13:06:27.697679 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:27.697958 29908 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:27.699491 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:27.700130 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:27.701921 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:27.696738  2691 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:27.702812  2668 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:27.703615  2691 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.705554  2690 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:27.706142  2982 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.674s	user 0.002s	sys 0.002s
I20260709 13:06:27.682093  2822 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:27.707939 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:27.707942 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.713951 29037 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:27.731350 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:27.732829  2690 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:27.697269  2865 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.736112  2986 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:27.736222 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.737787 29903 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:27.749466  2986 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:27.750473  2986 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:27.753970  2986 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:27.784802 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63112.4us
I20260709 13:06:27.786355 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:27.786944 29902 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:27.788507 29908 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:27.795150 29907 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:27.782987 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:27.698009  2879 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:27.800899 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:27.676162  2985 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:06:27.718225  2668 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:27.712428  2888 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:27.780488  2898 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:27.737808  2865 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:27.809291 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:27.802455  2879 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:27.806630  2978 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:27.800638  2943 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 13:06:27.832983  2971 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:27.833598  2971 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:27.834663  2971 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.824205  2865 raft_consensus.cc:2764] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:06:27.840171  2948 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.842324 29906 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:06:27.843055  2948 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:27.841607 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:27.825112  2879 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:27.853783 29902 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:27.854215  2965 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:27.855949 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 969) took 1927 ms. Trace:
I20260709 13:06:27.880213 29904 rpcz_store.cc:276] 0709 13:06:25.928312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.928432 (+   120us) service_pool.cc:224] Handling call
0709 13:06:26.585622 (+657190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.606036 (+ 20414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:27.855681 (+1249645us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54007a0150. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:27.855846 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510708096}
I20260709 13:06:27.859068  2804 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:27.859990 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:27.881736  2441 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:27.882407  2441 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:27.883666 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:27.884754  2965 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:27.866154  2458 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:27.877308  2985 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:27.885730  2965 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:27.886648  2971 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:27.885169 29874 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:27.893443  2965 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:27.894682 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:27.897567  3016 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:27.898663  3016 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:27.899186  2980 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.616s	user 0.003s	sys 0.000s
I20260709 13:06:27.883610  2441 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.901412  2441 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:27.899153  3016 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:27.916569  2879 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.920532  2712 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:27.920539  2949 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:27.919816  2879 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:27.950453  3016 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:27.953449  3016 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:27.696105 28964 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 17 to 19, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 19 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:27.993168 28964 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:28.005151 28964 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 10 to 12, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 12 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.012811 28964 catalog_manager.cc:5697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.017490 28964 catalog_manager.cc:5697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:28.030977 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 229993us
I20260709 13:06:28.033775 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:28.036391 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 342794us
I20260709 13:06:28.037955 29143 raft_consensus.cc:1217] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Deduplicated request from leader. Original: 11.10->[12.11-21.15]   Dedup: 17.13->[19.14-21.15]
I20260709 13:06:28.038848 29143 pending_rounds.cc:85] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 13
I20260709 13:06:28.039144 29143 pending_rounds.cc:107] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:06:28.039561 29143 raft_consensus.cc:2889] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:28.040095 29188 leader_election.cc:336] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.895s (SENT)
I20260709 13:06:28.041261 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:28.042610 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.046669 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2725) took 1508 ms. Trace:
I20260709 13:06:28.051083 29148 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:28.057466 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:28.057130 29905 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:28.061672 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:28.075426 29905 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 968) took 2149 ms (client timeout 2625 ms). Trace:
I20260709 13:06:28.058288 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:28.076040 29905 rpcz_store.cc:269] 0709 13:06:25.926374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:25.926516 (+   142us) service_pool.cc:224] Handling call
0709 13:06:28.075159 (+2148643us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b54007102d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:28.075316 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4877293440}
I20260709 13:06:28.035588 29140 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:28.079035  2336 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:28.080540  2446 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:28.081588  3016 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:28.081707 29188 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.942s (SENT)
I20260709 13:06:28.083045 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:28.086766  2782 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.087419  3002 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:28.087716  2782 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:28.083405  3016 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:28.091010 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.092772  2946 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:28.093606  3016 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:28.070539 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 985) took 1571 ms. Trace:
I20260709 13:06:28.094599 29144 rpcz_store.cc:276] 0709 13:06:26.472777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.472920 (+   143us) service_pool.cc:224] Handling call
0709 13:06:26.547023 (+ 74103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.748973 (+201950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.749001 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.749039 (+    38us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:26.749069 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.043128 (+1294059us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54002e8954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:28.043239 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.043805 (+   566us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463986432}
W20260709 13:06:28.062476 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.058780 29904 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:28.070034 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2788) took 1335 ms. Trace:
W20260709 13:06:28.063194 29142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2700) took 1824 ms (client timeout 1895 ms). Trace:
W20260709 13:06:28.100477 29142 rpcz_store.cc:269] 0709 13:06:26.238276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.238399 (+   123us) service_pool.cc:224] Handling call
0709 13:06:28.062715 (+1824316us) spinlock_profiling.cc:243] Waited 65.3 ms on lock 0x7b540027d8d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:28.062913 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150137088}
I20260709 13:06:28.052673 29219 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:06:28.107096 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.072026 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:28.047503 29139 rpcz_store.cc:276] 0709 13:06:26.537811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.540181 (+  2370us) service_pool.cc:224] Handling call
0709 13:06:26.540456 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.755032 (+214576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.755054 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.755133 (+    79us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:26.755160 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.046050 (+1290890us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54003cc3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:28.046122 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.046461 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493045504}
I20260709 13:06:28.098227 29066 rpcz_store.cc:276] 0709 13:06:26.734264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.816608 (+ 82344us) service_pool.cc:224] Handling call
0709 13:06:26.853352 (+ 36744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:27.325567 (+472215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.069346 (+743779us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b540012aa54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:28.069786 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1003396992}
I20260709 13:06:28.031791 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:28.119055  2782 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:28.119820 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2790) took 1294 ms. Trace:
I20260709 13:06:28.120601  2441 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:28.121090 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 1027) took 1234 ms. Trace:
W20260709 13:06:28.121202 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 49854.1us
I20260709 13:06:28.120290 29065 rpcz_store.cc:276] 0709 13:06:26.749698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.820819 (+ 71121us) service_pool.cc:224] Handling call
0709 13:06:28.044018 (+1223199us) spinlock_profiling.cc:243] Waited 86.4 ms on lock 0x7b5400186354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:28.044352 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198713728}
I20260709 13:06:28.122892 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:28.124607 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2722) took 1703 ms. Trace:
I20260709 13:06:28.125072 29141 rpcz_store.cc:276] 0709 13:06:26.421637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.421803 (+   166us) service_pool.cc:224] Handling call
0709 13:06:26.422451 (+   648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.748809 (+326358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:26.748840 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:26.748880 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:26.748914 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.123966 (+1375052us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b540029f254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:28.124089 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.124406 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":750106368}
I20260709 13:06:28.128111 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:28.129844 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:28.130630 29220 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:28.135005 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:28.135967 29902 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:28.137393 29216 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:06:28.139151 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:28.145037 29215 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:28.145287 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1024) took 1403 ms. Trace:
I20260709 13:06:28.145742 29219 rpcz_store.cc:276] 0709 13:06:26.742302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.742735 (+   433us) service_pool.cc:224] Handling call
0709 13:06:26.742987 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:28.110010 (+1367023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.144960 (+ 34950us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54002f62d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:28.145196 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3009088768}
I20260709 13:06:28.157064  2441 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:28.157727  2441 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:28.158190  2441 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:28.166088 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 43599us
I20260709 13:06:28.170487 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:28.140583 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2724) took 1680 ms. Trace:
I20260709 13:06:28.183439 29147 rpcz_store.cc:276] 0709 13:06:26.460417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.460578 (+   161us) service_pool.cc:224] Handling call
0709 13:06:26.537909 (+ 77331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.857732 (+319823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.139894 (+1282162us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b540001c554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:28.140274 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":720622336}
I20260709 13:06:28.184321  2939 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:28.141134 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:28.185038  2879 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:28.185647 29904 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:28.185278  2939 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:28.186152  2879 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:28.174077  3016 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:28.176810  2971 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:28.188324  2698 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:28.179525  2712 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:06:28.189386  2698 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:28.181046 29218 rpcz_store.cc:276] 0709 13:06:26.886332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.886494 (+   162us) service_pool.cc:224] Handling call
0709 13:06:28.108667 (+1222173us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000cc6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:28.120859 (+ 12192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2670812544}
I20260709 13:06:28.171674  2690 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:28.192274  2613 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.192438  2698 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:28.193444  2698 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:28.193645  2604 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:28.194289  2604 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:28.194424  2441 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:28.193296  2613 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:28.143121 29140 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:28.200954  2879 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:28.204703  3017 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:06:28.204413 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 173010us
I20260709 13:06:28.205358  2939 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:28.195987  2604 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:28.209584  2698 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:28.183609 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:28.211951 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:28.212935 29038 connection.cc:256] RPC call timeout handler was delayed by -1.3639581859997634s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1902s for a response
I20260709 13:06:28.213044 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:28.216777 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 143713us
I20260709 13:06:28.174742 28964 catalog_manager.cc:5697] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.237418  2690 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:28.265383 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 53719.7us
W20260709 13:06:28.266165 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 66525.5us
I20260709 13:06:28.274379 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:28.276515  2888 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:28.276844 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 93727.8us
I20260709 13:06:28.277633  2441 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:28.278599  2613 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:28.281216 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:28.281435  2604 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:28.281951 29186 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:28.261732  3017 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:28.291723 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:28.299278  2698 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:28.288453  2939 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:28.313365 29220 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:28.321269 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:28.324740  2458 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:06:28.298408 28964 catalog_manager.cc:5697] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 11 to 13, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 13 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.338676  2885 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.339661  2885 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:28.339762 28964 catalog_manager.cc:5697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 12 to 15, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 15 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.342620  2885 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:28.344259 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:28.345171 29215 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
W20260709 13:06:28.359308 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 77532.1us
I20260709 13:06:28.370872 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:28.367094 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 70415.8us
I20260709 13:06:28.363116  2446 raft_consensus.cc:2764] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:06:28.392544 29902 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:28.386886  2690 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:28.408357 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:28.408717 29217 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:06:28.409530 29069 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:28.410650 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:28.414492  2885 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:28.415300  2885 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:28.415848  2885 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:28.420747 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:28.421114 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2808) took 1450 ms. Trace:
I20260709 13:06:28.421485 29066 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:28.423054 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:28.424722 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:28.425045  2850 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:28.425650 29068 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:28.427073  2951 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:06:28.430277 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 93017.1us
I20260709 13:06:28.427405  2850 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:28.432617  2850 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:28.433997 29066 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:28.421710 29221 rpcz_store.cc:276] 0709 13:06:26.970762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:26.972342 (+  1580us) service_pool.cc:224] Handling call
0709 13:06:26.972668 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:26.972902 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:28.132976 (+1160074us) spinlock_profiling.cc:243] Waited 7.73 ms on lock 0x7b18004b8f20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f783a5053c7 00007f7836b179a7 00007f7836b270bf
0709 13:06:28.133287 (+   311us) log.cc:844] Serialized 14 byte log entry
0709 13:06:28.370180 (+236893us) log.cc:844] Serialized 14 byte log entry
0709 13:06:28.384866 (+ 14686us) log.cc:844] Serialized 14 byte log entry
0709 13:06:28.420193 (+ 35327us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:28.420291 (+    98us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:28.420408 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:28.420530 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:28.420913 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69592,"prepare.queue_time_us":372066,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":3766,"raft.queue_time_us":15257,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":188,"spinlock_wait_cycles":17771776,"thread_start_us":442129,"threads_started":2,"wal-append.queue_time_us":164421}
I20260709 13:06:28.439706  2949 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:28.440454  2949 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:28.444106  2949 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:28.448009 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 60576.9us
W20260709 13:06:28.456741  3029 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.068s	user 0.003s	sys 0.000s
I20260709 13:06:28.458962 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:28.460173 29216 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:28.464706 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:28.465749 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:28.466101 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 70048.1us
I20260709 13:06:28.466625 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:28.468333 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 176730us
I20260709 13:06:28.468626  2971 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:28.469329 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:28.469539  2971 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:28.471524 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:28.484952 29904 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:28.497076  2951 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:28.497689  2951 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:28.498143  2951 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:28.500077  2690 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:28.523679 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44361.7us
I20260709 13:06:28.535320  2690 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:28.536110  2782 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.541543  2782 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:28.544217  2458 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:28.536417 29221 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 13:06:28.536899 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:28.539561 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 86303.6us
I20260709 13:06:28.552141  2690 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:28.553042  2690 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:28.553582 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:28.553915 29140 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:28.554359 29145 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:28.554894 29148 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:06:28.573315  3017 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:28.576458  2782 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:28.580729  2690 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:28.584463  3026 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:06:28.575632  2888 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:06:28.615764 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:28.562654  2446 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:28.521515  2712 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:06:28.563197  2949 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.518076  2971 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:28.694413 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:28.695289 29073 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:28.698100 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 226744us
I20260709 13:06:28.699234 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:28.700254  2879 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:28.654446  2949 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:28.734537 29904 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
W20260709 13:06:28.727201 29188 leader_election.cc:336] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.934s (SENT)
W20260709 13:06:28.737746 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 264208us
I20260709 13:06:28.723661  2712 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:28.719177  2690 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:28.682938  2951 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:28.676856  2446 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:28.775383 29073 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
W20260709 13:06:28.775739 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 74887.1us
I20260709 13:06:28.615319 28964 catalog_manager.cc:5697] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 11 to 15, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 15 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:28.748077  2441 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:28.743577  2712 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:28.785203 29216 raft_consensus.cc:2410] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:06:28.810309 28964 catalog_manager.cc:5697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 7 to 16. New cstate: current_term: 16 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:28.811991 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 250740us
I20260709 13:06:28.812665 28964 catalog_manager.cc:5697] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 8 to 12, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 12 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.815510  2712 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:28.817338  2712 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:28.799428  2604 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.003s
I20260709 13:06:28.790012  2441 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:28.837572 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:28.804977  2885 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:28.837359  2939 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.005s
W20260709 13:06:28.852352 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 53561.4us
W20260709 13:06:28.853148 29039 leader_election.cc:336] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.544s (SENT)
I20260709 13:06:28.854090  3027 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:28.857164  2949 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:28.867832 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:28.875686 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 187823us
I20260709 13:06:28.868651 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:28.819793  2698 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:06:28.891057 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:28.903312 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:28.886268  2939 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:28.923089 29070 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:28.929369 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:28.934999 29188 leader_election.cc:336] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.071s (SENT)
I20260709 13:06:28.940454 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:28.941390 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:28.933600 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:28.951161 29071 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:29.006484  2949 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:29.013677  3043 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:29.014498 29071 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:06:29.015003  2949 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:29.016301  2949 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.020887  2446 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:29.024183 29067 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:06:29.032640 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 107947us
I20260709 13:06:29.034090 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:29.036343  3027 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:28.992496  2951 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:29.040015  2804 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:29.040503  2804 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:29.041481  2804 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.043010  2804 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:29.047197  2850 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.004s
I20260709 13:06:29.048569  2885 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:29.050470  2951 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:29.051232 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:29.014360 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:29.024290 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 200118us
W20260709 13:06:29.014071 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 91416.4us
I20260709 13:06:29.060844 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:29.061662 29073 raft_consensus.cc:2368] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16.
W20260709 13:06:29.062548 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 281775us
W20260709 13:06:29.064618 29068 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.638s	user 0.010s	sys 0.006s
I20260709 13:06:29.067369 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:29.067668 29068 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:29.069607 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:29.070642 29071 raft_consensus.cc:2368] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14.
I20260709 13:06:29.069479 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:29.075242 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 262454us
I20260709 13:06:29.096028 29070 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:29.095623 29065 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:29.097640 29066 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:29.105525 29222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "813e480732fd4061837e784d4d768b1f"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:29.106542 29222 raft_consensus.cc:606] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:29.106917 29222 raft_consensus.cc:614] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:29.112154  3027 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:29.136072  3008 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:06:29.144313 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 108512us
I20260709 13:06:29.146389 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.146730 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.147703 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.148101  3027 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:29.148458 29140 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:29.148355 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.149423 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.150286 29146 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:29.150343 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.152266 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.153057 29145 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:29.157822 29188 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.187s (SENT)
W20260709 13:06:29.159179 29188 leader_election.cc:336] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.214s (SENT)
W20260709 13:06:29.160148 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 59739.7us
I20260709 13:06:29.160926 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:29.161369 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.162300  2971 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:29.060750  2850 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:29.060372  2951 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.159602 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:29.165103  3044 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:06:29.165246  2971 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:29.185286 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:29.187004 29145 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:06:29.188860 29218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:29.189622 29218 raft_consensus.cc:606] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:29.189965 29218 raft_consensus.cc:614] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:29.195569 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:29.196655 29067 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:29.203536 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:29.204285 29068 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:29.208559 29070 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:06:29.209983 29065 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:29.211707 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:29.184401  2446 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:29.233860 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 74716.6us
I20260709 13:06:29.241238 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.242108 29148 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:29.240607 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.244824 29145 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
W20260709 13:06:29.243378 29188 leader_election.cc:336] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.553s (SENT)
I20260709 13:06:29.253805 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:29.254400 29069 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:29.256011 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:29.258070  2959 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:29.258409 29065 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:29.258816  2959 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.260210 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.228565  3055 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:06:29.260882 29146 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:29.261493  3055 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:29.261207  2971 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.261925 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.262287 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.263262 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.263940 29147 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:06:29.264349 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.264451 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:29.260767 29221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:29.268906 29144 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:29.265865 29221 raft_consensus.cc:606] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:29.269901 29221 raft_consensus.cc:614] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:29.271466 29139 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:29.235145  3027 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.265002 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:29.274048 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:29.281580 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.297406  3027 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.297600 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:29.298319 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.299304 29140 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:29.300243 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.301368 29146 raft_consensus.cc:2410] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:06:29.293816  2446 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:29.303896  2959 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:29.304919 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 70022.4us
I20260709 13:06:29.307418 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:29.260145  2850 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:29.261936  3055 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:06:29.309530 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 244104us
I20260709 13:06:29.316962  3027 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:29.326231 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.327175 29146 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:29.333274 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:29.335592 29065 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:29.336722 29221 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:29.337062 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:29.340988 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:29.342463 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.344358 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:29.334937  2804 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:29.351471 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 186521us
W20260709 13:06:29.340974  2712 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.523s	user 0.003s	sys 0.012s
W20260709 13:06:29.353470 29039 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.183s (SENT)
I20260709 13:06:29.358640 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:29.363579 29142 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:29.350270  3027 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:29.398423 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:29.400377 29188 leader_election.cc:336] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.468s (SENT)
I20260709 13:06:29.404059 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:29.409624  3050 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:06:29.409390  3055 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.411234  3055 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:29.411592  3059 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:06:29.381349  2804 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:29.416114 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.418435 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.419381 29148 raft_consensus.cc:2410] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:29.405851  2441 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:29.407127 29146 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:29.420334 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.407112 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.421147 29139 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:29.415863 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:29.389290  3027 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.422276 29145 raft_consensus.cc:2410] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:06:29.422614  3027 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:29.423499 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.424273 29139 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:29.425170 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.425874 29139 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:29.426977 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.405781  2712 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:06:29.428323 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:29.428555  2712 consensus_queue.cc:1243] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 12.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:06:29.428994 29142 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:29.431315  2804 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:29.432528  3057 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:06:29.433794 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:06:29.435985 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:06:29.450935  3030 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:06:29.386584  2949 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.457602  2949 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.451874  2951 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:29.462513 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 148758us
I20260709 13:06:29.465909 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.467255 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.467418 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:29.468398 29067 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:29.468837 29141 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:29.469446 29070 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:06:29.469976 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.464756  2446 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:29.472985 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:29.473184 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:29.474139 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 129739us
I20260709 13:06:29.474354 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.474997 29145 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
W20260709 13:06:29.475327 29039 leader_election.cc:336] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.620s (SENT)
I20260709 13:06:29.476259  2885 raft_consensus.cc:2764] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:06:29.476974 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.478339 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.479030 29143 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:29.480566 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:29.482841 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 115561us
W20260709 13:06:29.483608 29188 leader_election.cc:336] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.520s (SENT)
I20260709 13:06:29.477058  2885 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:29.484036 29188 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:29.485778  2879 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:29.486320  2879 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.486726  2879 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:29.487413  2850 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:29.487650  2804 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:29.489264  2939 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:29.489720 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:29.475862 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:29.490860 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 43820us
I20260709 13:06:29.475724 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:29.495728 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.479766 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.461818  2949 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:29.528753 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 278416us
I20260709 13:06:29.493077 29146 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:29.532516 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.540644 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.532024 29216 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:29.533370 29139 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:29.544510 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.541292 29147 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:29.534881 29143 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:29.550716 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:29.551887 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 60918.7us
W20260709 13:06:29.536406 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 231945us
I20260709 13:06:29.499279  2712 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:29.555706 29145 raft_consensus.cc:2410] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:06:29.584587 29904 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:06:29.586357 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:29.590184 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:29.591347 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:29.606426 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.608242 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.609761 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.601167 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.607306 29146 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:29.613761 29906 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:06:29.614279 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:29.615236 29145 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:29.615494 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:29.616286 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 141007us
W20260709 13:06:29.619115 29039 leader_election.cc:336] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.341s (SENT)
W20260709 13:06:29.620131 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.540269  2885 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:29.634644 29141 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
W20260709 13:06:29.604782 29185 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.866s (SENT) [suppressed 70 similar messages]
I20260709 13:06:29.644476 29140 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:29.619644 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:29.654592 29147 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
W20260709 13:06:29.661980 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 57243.3us
I20260709 13:06:29.663205 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:29.663991 29143 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:29.668565 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62580.7us
W20260709 13:06:29.670349 29188 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.603s (SENT)
I20260709 13:06:29.674131 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:29.610738  2441 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:29.676827 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:29.677241 29147 pending_rounds.cc:85] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 12
I20260709 13:06:29.681684 29147 pending_rounds.cc:107] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 12.13
I20260709 13:06:29.678406 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:29.652302  2999 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.681983 29147 raft_consensus.cc:2889] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:29.690984 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:29.691692 29215 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:29.693475 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.697746 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:29.698612 29072 raft_consensus.cc:2368] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14.
I20260709 13:06:29.700012 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:29.701761 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:29.703264 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:29.704274 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:29.704584 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:29.704700 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:29.705607 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:29.706305 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:29.707294 29910 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:06:29.708854 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:29.709933  2604 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.710500  2604 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:29.680823  2441 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.713845  2441 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:29.714236 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.666226  2939 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:29.704998 29902 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:06:29.724776 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 105699us
I20260709 13:06:29.728467 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:29.729493 29216 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:29.729776 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:29.730873  3060 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:29.731105  3027 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.653782  2712 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.731526  3060 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.735002 29142 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:29.707264 29903 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:29.735760 29911 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:29.684317  3067 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.756132 29902 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:29.770802 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:29.771883 29142 raft_consensus.cc:2376] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20. Re-sending same reply.
I20260709 13:06:29.733865  2712 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:29.778061 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 140421us
I20260709 13:06:29.795077 29072 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:06:29.827950 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 157616us
I20260709 13:06:29.830585 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:29.831691 29070 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:29.833896 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:29.838647 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:29.838632 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:29.882447  2885 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:29.792636  3050 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:06:29.840657 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:29.882514 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 43997.3us
W20260709 13:06:29.885695 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 148663us
W20260709 13:06:29.891644 29039 leader_election.cc:336] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.671s (SENT)
I20260709 13:06:29.868726 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:29.920715 29188 leader_election.cc:336] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.615s (SENT)
I20260709 13:06:29.931063  2795 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.138s
I20260709 13:06:29.919705  3067 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.932848 29871 leader_election.cc:400] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16.
I20260709 13:06:29.933312 29871 leader_election.cc:403] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:29.777936  2850 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.143s
I20260709 13:06:29.934957 29142 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:29.934983 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:29.694003  2999 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:29.878790  2939 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:29.938235 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:29.966966 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.968219 29142 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:29.973124 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:29.977730 29908 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:29.984793 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:29.941568  2613 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:29.950745  2939 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.993845 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.971248  2885 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:29.974625  2604 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:29.974123  3088 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:30.014132 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 93445us
W20260709 13:06:30.022013 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 273204us
I20260709 13:06:30.026242 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:30.027524 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:29.949895  3008 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:06:30.000464  2939 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:30.033047 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.036738 29112 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:30.043982 29902 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:06:30.055796 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 185088us
I20260709 13:06:30.031363  3060 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:30.053345  2951 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:06:30.065618 29188 leader_election.cc:336] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.888s (SENT)
I20260709 13:06:30.066653 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:29.995478  2446 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:06:30.079649 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.082516 29905 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
W20260709 13:06:30.096067 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.035173  2613 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:30.096555  3027 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:30.125406 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.130882  3098 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.131632  3098 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:30.134773  3104 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:30.126606 29188 leader_election.cc:336] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.026s (SENT)
W20260709 13:06:30.135497 29039 leader_election.cc:336] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.672s (SENT)
I20260709 13:06:30.135848 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:30.135936 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:30.136595  3081 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 13:06:30.137957  3082 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
W20260709 13:06:30.138170 29039 leader_election.cc:336] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.739s (SENT)
I20260709 13:06:30.138690 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:30.140379  2804 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:30.140620  3086 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:06:30.141208  2804 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:30.142515 29039 leader_election.cc:336] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.798s (SENT)
I20260709 13:06:30.143416  3097 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:30.142951 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:30.081386  2951 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:06:30.144487  2885 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:30.151458 29142 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:30.158143 29215 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:30.076514  3088 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:30.167716 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 295479us
W20260709 13:06:30.174443  3092 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.120s	user 0.001s	sys 0.001s
W20260709 13:06:30.174546 29039 leader_election.cc:336] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.883s (SENT)
I20260709 13:06:30.175204 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:30.175562  2879 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.000s
I20260709 13:06:30.176359  3109 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:06:30.176645 29039 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.902s (SENT)
I20260709 13:06:30.176905  3109 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:30.178018  2885 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:06:30.179560 29039 leader_election.cc:336] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.011s (SENT)
I20260709 13:06:30.169456  3049 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:30.180943 29039 leader_election.cc:336] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.162s (SENT)
I20260709 13:06:30.180663  3049 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:30.160049  3093 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.181859  3093 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:30.188547 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:30.176040  2879 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:30.183763 29039 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:30.206355 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.205188 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:30.207484 29215 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:06:30.211084 29143 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.004s
I20260709 13:06:30.211836 29143 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
W20260709 13:06:30.217990 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 163203us
I20260709 13:06:30.237505 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.238263 29141 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:30.239745 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.240653 29139 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:30.244289 29217 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:06:30.245057 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:30.239122 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:30.248312 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:30.249402 29143 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:30.246802 29218 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:30.252314 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:30.211342  3055 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16.
I20260709 13:06:30.255951 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.276597 29143 raft_consensus.cc:2368] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18.
I20260709 13:06:30.236225  2959 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:30.257912 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:30.258682 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:30.260185 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.263026 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.265228  2879 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:30.280140 29144 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:30.283430 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:30.284170 29221 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:30.285130 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:30.266701  2441 consensus_meta.cc:287] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.552s	user 0.007s	sys 0.004s
I20260709 13:06:30.267666 29219 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:30.267699 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:30.272593 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:30.273275  3109 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:30.274612 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:30.210956  3067 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:30.236078  2885 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:30.280045 29141 raft_consensus.cc:2376] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15. Re-sending same reply.
I20260709 13:06:30.280140 29146 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:30.281991 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.265331 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:30.289492 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.289336 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.298471 29139 raft_consensus.cc:2410] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:06:30.276160  2604 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:30.312960 29144 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:06:30.297467 29148 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:30.277453  3121 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:30.318341 29140 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:06:30.293372  3027 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:06:30.322781 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 141916us
I20260709 13:06:30.324599  3105 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:30.325769  3027 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.326416  3027 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:30.329617  3088 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:30.332126  3121 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:30.334832 29146 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:30.286535  3094 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:30.337743  3041 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
W20260709 13:06:30.371749 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 439090us
W20260709 13:06:30.389427 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 57525.4us
I20260709 13:06:30.318131  2879 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:06:30.372430 29039 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.615s (SENT)
I20260709 13:06:30.389482 29215 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:30.291945  2441 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:30.381989 29142 raft_consensus.cc:1217] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 7.4->[9.5-20.10]   Dedup: 14.7->[16.8-20.10]
W20260709 13:06:30.364248 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 97423.7us
I20260709 13:06:30.382908  2712 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:30.394857 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:30.388856 29139 pending_rounds.cc:85] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 7
I20260709 13:06:30.403390 29139 pending_rounds.cc:107] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:06:30.403698 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:30.398384 29142 pending_rounds.cc:85] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 7
I20260709 13:06:30.403815 29139 raft_consensus.cc:2889] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:30.405565 29142 pending_rounds.cc:107] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 17.8
I20260709 13:06:30.406494 29142 raft_consensus.cc:2889] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:30.390930  3109 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:30.392499 29221 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
W20260709 13:06:30.410465 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 136093us
I20260709 13:06:30.414999  3108 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.415179  2939 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:30.417071  2999 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:30.397332  3101 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:30.423935  3067 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.419449 29871 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:30.435302  3098 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:30.436725 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:30.443570 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.403093  2712 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:30.431806 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:30.405992  2879 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:30.462795 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:30.468748 29218 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:30.440490  3101 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:30.470405 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.431489  3055 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:30.482023 29148 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:30.489504 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70145.3us
W20260709 13:06:30.500732 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 61783.5us
I20260709 13:06:30.420123  2795 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:06:30.487454 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.509177 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:30.518235 29909 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:30.509825  3042 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:30.458038  2613 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:30.442170  2885 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:06:30.415688  3108 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:30.519577 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:30.520017 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:30.525308 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.525966 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.548791  2613 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:30.568951  3129 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:30.571247  3101 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:30.571610  3123 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:06:30.572151  2712 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:30.549101  3043 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:30.573452 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.559108  3049 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:30.574117 29146 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:30.575069 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.574709  2879 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:30.542429  3041 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:06:30.577565 29871 leader_election.cc:400] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 13 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 13. Current term is 14.
W20260709 13:06:30.577812  3128 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:30.579070 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.598527 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.611234 29147 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:30.612313 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.480603  3067 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:30.612962 29148 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:30.558426  3097 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:30.621285 29144 raft_consensus.cc:2368] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18.
I20260709 13:06:30.560602  3098 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:30.651575 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 74202.7us
W20260709 13:06:30.656325 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 136805us
I20260709 13:06:30.658016 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:30.658259 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.659047 29145 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:30.659049 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.628180 29909 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:30.659969 29146 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:30.600769  2613 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:30.578668  3121 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:30.666843 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.614600  2951 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:30.670589 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:30.675783 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 246164us
I20260709 13:06:30.679126 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:30.627445  3043 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:30.680534 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:30.618007  2885 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:30.684876 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.670771  2795 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:06:30.702042 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 223356us
I20260709 13:06:30.618355  3027 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:30.668102  2446 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 13:06:30.727979 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70052.6us
I20260709 13:06:30.686172  2885 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:06:30.730895 29188 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.190s (SENT)
I20260709 13:06:30.731591 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.732571 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:30.680545  2951 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:30.812345 29144 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:30.806221 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:30.769758 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:30.816779 29188 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:30.708031  3110 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:30.828115  3122 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.828595 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.828754  3122 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:30.829383 29148 raft_consensus.cc:2368] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20.
I20260709 13:06:30.829906 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.833123 29144 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
W20260709 13:06:30.835656 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 159106us
I20260709 13:06:30.836014 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:30.836607 29216 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:30.839025 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:30.839895 29221 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:30.832139 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:30.860625 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:30.806159  2939 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:30.852321  3125 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:06:30.865733  3101 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:30.867111  3132 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:06:30.867766 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:30.869967  3088 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.870779  3122 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:30.871485 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:30.872550 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:30.892897 29070 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:30.877509  2441 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:30.885915  3137 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.887127 29067 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:30.685972  3043 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:30.889524 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:30.919553 29065 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:30.869428  3121 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:30.902050 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:30.922164 29215 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:30.922693 29217 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:30.897584 29072 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:30.728113  3017 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.127s
I20260709 13:06:30.897493 29066 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:30.888116 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:30.890583 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:30.930249 29064 raft_consensus.cc:2376] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17. Re-sending same reply.
I20260709 13:06:30.930475 29146 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:30.855455  3110 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:30.921986  3121 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:30.954113 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 228510us
I20260709 13:06:30.932967  2939 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:30.954711 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 576847us
W20260709 13:06:30.964383 29039 leader_election.cc:336] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.628s (SENT)
I20260709 13:06:30.965739  3027 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:30.969913 29221 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
W20260709 13:06:30.976694  3136 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:06:30.976500 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:30.984504 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:30.987001 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.014560 29066 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:31.015137 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:31.015434 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:31.015864 29067 raft_consensus.cc:2368] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17.
I20260709 13:06:30.942080  2441 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:31.017438  2441 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:31.019084 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.016029 29066 raft_consensus.cc:2376] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19. Re-sending same reply.
I20260709 13:06:30.999284  3108 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:30.911800  3088 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:31.020615 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56277.5us
W20260709 13:06:31.021663 29039 leader_election.cc:336] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.818s (SENT)
I20260709 13:06:31.022082 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:31.022563  3088 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:31.000923 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:31.003851  3067 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:31.004698  3043 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:30.961081  3121 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:30.921676  3137 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:31.026688 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 504693us
W20260709 13:06:31.000650  3092 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:30.912580  2613 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:30.956801  3098 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 13:06:31.043949 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:30.994735  2939 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:30.955811  2885 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:31.070616 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:31.072238 29070 raft_consensus.cc:2376] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17. Re-sending same reply.
I20260709 13:06:31.080466 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:31.085058 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63430.7us
I20260709 13:06:31.083412 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:31.089807 29905 raft_consensus.cc:2376] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16. Re-sending same reply.
I20260709 13:06:31.101502 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:31.101931 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:31.107034 29902 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:31.076104  3125 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:31.105718 29907 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:31.111723 29148 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:31.093159  2951 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:31.127156 29039 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.740s (SENT)
I20260709 13:06:31.138293 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:31.093580  2885 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:31.113890  3134 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:06:31.099503 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:31.124127  3125 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:31.156454 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 57101.7us
W20260709 13:06:31.160856 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 347340us
I20260709 13:06:31.144312  3125 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:31.172503 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45387.1us
I20260709 13:06:31.173728 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:31.171170 29902 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:31.178076 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:31.182060 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:31.188148 29039 leader_election.cc:336] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.991s (SENT)
I20260709 13:06:31.189755 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:31.191190 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:31.196116 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:31.190551 29144 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:31.192014 29146 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:31.171669  3102 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:31.198455 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:31.200142 29112 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:31.186439  3055 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:31.199216  3109 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.755s	user 0.007s	sys 0.007s
I20260709 13:06:31.208729 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:31.220016 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:31.198350  3067 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:31.204092  3137 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:31.207780  2951 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:31.217504  3105 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:31.226361  3067 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:31.205639  3056 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:06:31.237074  2879 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:31.238832  3105 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.223361  3055 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:31.403059 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 373877us
W20260709 13:06:31.403949 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 233571us
I20260709 13:06:31.269485  2951 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:31.409153 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.370507  3067 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:31.406886  3105 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:31.413548  3143 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:31.356662  2939 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:31.417634 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 200250us
W20260709 13:06:31.411194  3136 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:06:31.418612 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 198724us
I20260709 13:06:31.423038 29871 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:31.419206 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:31.414476  3148 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 13:06:31.446139  3137 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:31.447069  3137 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:31.450516  3137 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:31.420471  3145 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
I20260709 13:06:31.462817 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:31.408145  2999 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
W20260709 13:06:31.469134 29216 consensus_meta.cc:287] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.610s	user 0.008s	sys 0.004s
I20260709 13:06:31.048484  3110 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.473870 29216 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:31.460392  3041 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:31.471297  3141 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.326s	user 0.000s	sys 0.001s
W20260709 13:06:31.476712  3146 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
W20260709 13:06:31.495105  3144 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.316s	user 0.001s	sys 0.000s
W20260709 13:06:31.507411 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 325864us
I20260709 13:06:31.486255  3042 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:06:31.515636 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47710.6us
W20260709 13:06:31.510809 29039 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.282s (SENT)
I20260709 13:06:31.515447  3042 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:31.509686  3140 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:31.539258 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2750) took 1514 ms. Trace:
I20260709 13:06:31.540364 29141 rpcz_store.cc:276] 0709 13:06:30.025298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.297281 (+271983us) service_pool.cc:224] Handling call
0709 13:06:30.319567 (+ 22286us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:30.625793 (+306226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.625827 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:30.661607 (+ 35780us) log.cc:844] Serialized 52 byte log entry
0709 13:06:30.926140 (+264533us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:31.233415 (+307275us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.239201 (+  5786us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.269617 (+ 30416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.269700 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.539046 (+269346us) raft_consensus.cc:1685] finished
0709 13:06:31.539065 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.539199 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":449602,"prepare.queue_time_us":70775,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":26395,"raft.queue_time_us":374089,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":13793,"thread_start_us":505844,"threads_started":2,"wal-append.queue_time_us":387656}
I20260709 13:06:31.545164 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.501091  3041 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.547118  3055 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:31.518725  3151 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:31.556202 29071 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:31.533869  3042 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:31.526943  3157 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:31.583052 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 72278.8us
W20260709 13:06:31.589807 29188 leader_election.cc:336] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.589s (SENT)
I20260709 13:06:31.590711 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:31.516044  2951 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.558755  3151 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:31.595296 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 105634us
I20260709 13:06:31.598721 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1058) took 1091 ms. Trace:
I20260709 13:06:31.599102 29219 rpcz_store.cc:276] 0709 13:06:30.506844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.507183 (+   339us) service_pool.cc:224] Handling call
0709 13:06:30.525322 (+ 18139us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.548538 (+ 23216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.548576 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.597687 (+ 49111us) log.cc:844] Serialized 26 byte log entry
0709 13:06:30.915399 (+317712us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:30.915431 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:30.915485 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.598521 (+683036us) raft_consensus.cc:1685] finished
0709 13:06:31.598544 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.598662 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31751,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":18948,"thread_start_us":220372,"threads_started":1,"wal-append.queue_time_us":460884}
W20260709 13:06:31.606194 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 89153.1us
I20260709 13:06:31.608237 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:31.608556 29039 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 3.045s (SENT)
I20260709 13:06:31.557667  3143 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:31.613205 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:31.606047  3152 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:06:31.598188  3109 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:31.598467  3151 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:31.624330  3155 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 13:06:31.628584  3158 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
W20260709 13:06:31.632622  3173 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:06:31.635529  3121 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.003s
I20260709 13:06:31.641011  3041 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:31.641997 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2759) took 1237 ms. Trace:
W20260709 13:06:31.642045  3116 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.255s	user 0.021s	sys 0.013s
I20260709 13:06:31.645102  3157 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:31.647742 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 43741.2us
W20260709 13:06:31.650883 29039 leader_election.cc:336] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.560s (SENT)
I20260709 13:06:31.651329 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:31.652153 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2757) took 1269 ms. Trace:
I20260709 13:06:31.616640  2613 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:31.652529 29140 rpcz_store.cc:276] 0709 13:06:30.382771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.384222 (+  1451us) service_pool.cc:224] Handling call
0709 13:06:30.396723 (+ 12501us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:30.671806 (+275083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.671840 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:30.680003 (+  8163us) log.cc:844] Serialized 78 byte log entry
0709 13:06:30.990019 (+310016us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:31.002240 (+ 12221us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.008316 (+  6076us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.008708 (+   392us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.009391 (+   683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.009484 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.651560 (+642076us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b180016fba0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783ba36aa1 00007f783ba2db37 00007f783ba2d9ca 00007f783bb2526f
0709 13:06:31.651899 (+   339us) raft_consensus.cc:1685] finished
0709 13:06:31.651950 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.652094 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":388225,"prepare.queue_time_us":123438,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":86,"raft.queue_time_us":269818,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":36439,"spinlock_wait_cycles":100639744,"thread_start_us":255922,"threads_started":1,"wal-append.queue_time_us":472170}
I20260709 13:06:31.642475 29143 rpcz_store.cc:276] 0709 13:06:30.404812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.425012 (+ 20200us) service_pool.cc:224] Handling call
0709 13:06:30.456096 (+ 31084us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:30.686418 (+230322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.686466 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:30.700980 (+ 14514us) log.cc:844] Serialized 26 byte log entry
0709 13:06:31.146253 (+445273us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:31.175163 (+ 28910us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.187506 (+ 12343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.187610 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.641727 (+454117us) raft_consensus.cc:1685] finished
0709 13:06:31.641752 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.641919 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":318692,"prepare.queue_time_us":31436,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":256340,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":34554,"thread_start_us":283345,"threads_started":1,"wal-append.queue_time_us":418726}
I20260709 13:06:31.641314  3108 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:06:31.641287  2446 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:06:31.658552  3121 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:31.659332  3167 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:31.676226  2712 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:31.677338  3171 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.678045  3171 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.673496  3179 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:31.688607 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.689275 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.689780 29069 raft_consensus.cc:2410] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:06:31.691256  3168 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.691879  3168 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.692653  3143 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.694996  3093 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:31.680042  3043 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:31.697346  3147 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.491s	user 0.001s	sys 0.000s
I20260709 13:06:31.700065 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2754) took 1419 ms. Trace:
I20260709 13:06:31.700440 29139 rpcz_store.cc:276] 0709 13:06:30.284892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.313111 (+ 28219us) service_pool.cc:224] Handling call
0709 13:06:30.338968 (+ 25857us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:30.675764 (+336796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.675805 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:30.679782 (+  3977us) log.cc:844] Serialized 52 byte log entry
0709 13:06:31.125561 (+445779us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:31.125617 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.137787 (+ 12170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.696686 (+558899us) spinlock_profiling.cc:243] Waited 8.6 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:31.699734 (+  3048us) raft_consensus.cc:1685] finished
0709 13:06:31.699755 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.699995 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":242210,"raft.queue_time_us":265735,"spinlock_wait_cycles":23374592,"thread_start_us":354386,"threads_started":1,"wal-append.queue_time_us":489178}
I20260709 13:06:31.701393 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.702075 29064 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:31.703399 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:31.704090  3125 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.519s	user 0.006s	sys 0.006s
I20260709 13:06:31.704452 29072 raft_consensus.cc:2410] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:06:31.704639  3125 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.709082 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:31.709584  3129 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:31.710022  3129 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:31.718745 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:31.687480  2885 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.544s	user 0.005s	sys 0.008s
I20260709 13:06:31.716464  3110 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:31.726675 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:31.727134  3097 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:31.727524 29904 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:31.728664 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.684072  3121 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:31.732892 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 82034.9us
W20260709 13:06:31.733420 29039 leader_election.cc:336] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.667s (SENT)
I20260709 13:06:31.733659 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:31.737197 29037 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:31.740466 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:31.741211 29139 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:31.745071 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:31.749096 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:31.750923 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:31.688958  3109 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:06:31.720316  2446 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:31.763962 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.759202 29069 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:31.759717 29904 raft_consensus.cc:2376] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16. Re-sending same reply.
I20260709 13:06:31.663452  3157 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.756156  3129 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:31.776302  3179 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:31.777263  3179 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:31.782023 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:31.784703 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 51294us
I20260709 13:06:31.783746 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.797960 29909 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:06:31.755505  2885 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:31.800038 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:31.801105 29039 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.735s (SENT)
I20260709 13:06:31.801934 29221 raft_consensus.cc:2410] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:06:31.803442 29064 raft_consensus.cc:2410] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:06:31.801966 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.809962 29071 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:31.814123  3137 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:31.814730  3137 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:31.816994  3137 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:31.824791 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:31.825697 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.827497 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.833271  3157 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:31.834743  3042 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.836174 29069 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:31.837289 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.837368 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.838333 29067 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:31.837564  3042 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:31.838699 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.839597 29064 raft_consensus.cc:2410] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:06:31.842146 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 1038) took 1318 ms. Trace:
I20260709 13:06:31.842516 29908 rpcz_store.cc:276] 0709 13:06:30.523421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.524182 (+   761us) service_pool.cc:224] Handling call
0709 13:06:30.561877 (+ 37695us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:30.651653 (+ 89776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:30.651695 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:30.677727 (+ 26032us) log.cc:844] Serialized 52 byte log entry
0709 13:06:30.918196 (+240469us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:06:31.220932 (+302736us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.841676 (+620744us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b54004b58a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a88733 00007f7836b0ddbf 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c
0709 13:06:31.841725 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:31.841801 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:31.841839 (+    38us) raft_consensus.cc:1685] finished
0709 13:06:31.841931 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:31.842066 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15020,"prepare.queue_time_us":155005,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":164517,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":45575,"spinlock_wait_cycles":47139584,"thread_start_us":839620,"threads_started":3,"wal-append.queue_time_us":394575}
I20260709 13:06:31.834712  3151 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.848090  3151 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:31.825809 29218 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:31.853951 29139 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:31.769680  2613 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:31.857477 29185 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:31.862901 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:31.862680 29188 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.003s (SENT)
W20260709 13:06:31.863853 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.842949  3179 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.866554 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:31.866247 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68199.8us
I20260709 13:06:31.867221 29069 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
W20260709 13:06:31.865828  3187 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:06:31.867486 29039 leader_election.cc:336] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.933s (SENT)
I20260709 13:06:31.869014 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:31.869884 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:31.870744 29065 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:06:31.802700  2885 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:31.872874  2885 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:31.876461  3192 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:31.876948  3067 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:31.879482 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:31.879412 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:31.879807  3171 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:31.882090 29905 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:31.883486 29904 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:31.848830  3114 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:06:31.882119  3172 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:31.799110  2446 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:31.886898  3179 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:31.887501  2446 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:31.885612  3172 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:31.888362 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:31.889132  3125 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:31.889189  3172 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.877063  3192 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:31.894678  3192 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:31.865845 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:31.911084 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:31.913059  2446 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:06:31.923713  2446 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:31.856503  3143 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:31.895416  3105 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:31.879513  3067 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:31.934849 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:31.939990 29215 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:06:31.911485  3121 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:31.944411 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:31.944950  3156 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:31.945225 29221 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:31.945595  3156 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:31.948060  3131 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:31.953014 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 95624.5us
I20260709 13:06:31.954722 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:31.943044 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:31.895689  3168 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:31.965840 29139 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:31.927990  3129 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:31.924192  2446 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:31.964638 29071 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:31.970283 29215 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:31.918807  3192 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
W20260709 13:06:31.979578 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 112138us
I20260709 13:06:31.979766 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:31.981180 29067 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:31.965080  2939 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:31.987249 29185 leader_election.cc:400] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17.
I20260709 13:06:31.964062  3156 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:31.968456  3121 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:32.001343  2441 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:32.005218 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:31.936256  3143 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:32.010131 29039 leader_election.cc:336] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.088s (SENT)
I20260709 13:06:32.006423 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:32.022603  3143 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:31.989145  3108 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:32.027737 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:32.028818 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:32.031635 29073 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:32.037672 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:32.038398 29072 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:32.040585  3097 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:32.041325  3097 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:32.042392  3097 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:31.979100  3114 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:06:32.007910  2939 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:32.007861  3121 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:32.045703 29039 leader_election.cc:336] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.196s (SENT)
W20260709 13:06:32.053697 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:32.042618  2446 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.056224 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:32.028365  3108 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:32.068302 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:32.071791  3157 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:32.054997  3121 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:32.086582 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:32.089838 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:32.094733 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:32.096463 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:32.097431 29073 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:32.099632 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:32.098446 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:32.100756 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:32.101905 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:32.104501 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:32.105892 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:32.107239 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:32.109328 29222 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:32.083545  3151 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.113857 29218 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:06:32.114617  3198 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 13:06:32.115700  3194 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:06:32.116183 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:32.117745  3108 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.118903  3108 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:32.120062  3157 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:32.122404 29071 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:06:32.122793 29071 pending_rounds.cc:85] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 9
I20260709 13:06:32.123003 29071 pending_rounds.cc:107] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:06:32.123224 29071 raft_consensus.cc:2889] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:32.124501 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:32.125041 29071 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:32.105041  2446 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:32.126439 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:32.128775  3098 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:32.130152 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:32.130326  3043 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:32.131446 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:32.132491 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:32.136656  2939 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:32.138916 29221 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:06:32.136982 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:32.145222 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:32.149736 29067 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:06:32.146373 29213 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:32.151783 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:32.152581 29073 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:32.153940 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:32.156082  3131 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:32.157922 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:32.158510  3157 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:32.159354 29070 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 2747) took 1185 ms (client timeout 1576 ms). Trace:
I20260709 13:06:32.159603  3156 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:32.159978 29070 rpcz_store.cc:269] 0709 13:06:30.973539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.976088 (+  2549us) service_pool.cc:224] Handling call
0709 13:06:32.159154 (+1183066us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b5400129654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:32.159270 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133546752}
I20260709 13:06:32.165514 29071 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:06:32.169095 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:32.172423 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:32.173501 29071 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:32.185144 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:32.185276 29217 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:32.168471  3105 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:32.188989 29067 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:32.149808  3151 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.170409  3156 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:06:32.170536  3157 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:06:32.298017 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:32.244585 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:32.248600 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.189328  3168 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.163s
I20260709 13:06:32.352000 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:32.352310  2885 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.354000 29139 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:32.355209 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:32.355825 29219 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:32.357136 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.358389 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.359129 29145 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:32.324819 29072 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:32.359872 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.362762 29145 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:32.363528 29219 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:32.364020 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.248616 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.365051 29145 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:32.270687 29221 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:32.365442 29147 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:32.366744 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:32.238564 29039 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.354s (SENT)
I20260709 13:06:32.265625 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.368247 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:32.216962 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:32.243479 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:32.370733 29144 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:32.199261 29073 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:32.248769 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.372982 29143 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:32.364960 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:32.283146 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:32.243371 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:32.227728 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 159541us
I20260709 13:06:32.365772 29148 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:32.441183 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 76312.5us
I20260709 13:06:32.442448 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:32.462621 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:32.463277 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:32.230063 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 180803us
I20260709 13:06:32.465785 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:32.292063 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.368175 29145 raft_consensus.cc:2376] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14. Re-sending same reply.
W20260709 13:06:32.236931 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 148886us
I20260709 13:06:32.463310 29067 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:32.473414 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.474874 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.480581 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:32.480922 29072 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:32.481648 29141 raft_consensus.cc:2368] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20.
I20260709 13:06:32.482386 29072 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:06:32.485013 29069 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:32.373983  2885 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:32.506703 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:32.508469 29221 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:32.293329  3167 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:32.497048 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 280166us
I20260709 13:06:32.517110 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:32.519531 29071 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:06:32.476719  3150 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:32.466715  2613 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:32.487763 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:32.216544  2712 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.466377  3140 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:32.497939 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:32.498172 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:32.550776 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52902us
I20260709 13:06:32.531354 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:32.483219  3192 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:32.554872 29188 leader_election.cc:400] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20.
I20260709 13:06:32.555348 29188 leader_election.cc:403] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:32.556249 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:32.504993  3055 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:32.441686  3043 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.139s
I20260709 13:06:32.553424 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:32.543787 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 305234us
W20260709 13:06:32.568459 29039 leader_election.cc:336] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.369s (SENT)
I20260709 13:06:32.567528 29871 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:32.554580  2712 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.570518  3109 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20.
I20260709 13:06:32.571696  3109 raft_consensus.cc:2764] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:06:32.572775  3172 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:32.573608  3172 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:32.569038 29039 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:32.576540  3041 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:32.577761  3041 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:32.577831 29871 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:32.578406  3041 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.578958  3041 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:32.580135  3088 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:32.580744  3088 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.581228  3088 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:32.561582  3208 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:32.601514  3208 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:32.602062  3208 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:32.574893  3172 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.549082  2999 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:32.559095  3150 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:06:32.629325  3121 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.000s
W20260709 13:06:32.641346  3098 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.000s
I20260709 13:06:32.626573  3140 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:32.552234  2613 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:32.828384 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 254572us
I20260709 13:06:32.534526  3167 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:32.841142 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:32.843255 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 274840us
I20260709 13:06:32.736182  3121 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.834061  3168 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:32.868760 29215 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:32.868652  3168 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.870146  3121 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:32.871311  3168 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:32.832624 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 253057us
I20260709 13:06:32.875552 29871 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:32.879526  3055 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:32.880175  3055 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:32.881143  3055 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.830715  3211 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.886615  3211 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.890354  3211 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:32.832113  2999 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:32.891571  2999 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:32.895368  3217 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:32.851543  3125 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.852147  2613 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:32.854096 29066 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:32.856380  3167 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:32.829771  3157 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:32.841989  2446 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.715s	user 0.008s	sys 0.008s
I20260709 13:06:32.906888  3157 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:32.860518  3108 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.741s	user 0.013s	sys 0.006s
I20260709 13:06:32.908998 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:32.910338  3157 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:32.911592  3151 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:32.862586 29067 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:32.807863  3098 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:32.901592 29144 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:32.901882 29072 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:32.906278 29148 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:32.896751  3125 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.924327 29143 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:32.928264 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:32.928951 29908 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:06:32.915024  3088 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:32.956171  3088 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:32.965756 29218 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:06:32.982491 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 909) took 1022 ms. Trace:
I20260709 13:06:32.948338  3223 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:06:32.983072 29068 rpcz_store.cc:276] 0709 13:06:31.960358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.216555 (+256197us) service_pool.cc:224] Handling call
0709 13:06:32.216901 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:32.217296 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.217326 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:32.217876 (+   550us) log.cc:844] Serialized 52 byte log entry
0709 13:06:32.543046 (+325170us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:32.850758 (+307712us) log.cc:844] Serialized 14 byte log entry
0709 13:06:32.982118 (+131360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.982201 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:32.982245 (+    44us) raft_consensus.cc:1685] finished
0709 13:06:32.982258 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:32.982425 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":340787,"prepare.queue_time_us":307757,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":42317,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":4570,"thread_start_us":403677,"threads_started":2,"wal-append.queue_time_us":673992}
I20260709 13:06:32.990985 29214 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:32.947000  3150 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:33.018793 29039 leader_election.cc:336] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.735s (SENT)
I20260709 13:06:33.019555 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:33.108130 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 89396.3us
W20260709 13:06:33.105020 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50932.2us
W20260709 13:06:33.184405 29188 leader_election.cc:336] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.668s (SENT)
I20260709 13:06:33.185127 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:33.128012  3208 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.525s	user 0.013s	sys 0.000s
W20260709 13:06:33.203954  3224 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 13:06:33.101881  3157 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:33.223351  3167 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:33.224048  2613 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:33.198450  3098 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:33.109640  3125 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.240834  3228 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:33.237422  3208 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:33.308648 29039 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.631s (SENT)
I20260709 13:06:33.309551 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:33.324177  3220 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:33.264635  3167 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.259395  2613 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.338786  3167 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:33.357126  3041 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.778s	user 0.013s	sys 0.002s
I20260709 13:06:33.323683  3228 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.386605  3172 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:33.401919 29908 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:33.388267  3041 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:33.411254 29185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.111s	sys 0.280s
W20260709 13:06:33.411690 29185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.111s	sys 0.280s
W20260709 13:06:33.412060 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 922184us
I20260709 13:06:33.409484  3208 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:33.403548  3172 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.429062 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:33.434118 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 111662us
W20260709 13:06:33.436590 29039 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.708s (SENT)
I20260709 13:06:33.436414  3220 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:33.437498  3220 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.430099  3167 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:33.450445  3228 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.452167  3167 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.441488 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:33.435971  3227 raft_consensus.cc:2764] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:06:33.440035  2613 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.444260  3220 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.445958  3172 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.447109  3041 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.456759  3228 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:33.453939  3218 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.850s	user 0.004s	sys 0.008s
I20260709 13:06:33.458798 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2755) took 3170 ms. Trace:
I20260709 13:06:33.462599  3180 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:33.463352  2959 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.464380  3203 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:33.464516 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:33.469085  3167 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.469012 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:33.469878  3112 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:33.471555  3027 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:33.471967 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:33.474278 29039 leader_election.cc:336] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.821s (SENT)
W20260709 13:06:33.485849 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.486239 29039 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:33.486613 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:33.480146 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 51179.7us
I20260709 13:06:33.487761 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:33.479172 29142 rpcz_store.cc:276] 0709 13:06:30.288365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.319796 (+ 31431us) service_pool.cc:224] Handling call
0709 13:06:30.352459 (+ 32663us) raft_consensus.cc:1471] Updating replica for 6 ops
0709 13:06:30.806373 (+453914us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:06:30.895780 (+ 89407us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.895618 (+999838us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b1800033640. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a 00007f7836b17a2a
0709 13:06:31.981418 (+ 85800us) log.cc:844] Serialized 14 byte log entry
0709 13:06:31.985074 (+  3656us) log.cc:844] Serialized 14 byte log entry
0709 13:06:32.055612 (+ 70538us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:32.059476 (+  3864us) log.cc:844] Serialized 78 byte log entry
0709 13:06:32.061557 (+  2081us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:32.062380 (+   823us) log.cc:844] Serialized 14 byte log entry
0709 13:06:32.202319 (+139939us) spinlock_profiling.cc:243] Waited 8.69 ms on lock 0x7b1800033640. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f783ba36328 00007f783ba2d901 00007f783ba2d8a7 00007f783bb25288 00007f783bb24f37 00007f783bb3955c 00007f783bb39508
0709 13:06:32.220618 (+ 18299us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b1800033640. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:06:32.221013 (+   395us) log.cc:844] Serialized 14 byte log entry
0709 13:06:32.221363 (+   350us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.454343 (+1232980us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b1800033640. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f783a9e158f 00007f7833a20571 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd
0709 13:06:33.458439 (+  4096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.458515 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.458556 (+    41us) raft_consensus.cc:1685] finished
0709 13:06:33.458570 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.458706 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3514889,"prepare.queue_time_us":175864,"prepare.run_cpu_time_us":976,"prepare.run_wall_time_us":5005,"raft.queue_time_us":1835361,"raft.run_cpu_time_us":10270,"raft.run_wall_time_us":604121,"spinlock_wait_cycles":48841600,"thread_start_us":140234,"threads_started":3,"wal-append.queue_time_us":1089210,"wal-append.run_cpu_time_us":6150,"wal-append.run_wall_time_us":1330672}
I20260709 13:06:33.499295 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.500191 29222 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:33.509804 29908 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:33.503271 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:33.510772 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:33.511646 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.512322 29216 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:33.512951 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 916) took 1551 ms. Trace:
I20260709 13:06:33.513334 29065 rpcz_store.cc:276] 0709 13:06:31.961957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.354353 (+392396us) service_pool.cc:224] Handling call
0709 13:06:32.354722 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:32.355127 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:32.355163 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:32.355716 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:06:32.465010 (+109294us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:32.465073 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:32.465175 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.453260 (+988085us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:33.506578 (+ 53318us) raft_consensus.cc:1685] finished
0709 13:06:33.506609 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.512888 (+  6279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1814,"raft.queue_time_us":13660,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":87769,"spinlock_wait_cycles":27452672,"thread_start_us":99477,"threads_started":1,"wal-append.queue_time_us":149410}
I20260709 13:06:33.516690 29904 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:33.505903  2949 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:33.518656 29905 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:33.520293 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.520925 29221 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:33.522723  3110 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.505175  3222 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:33.523375  3222 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.523952  3142 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:33.524484  3142 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:33.525594  3222 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:33.525363  3142 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.506088  3124 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:33.487573  3203 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.535530  3203 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:33.489001  3112 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.511219  3097 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:33.541555  3097 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:33.485687  3180 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.542564  3097 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.523285  3110 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.488067  2959 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.546255  2959 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:33.546383  3110 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:33.548418  3112 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:33.549809  3180 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.553432  3228 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:33.554853  2949 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:33.554802  3228 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.555881  2949 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.558205  3228 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.562475 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:33.562366 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:33.564426 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:33.565274 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:33.565982 29902 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:33.566808 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:33.567569 29911 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:33.570185 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.571004 29221 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:33.575479  2712 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.581336 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.582284 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2818) took 1333 ms. Trace:
I20260709 13:06:33.582361 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:33.584453 29216 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:33.497951  3167 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:33.582893  3221 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:33.584570 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 1073) took 1460 ms. Trace:
I20260709 13:06:33.584903 29221 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:33.590175 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:33.584340 29066 rpcz_store.cc:276] 0709 13:06:32.248515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.263390 (+ 14875us) service_pool.cc:224] Handling call
0709 13:06:33.582123 (+1318733us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b5400208ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:33.582226 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1178994560}
I20260709 13:06:33.590816 29148 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:33.466696 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:33.589953 29218 rpcz_store.cc:276] 0709 13:06:32.124566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.136605 (+ 12039us) service_pool.cc:224] Handling call
0709 13:06:33.584361 (+1447756us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b540016c454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:33.584502 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1769390976}
W20260709 13:06:33.587564 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 76877.7us
I20260709 13:06:33.588789 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2913) took 1343 ms. Trace:
I20260709 13:06:33.592986 29143 rpcz_store.cc:276] 0709 13:06:32.245487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.482995 (+237508us) service_pool.cc:224] Handling call
0709 13:06:32.918510 (+435515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:32.924765 (+  6255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.588610 (+663845us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54001bc1d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:33.588731 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935464192}
I20260709 13:06:33.596484 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:33.600898 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:33.604209 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:33.588672 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:33.614003 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.614940 29904 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:33.614027 29143 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:33.616727 29216 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:06:33.618809 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.497372  3027 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.621178 29218 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:06:33.622341 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.625694 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:33.626348 29143 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:33.611327 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:33.628402 29039 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.930s (SENT)
I20260709 13:06:33.628863 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:33.630330 29221 raft_consensus.cc:2410] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:06:33.503908  3041 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:33.489022  2441 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.606096 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:33.632498 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:33.636708 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:33.684374 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 58475.7us
I20260709 13:06:33.689455 29219 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:33.693257 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.694257 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.697224 29908 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:33.695127 29213 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:33.693396  3204 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 13:06:33.605376  3203 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:33.698058 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:33.700155 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 231270us
I20260709 13:06:33.690341  2999 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:33.686640  3167 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:33.699318  3041 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:33.728247  3041 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.728760  3041 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:06:33.730367 29214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 1071) took 1626 ms (client timeout 2011 ms). Trace:
W20260709 13:06:33.730939 29214 rpcz_store.cc:269] 0709 13:06:32.104420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.139772 (+ 35352us) service_pool.cc:224] Handling call
0709 13:06:33.730127 (+1590355us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b540033b3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:33.730284 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700157696}
I20260709 13:06:33.715772 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:33.622653  3027 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.622680  3125 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.096s
I20260709 13:06:33.733852  3125 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:33.734263  3125 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:33.613142  3221 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.737800  3221 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:33.709183  3203 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.738747  3203 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:33.637977  3180 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 13:06:33.735271  3125 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.747522  3208 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:33.687219  3112 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:06:33.687901  2441 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.716631  2999 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:33.689867  3119 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:06:33.697094  3110 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:06:33.757279 29143 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:33.697228  2959 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:06:33.769238 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.781873 29215 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:33.777987 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.768483 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.779675 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.784365  3110 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:33.784967 29218 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:33.785136  3110 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:33.785755  3110 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.783255 29222 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:33.794132  3043 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:33.794881  3043 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.796213  2804 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:33.801963 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:33.805826 29039 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.155s (SENT)
I20260709 13:06:33.806428 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:33.800588 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:33.810292  3203 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.812973 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.815007 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:33.815713 29221 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:33.819568  2959 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:33.820215  2959 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:33.776427  2999 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:33.822567  3203 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.826110 29221 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:33.828810  2441 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:33.830339 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:33.776083  3124 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:33.833250  3225 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.595s	user 0.000s	sys 0.011s
I20260709 13:06:33.758055  3208 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:33.838024  3156 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:33.838553  3156 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:33.838922  3156 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:33.840049  3027 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:33.801542  3125 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:33.848167  3109 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:33.849076  3109 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.796809  2804 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.859416 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:33.798589  3220 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:33.880017 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:33.881276 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 75499.9us
I20260709 13:06:33.860519  3208 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:33.878981  2879 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:33.883020  3112 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:33.884473  3124 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:33.887727  3109 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:33.928000  3055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.016s	sys 0.013s
W20260709 13:06:33.928735  3055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.017s	sys 0.013s
I20260709 13:06:33.929733 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:33.930784 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:33.931819 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:33.935670 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:33.938095 29902 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:33.856155  2959 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:33.943702 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 311347us
W20260709 13:06:33.944025 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 311635us
I20260709 13:06:33.884613  3220 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:33.918928  3112 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:33.969143 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 228799us
W20260709 13:06:33.971400 29188 leader_election.cc:336] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.800s (SENT)
I20260709 13:06:33.972084 29188 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:33.978788 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2917) took 1732 ms. Trace:
I20260709 13:06:33.926290  3043 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:33.925729  3110 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:33.925415  3124 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:33.979363 29139 rpcz_store.cc:276] 0709 13:06:32.246543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.486582 (+240039us) service_pool.cc:224] Handling call
0709 13:06:32.523823 (+ 37241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.470504 (+946681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:33.470534 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:33.471042 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:06:33.508559 (+ 37517us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:33.978066 (+469507us) log.cc:844] Serialized 14 byte log entry
0709 13:06:33.978398 (+   332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:33.978493 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:33.978544 (+    51us) raft_consensus.cc:1685] finished
0709 13:06:33.978561 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:33.978715 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":930716,"raft.queue_time_us":932924,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":325175,"threads_started":2,"wal-append.queue_time_us":149613}
W20260709 13:06:33.982131 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 284141us
I20260709 13:06:33.999083 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:33.973255  3180 consensus_queue.cc:1243] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:06:33.988531 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.989593 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:33.982776 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 481895us
W20260709 13:06:33.973420 29188 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.243s (SENT)
I20260709 13:06:33.971823  3112 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:33.968640  3168 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:34.005060 29112 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:34.012596 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:33.980473  3221 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:34.006497 29037 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:34.018908 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 43717.3us
W20260709 13:06:34.024003 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 313523us
I20260709 13:06:34.024856 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:34.027949 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 227435us
I20260709 13:06:34.029140 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:33.997287  2959 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:34.031265 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:34.033670 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44202.4us
I20260709 13:06:34.017382  3180 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:34.041800 29038 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:34.029069  3150 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:34.055980 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:33.998021  3055 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:06:34.060436 29067 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:06:34.038960  2613 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:34.072616 29215 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:34.072616 29071 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:06:34.039157  2441 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.077911 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:34.081050 29066 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:34.047506  3167 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:34.053668  3204 consensus_queue.cc:1243] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 22.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:06:34.047248  3140 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:34.090116  3140 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:34.092244 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:34.093044 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:34.089529  3238 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:06:34.056351  3228 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:34.095184 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 50225.7us
I20260709 13:06:34.099992 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:34.026366  3168 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:34.072690  2613 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:34.094920 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70148.7us
I20260709 13:06:34.056813  3240 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:34.112118  3240 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:34.112982  3240 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:34.114457  2613 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:34.066499  3150 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:34.047282  2885 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260709 13:06:34.100821 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.079814  3250 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:34.108104 29219 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:06:34.085887  3043 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:34.111275  3168 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.083330  3167 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:34.136583  3167 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:34.112152  3151 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:34.116168  3150 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:34.131822 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.126721  3250 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:34.148687 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 48800.7us
I20260709 13:06:34.130591  3043 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:34.139194  3151 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:34.198129 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 80388.5us
I20260709 13:06:34.202838 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:34.133654  2804 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:34.212411 29215 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:06:34.213757 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:34.220381 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:34.195619  3257 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:06:34.217747 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:34.105753  3228 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:34.223835 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:34.224471 29213 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
W20260709 13:06:34.209443  3239 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.107s	user 0.005s	sys 0.005s
I20260709 13:06:34.227739 29221 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:34.151156  3250 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:34.207692  3043 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:34.233284  3041 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.000s
I20260709 13:06:34.233762  3041 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.214931  3151 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:34.147523  2999 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.148658  3180 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:06:34.223277  3110 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.095315  3124 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:34.235905  3041 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.237643  3220 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:34.239694  2613 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:34.289925  3098 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:34.244307  3223 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:06:34.249570 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:34.251803 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:34.290910  3156 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.255337 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.250476  3182 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.042s
W20260709 13:06:34.317566 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 62345.5us
I20260709 13:06:34.257825  3254 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:34.318797  3261 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:34.318602  3254 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:34.272878 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:34.274092 29909 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:06:34.319738 29072 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:34.294397 29906 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 13:06:34.257985  3246 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:06:34.321127  3254 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.238919 29069 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:06:34.322441 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:34.322577  3156 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:34.323099 29069 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:34.324088 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:34.324725 29069 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:34.324676  3125 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:06:34.325856 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:34.327234 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:34.259967 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:34.295298  2999 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.268864 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:34.331903  3240 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.269621 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:34.333178 29908 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:06:34.333277 29911 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:34.266716 29073 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:34.296857 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:34.340512 29213 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:34.339993  3112 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.226382  2885 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:06:34.287362  3167 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:34.343022  3112 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:34.344076  3167 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.237505  3228 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.304852  2804 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:34.306252 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2847) took 1692 ms. Trace:
I20260709 13:06:34.314293  3151 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.346517  2885 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:34.308239  3150 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.349733  3150 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:34.351744 29145 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:34.350359 29217 rpcz_store.cc:276] 0709 13:06:32.614013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:32.614966 (+   953us) service_pool.cc:224] Handling call
0709 13:06:32.653490 (+ 38524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:33.532280 (+878790us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:33.861433 (+329153us) log.cc:844] Serialized 14 byte log entry
0709 13:06:34.014881 (+153448us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:34.023549 (+  8668us) log.cc:844] Serialized 26 byte log entry
0709 13:06:34.023767 (+   218us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:34.024574 (+   807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:34.304994 (+280420us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b54001333d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:34.305053 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:34.305106 (+    53us) raft_consensus.cc:1685] finished
0709 13:06:34.306052 (+   946us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:34.306191 (+   139us) inbound_call.cc:177] Queueing success response
0709 13:06:34.308054 (+  1863us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b180026a820. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20522 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
Metrics: {"mutex_wait_us":12675,"prepare.queue_time_us":466479,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":10378,"raft.queue_time_us":23143,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":9607,"spinlock_wait_cycles":2045447168,"thread_start_us":421173,"threads_started":2,"wal-append.queue_time_us":224752}
I20260709 13:06:34.307960  3172 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.353147  3172 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.355412  3172 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.345067  2804 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:34.351238  3240 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.358201  2804 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.347782  3151 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.364039 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:34.366014 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.367635 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:34.374980 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:34.377696  2804 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:34.378335  2804 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:34.378943  2804 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:34.306690  3265 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:34.316144  3098 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.381856  3265 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.385331 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:34.318961 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:34.388445 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 69589us
I20260709 13:06:34.388697 29069 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: []
W20260709 13:06:34.264683 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 160073us
I20260709 13:06:34.390303 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.324194  2613 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.274443 29066 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:34.410528 29073 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
W20260709 13:06:34.428355 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 72962.5us
W20260709 13:06:34.436157 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 45977.2us
I20260709 13:06:34.440733 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:34.441941  3221 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:34.442790  3109 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:34.443233  3109 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:34.443645  3109 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:34.442216 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:34.454751 29871 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:34.456089  3240 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:34.456640  3240 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:34.457454  3240 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.459429  3150 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:34.459939  3150 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.461999  3150 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:34.463220  3172 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:06:34.465256 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:34.466414 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:34.468091 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:34.469056 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.470594 29067 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:06:34.355477  3043 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:34.454248  3266 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.189s	user 0.004s	sys 0.000s
I20260709 13:06:34.483788 29070 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:34.436439  3208 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.454658  3112 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.492300 29218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:34.493036 29218 raft_consensus.cc:606] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.493459 29218 raft_consensus.cc:614] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:34.476142  3222 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:34.482777  3124 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.508208  3222 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:34.531490  3222 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:34.535673  3041 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:06:34.536170 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 99001.2us
I20260709 13:06:34.536952 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.538111 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.539165 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.542373 29214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:34.542958  3245 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.543077 29214 raft_consensus.cc:606] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.543836  3245 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:34.544643  3245 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:34.547080 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:34.583451 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 113211us
I20260709 13:06:34.552943  3124 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:34.551254  3098 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:34.606132 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59078.3us
I20260709 13:06:34.607316 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.608961 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.610776  3265 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:34.613637  3112 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:34.622529  3250 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:34.624584  3270 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:06:34.624859  3250 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.627609 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:34.625064  3119 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:34.640686 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.643005 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.634977  3119 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:34.633445  3244 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:34.649133 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.652412  3244 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:34.644068  2949 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:34.663583  3222 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:34.668655 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.673732  3222 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:34.674153  3140 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.673590  2804 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:34.675979 29039 leader_election.cc:336] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.106s (SENT)
I20260709 13:06:34.676483 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:34.677217  3140 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:34.677558 29143 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:34.677992 29143 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:34.677750  3109 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:34.678582 29039 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.107s (SENT) [suppressed 33 similar messages]
I20260709 13:06:34.680264  3043 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:34.680593  3097 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:34.681326  3097 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:34.685679  3043 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:34.686957  3168 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:34.687311  3168 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:34.656129  3119 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:34.688418  3119 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:34.688141  3168 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.688522  2804 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:34.689154  3168 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:34.689556  3109 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:34.657060  3268 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:06:34.645395  2441 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:06:34.636650  3112 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:06:34.694563 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:34.658542  3252 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:06:34.656445  3271 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:06:34.648104  3275 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:06:34.722878 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:34.723778 29066 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:34.718078 29906 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:34.713150  3252 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:34.730623  3252 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:34.737612 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:34.739348 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:34.738267 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 95300.7us
I20260709 13:06:34.734138 29066 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:34.742856 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:34.744639 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:34.749065 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:34.749770 29065 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:34.749141 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:34.755749 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.757021 29144 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:34.749571 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:34.758538 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:34.759356 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:34.760351 29221 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:34.755949 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.759176 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:34.765362 29871 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:34.731874  3252 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:34.771446 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.772177 29139 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:34.772513 29218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:34.773097 29218 raft_consensus.cc:606] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.770392 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:34.773890 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.776677 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:34.781122 29142 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:34.781360 29218 raft_consensus.cc:614] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:34.767071  3275 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:34.762522  2441 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:34.729388  3112 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:06:34.798005 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:34.801313 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:34.803282 29217 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:34.806164 29221 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:34.764215  3140 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:34.785681 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:34.787206 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:34.779012 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 100421us
I20260709 13:06:34.807811 29139 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:34.799149  3125 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 13:06:34.817615 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:34.791688  2441 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:34.792318  3223 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:34.820431 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:34.820775 29218 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:06:34.794988  3275 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:34.823282  3140 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:34.829800 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:34.876005 29141 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:34.807075 29065 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:34.836054 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:34.840167 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:34.844483 29142 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:34.877542 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:34.871523 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.878655 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:34.881345 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.859967 29143 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:34.886318 29143 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:34.888298 29216 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:34.858713  2441 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:34.904134 29221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:34.904803 29221 raft_consensus.cc:606] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:34.901886 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:34.905174 29221 raft_consensus.cc:614] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:34.903056 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.913189 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:34.913738 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:34.914228 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:34.915868 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:34.916599 29073 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:34.925998 29073 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:34.927501 29143 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:06:34.929669 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.913992 29069 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
W20260709 13:06:34.931761 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:34.931839 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.935334 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.936677 29039 leader_election.cc:400] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20.
I20260709 13:06:34.937021 29039 leader_election.cc:403] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:06:34.936525 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.938120 29142 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:34.940477 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:34.943092 29141 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:34.944643 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:34.896211  3284 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:06:34.946486 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:34.946556 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.958019 29070 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:06:34.961424 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:34.891973  3223 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:34.964257 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:34.903959  3227 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:34.903429  3236 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:34.972762 29070 raft_consensus.cc:1217] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Deduplicated request from leader. Original: 20.15->[21.16-21.16]   Dedup: 21.16->[]
I20260709 13:06:34.976936 29070 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:06:34.974175 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.978428 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:34.946300  3228 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:34.948468  2951 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
W20260709 13:06:34.971257  3266 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:06:34.981551 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.936405  3142 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:34.982370 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:34.991151 29144 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:34.954208 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:34.991255  3142 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:34.992066 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:34.958952 29065 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:34.992519 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:34.980934 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 105094us
I20260709 13:06:34.985028 29220 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:34.984306 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:34.990856 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.993103 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:34.975335  2939 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20.
W20260709 13:06:35.032078 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47863.9us
I20260709 13:06:35.034675 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:35.009451 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:34.976487  3236 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:35.011413 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:35.065721 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 56358.8us
I20260709 13:06:34.978015  3097 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:06:35.089603 29215 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 21: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:34.984754  3208 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:34.992799  2885 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:35.112691 29904 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:06:35.058480 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:35.114123 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:35.105779 29906 raft_consensus.cc:1217] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Deduplicated request from leader. Original: 20.15->[21.16-21.16]   Dedup: 21.16->[]
I20260709 13:06:35.114880 29908 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 21: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:35.114952 29906 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
W20260709 13:06:35.112164 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 121376us
I20260709 13:06:35.106977 29909 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
W20260709 13:06:35.108516 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.109185 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:34.953034 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:35.022702 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 94332.6us
I20260709 13:06:35.008164 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:34.997556 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 120916us
I20260709 13:06:35.049916 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.111143 29911 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:06:34.984249  3227 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:35.050586  3261 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:35.021742  3180 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.140618 29065 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:35.141775 29143 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:35.143424 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.144291 29143 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
W20260709 13:06:35.144724 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.146973 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:35.148463 29213 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:06:35.151319 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.115886 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:35.135385 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:35.155053 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:35.156484 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.148195 29144 raft_consensus.cc:2376] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15. Re-sending same reply.
I20260709 13:06:35.158802 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:35.159654 29144 raft_consensus.cc:2376] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19. Re-sending same reply.
I20260709 13:06:35.161964 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.160413 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.164006 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.089763  3157 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:35.171816 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.178999 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:35.181155 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:35.183231 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.184023 29141 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:35.186064 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.186872 29144 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:35.117096  3142 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:35.120659  3285 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:35.187881 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.140301  3208 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.193714  3261 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:06:35.196219  3256 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:06:35.010818  2804 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.136s
I20260709 13:06:35.195767 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.252475 29142 raft_consensus.cc:2410] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:06:35.188273 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:35.120565  3151 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 13:06:35.010942  3140 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:35.170686  2959 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:35.185711  3265 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:35.153458  3180 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:35.007184  3275 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:35.247520  3287 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.367s	user 0.002s	sys 0.000s
I20260709 13:06:35.188685 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.266816 29217 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:06:35.214389  3203 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:35.212743  3157 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:35.287442 29141 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:35.213541  3208 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:35.214143  3228 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:06:35.316285 29143 raft_consensus.cc:2376] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17. Re-sending same reply.
I20260709 13:06:35.170547  3223 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:35.300104  3203 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:35.198482  3236 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:35.349996 29148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.000s	sys 0.027s
W20260709 13:06:35.351224 29148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.000s	sys 0.028s
I20260709 13:06:35.348881  3203 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:35.383262  3142 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:35.385854 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:35.386904  2804 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:35.387632  2804 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:35.388020 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:35.389381 29217 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:35.388751  2804 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:35.365542  3295 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 13:06:35.396926 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:35.411432  3302 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:35.438925  3124 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:35.422281  3221 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:35.426102 29065 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:35.428617  3299 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 13:06:35.430693  3293 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:06:35.447311 29139 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:06:35.459117 29139 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: [17.10-20.11]
W20260709 13:06:35.418237  3119 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.729s	user 0.006s	sys 0.005s
W20260709 13:06:35.451622 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 54786.9us
W20260709 13:06:35.461944 29145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.033s	sys 0.040s
W20260709 13:06:35.462316 29145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.033s	sys 0.040s
I20260709 13:06:35.462508 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:35.315443  3227 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:35.455937  3263 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:06:35.452855 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:35.461306 29139 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:06:35.475246 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:35.455974  3291 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:06:35.485798 29144 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:06:35.486402 29144 pending_rounds.cc:85] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 11
W20260709 13:06:35.476979 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 321962us
I20260709 13:06:35.487953 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:35.490113 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:35.455087  3124 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:35.493139 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 44151.7us
W20260709 13:06:35.495194 29067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.010s	sys 0.017s
I20260709 13:06:35.498152 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:35.500361 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:35.505883 29073 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:35.506108 29144 pending_rounds.cc:107] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:06:35.506407 29144 raft_consensus.cc:2889] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:35.501300 29069 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:35.503294 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:35.507741 29067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.010s	sys 0.018s
I20260709 13:06:35.509837 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:35.456969  3221 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:35.514169 29070 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:35.518882 29218 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.009s	sys 0.009s
I20260709 13:06:35.519063 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:35.519399 29218 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.009s	sys 0.009s
I20260709 13:06:35.521015 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.523032 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.525060 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.525149 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:35.527208 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.529172 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.531069 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:35.534296 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:35.542413 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:35.492540  3119 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:06:35.540095 29145 raft_consensus.cc:2955] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 8 to 10, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:35.509058  3142 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:35.557514 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:35.504475  3296 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:06:35.572449 29214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.007s	sys 0.022s
I20260709 13:06:35.541054  3265 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:06:35.547413  3055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.001s	sys 0.005s
W20260709 13:06:35.577688 29214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.009s	sys 0.022s
I20260709 13:06:35.591423 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:35.491196  3304 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 13:06:35.536628  3261 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:35.502398  2959 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:06:35.550704  3228 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:06:35.593554 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51385.1us
I20260709 13:06:35.594214  3265 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:35.595366  3248 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:35.595837  3124 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:35.599563  3055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.002s	sys 0.006s
I20260709 13:06:35.556172  3097 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:35.567837  2885 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:35.618763  3180 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:35.563855  3307 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:35.822903 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 278678us
I20260709 13:06:35.612445  3221 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:35.607748  3208 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:35.831866 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 306947us
I20260709 13:06:35.948239  3265 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:36.040410 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:36.042268 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:35.898947  2441 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.949s	user 0.012s	sys 0.016s
W20260709 13:06:35.904711  3252 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.127s	user 0.015s	sys 0.007s
I20260709 13:06:35.907382  3302 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:35.908229 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 317046us
I20260709 13:06:35.593415  3308 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:35.910916  3172 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:35.912158  3285 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:35.844190  3182 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.011s	sys 0.021s
W20260709 13:06:36.080372 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 44725.8us
I20260709 13:06:35.918000 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2786) took 1149 ms. Trace:
W20260709 13:06:36.018226 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 64785us
I20260709 13:06:36.083878 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:36.046620  3309 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:36.084479 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.036136  3124 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:36.048866 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:36.078531 29188 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:36.020388  2959 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.079512 29219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7517b9e93904801886467474f000b2d"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
W20260709 13:06:35.934834  3203 consensus_meta.cc:287] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.534s	user 0.014s	sys 0.011s
I20260709 13:06:36.035666  2885 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:36.053835  3208 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:36.020315  3140 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:36.029042  3313 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:36.080160  3182 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.140s	user 0.011s	sys 0.022s
I20260709 13:06:36.094201 29219 raft_consensus.cc:606] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:36.087661 29147 rpcz_store.cc:276] 0709 13:06:34.768375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.769405 (+  1030us) service_pool.cc:224] Handling call
0709 13:06:34.783803 (+ 14398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:34.784170 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.784200 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:34.794965 (+ 10765us) log.cc:844] Serialized 26 byte log entry
0709 13:06:34.989044 (+194079us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:35.536281 (+547237us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.821601 (+285320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.821710 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:35.821771 (+    61us) raft_consensus.cc:1685] finished
0709 13:06:35.821790 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:35.917915 (+ 96125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":322262,"prepare.queue_time_us":157027,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":71799,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":16228,"spinlock_wait_cycles":942464,"thread_start_us":918346,"threads_started":3,"wal-append.queue_time_us":349718}
I20260709 13:06:36.079757  3252 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:36.134135 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3003) took 1101 ms. Trace:
I20260709 13:06:36.134899 29221 rpcz_store.cc:276] 0709 13:06:35.033048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.033154 (+   106us) service_pool.cc:224] Handling call
0709 13:06:35.091292 (+ 58138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.091520 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:06:35.508321 (+416801us) log.cc:844] Serialized 14 byte log entry
0709 13:06:36.133541 (+625220us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:36.133650 (+   109us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:36.133725 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.133849 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.134048 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95225,"prepare.queue_time_us":273090,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":13567,"raft.queue_time_us":226446,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":25682,"thread_start_us":917105,"threads_started":3}
I20260709 13:06:36.093904  3248 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:36.084692  3309 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:36.053996  2441 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.114753  2885 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:36.105778  3221 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:36.044425  3097 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.131860  3244 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:06:36.087551  3172 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.083190  3302 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.114599  3180 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:06:36.106112  2959 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.197295 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:36.089088  3285 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:36.199481 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 105203us
I20260709 13:06:36.107419  3203 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.124333  3112 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:06:36.215533 29140 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:36.154893  3252 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:36.218591 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.089244  3308 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:36.220199 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.151566  3337 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.221573 29904 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:06:36.196729  3331 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:06:36.190490  3248 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:06:36.237690 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:36.215132 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 166371us
I20260709 13:06:36.240471 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:36.239385 29911 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:36.147186  3140 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.242347 29141 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:36.216920  2804 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:36.248931 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:36.257278 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:36.249684 29222 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:36.258736 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:36.262457 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:36.288350 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:36.125456  3261 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.228639  2951 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.340418 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2805) took 1625 ms. Trace:
I20260709 13:06:36.243350  3252 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:36.340993 29067 rpcz_store.cc:276] 0709 13:06:34.714696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.736599 (+ 21903us) service_pool.cc:224] Handling call
0709 13:06:34.736924 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:34.737363 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:34.737399 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:34.761621 (+ 24222us) log.cc:844] Serialized 26 byte log entry
0709 13:06:35.508291 (+746670us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:35.508343 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.508412 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.339930 (+831518us) raft_consensus.cc:1685] finished
0709 13:06:36.339955 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.340343 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18486,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":7829,"thread_start_us":716498,"threads_started":1,"wal-append.queue_time_us":1379957}
I20260709 13:06:36.254752  3142 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.380676 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:36.364722 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 102366us
I20260709 13:06:36.345021  2804 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:36.301599  3227 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.088s
W20260709 13:06:36.396843 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 169159us
I20260709 13:06:36.344132  2951 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.417484  3097 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.417833  3140 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.418098  3097 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.418344  3227 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.418646  3097 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.419005  2951 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.418952  3227 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.419703  2951 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:36.420449  2951 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:36.389583  3142 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.390727  3248 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.421270  2951 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.418960  2959 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.423036 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.423672  2959 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.423815  3335 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.396448  2804 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:36.424609  3316 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.424696  2804 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:36.416445  3126 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:36.416796  3222 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.385385  3252 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:36.424279  2959 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.416106  3310 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.391252  3351 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.243573  3337 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:36.435213  3311 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:06:36.422402  3142 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.265990  3309 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.419729  3172 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.432511 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:36.491104 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 94308.8us
I20260709 13:06:36.506731 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:36.451115 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:36.431975  3140 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:36.515345 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.502880 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:36.491439  3314 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 13:06:36.430222  3126 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:06:36.515401 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 83828.1us
I20260709 13:06:36.425205  3316 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:36.544736 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:36.553174  3319 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.343s	user 0.001s	sys 0.000s
I20260709 13:06:36.553690  3355 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:36.554345  3355 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.506065  3142 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.554858 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1101) took 1789 ms. Trace:
I20260709 13:06:36.555557 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2784) took 1787 ms. Trace:
I20260709 13:06:36.505846  3282 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.509301  3324 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:36.424365  3335 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.511257  2441 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.559701 29214 rpcz_store.cc:276] 0709 13:06:34.765021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.767446 (+  2425us) service_pool.cc:224] Handling call
0709 13:06:34.767842 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:34.777056 (+  9214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.777112 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:34.777763 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 13:06:34.828601 (+ 50838us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b1800343780. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:06:35.895138 (+1066537us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:35.914453 (+ 19315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.914540 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.554546 (+640006us) raft_consensus.cc:1685] finished
0709 13:06:36.554567 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.554793 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10269,"raft.queue_time_us":14229,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":8145,"spinlock_wait_cycles":3729664,"thread_start_us":779550,"threads_started":1,"wal-append.queue_time_us":1163032}
I20260709 13:06:36.559762 29145 rpcz_store.cc:276] 0709 13:06:34.767765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.768737 (+   972us) service_pool.cc:224] Handling call
0709 13:06:34.769471 (+   734us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:34.776274 (+  6803us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.776340 (+    66us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:34.795718 (+ 19378us) log.cc:844] Serialized 590 byte log entry
0709 13:06:35.462711 (+666993us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:35.917706 (+454995us) log.cc:844] Serialized 14 byte log entry
0709 13:06:36.020857 (+103151us) log.cc:844] Serialized 14 byte log entry
0709 13:06:36.021105 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.021203 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.555331 (+534128us) raft_consensus.cc:1685] finished
0709 13:06:36.555363 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.555494 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49402,"prepare.queue_time_us":313966,"prepare.run_cpu_time_us":170,"prepare.run_wall_time_us":14243,"raft.queue_time_us":83908,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":82355,"thread_start_us":632229,"threads_started":1,"wal-append.queue_time_us":1096131}
I20260709 13:06:36.562714  3355 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.509737  3350 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.514937  3227 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.574383  3354 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:36.575078  3355 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:36.577451  3354 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.578876  3335 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:36.579433  3329 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:06:36.579502  3346 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:36.579563  3353 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:36.580681  3337 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.598302  3337 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:06:36.587541 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:36.495276  3310 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.590021 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.604460 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:36.566579 29147 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:36.563294  3324 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:36.504750  3351 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.516211  3140 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:36.577716  3338 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 13:06:36.452318  3222 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:36.582190  3340 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 13:06:36.599167  3337 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:36.528079  3333 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.082s
W20260709 13:06:36.624822 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 118187us
I20260709 13:06:36.595327  3335 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:36.580976  3097 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.582353  3302 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.627657 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:36.577828  3339 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:36.575467  3356 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:36.592918  2959 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.594743  3354 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:36.608894  3320 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.641880  3156 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:36.641963  3353 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.642693  3140 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.643307  3140 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:36.567408  3142 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.616279  3347 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.634677 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2783) took 1870 ms. Trace:
I20260709 13:06:36.635823  3261 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:36.619793  3285 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.642462  3156 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.657721 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:36.642771  3353 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.610121  3324 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:36.636580 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 80358.2us
I20260709 13:06:36.617313  3308 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.583631  3355 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.652424 29148 rpcz_store.cc:276] 0709 13:06:34.763767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.763888 (+   121us) service_pool.cc:224] Handling call
0709 13:06:34.764230 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:34.764572 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:34.764600 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:34.766614 (+  2014us) log.cc:844] Serialized 26 byte log entry
0709 13:06:35.351617 (+585003us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:35.378950 (+ 27333us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.379237 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.379334 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.634442 (+1255108us) raft_consensus.cc:1685] finished
0709 13:06:36.634464 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.634613 (+   149us) inbound_call.cc:177] Queueing success response
0709 13:06:36.643636 (+  9023us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b180014c200. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
Metrics: {"mutex_wait_us":2888,"prepare.queue_time_us":27059,"prepare.run_wall_time_us":55,"raft.queue_time_us":2360,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":141,"spinlock_wait_cycles":96700032,"thread_start_us":584568,"threads_started":1,"wal-append.queue_time_us":1370697}
I20260709 13:06:36.570675  3282 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.664341 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.665098 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.666043 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.665980 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.664981 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.680385 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.640424  3097 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:36.582432  3350 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.690433 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3013) took 1654 ms. Trace:
I20260709 13:06:36.644340  3335 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:36.643823  3140 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.694766 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:36.640988  3339 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.653505  3330 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.729498 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:36.693408  3333 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:36.732121 29071 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:06:36.697223  3354 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:36.730387 29067 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:36.740451  3316 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:36.740600 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:36.741447 29071 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:36.742617  3282 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.745244  3140 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.746635  3222 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.746919  3360 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:36.748670  3350 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.750794 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2794) took 1937 ms. Trace:
I20260709 13:06:36.751242 29144 rpcz_store.cc:276] 0709 13:06:34.813343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.250773 (+437430us) service_pool.cc:224] Handling call
0709 13:06:35.251090 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.537816 (+286726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.750560 (+1212744us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b540029e354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:36.750719 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513091456}
I20260709 13:06:36.752368  3365 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:36.753096 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.752905  3365 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:36.754467 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.755164 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:36.729756  3302 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.757085  3180 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.758016 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.723029  3335 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:36.759316 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.759785 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2800) took 1945 ms. Trace:
I20260709 13:06:36.760612 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.761066  3339 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.679299  2959 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:06:36.661612  3261 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:36.732072  3323 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:06:36.765043  3261 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.645073  3356 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.724864  3097 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.760316 29141 rpcz_store.cc:276] 0709 13:06:34.814829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.315230 (+500401us) service_pool.cc:224] Handling call
0709 13:06:35.315618 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.254440 (+938822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.759553 (+505113us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b54003d36d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:36.759719 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2130809216}
I20260709 13:06:36.767853  3097 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:36.701167  3349 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:36.699776 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:36.768699  3349 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.759318  3335 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.769749 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.770439  3335 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:36.771704  3343 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.634s	user 0.004s	sys 0.000s
I20260709 13:06:36.756916  3302 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:36.772743  3336 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:06:36.757725  3180 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.774971 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3017) took 1386 ms. Trace:
I20260709 13:06:36.775352 29217 rpcz_store.cc:276] 0709 13:06:35.388424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.389899 (+  1475us) service_pool.cc:224] Handling call
0709 13:06:35.390156 (+   257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:35.484714 (+ 94558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.484745 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:35.485231 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 13:06:35.806569 (+321338us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:35.806631 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:35.806732 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.774769 (+968037us) raft_consensus.cc:1685] finished
0709 13:06:36.774791 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.774902 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105656,"raft.queue_time_us":102633,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":30137,"thread_start_us":258099,"threads_started":1,"wal-append.queue_time_us":1136167}
W20260709 13:06:36.778028  3341 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.214s	user 0.000s	sys 0.004s
I20260709 13:06:36.784538 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:36.779536  3342 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.216s	user 0.001s	sys 0.001s
W20260709 13:06:36.734233  3249 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.028s	sys 0.024s
I20260709 13:06:36.785061  3349 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:36.785625  3249 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.028s	sys 0.025s
I20260709 13:06:36.785084  3355 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.786604  3157 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:36.709661 29213 rpcz_store.cc:276] 0709 13:06:35.044292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.208767 (+164475us) service_pool.cc:224] Handling call
0709 13:06:35.211233 (+  2466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:35.576587 (+365354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:35.576624 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:35.597909 (+ 21285us) log.cc:844] Serialized 26 byte log entry
0709 13:06:36.075692 (+477783us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:36.075740 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.598996 (+523256us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400339ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:36.599051 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:36.681789 (+ 82738us) raft_consensus.cc:1685] finished
0709 13:06:36.681836 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.690321 (+  8485us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124806,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":293360512,"thread_start_us":712749,"threads_started":2,"wal-append.queue_time_us":595014}
I20260709 13:06:36.783242  3302 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.789079  3353 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.789786  3334 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:36.791149  2951 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:36.738539  3330 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.793550 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.794651 29911 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:36.794899  3330 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.658723  3347 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.799094  3347 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.799763  3344 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:36.800400  3344 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.800838 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.801585 29911 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:36.802855 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:36.805099 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.806196 29911 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:36.808357  3156 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:36.808661  3359 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:06:36.809253  3355 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:36.652925  3320 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.811563 29148 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:06:36.812014 29148 pending_rounds.cc:85] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 12
I20260709 13:06:36.812296 29148 pending_rounds.cc:107] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:06:36.812590 29148 raft_consensus.cc:2889] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:36.813009 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:36.813759  3124 consensus_meta.cc:287] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.716s	user 0.004s	sys 0.004s
I20260709 13:06:36.802796  3344 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.806473  3354 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.910679  3308 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:36.911867 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2799) took 2097 ms. Trace:
I20260709 13:06:36.912520  3320 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.913123  3310 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:36.913945 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 214234us
I20260709 13:06:36.912365 29142 rpcz_store.cc:276] 0709 13:06:34.814600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.258245 (+443645us) service_pool.cc:224] Handling call
0709 13:06:35.258566 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:35.258805 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:35.281330 (+ 22525us) log.cc:844] Serialized 14 byte log entry
0709 13:06:35.516439 (+235109us) log.cc:844] Serialized 14 byte log entry
0709 13:06:36.763104 (+1246665us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b1800478b40. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:06:36.910381 (+147277us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:36.910442 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:36.910496 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.911681 (+  1185us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:36.911805 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15217,"prepare.queue_time_us":41837,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":87369,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":29334,"spinlock_wait_cycles":131782528,"thread_start_us":352324,"threads_started":2,"wal-append.queue_time_us":334839}
I20260709 13:06:36.921727 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.921923 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:36.923149 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:36.924010 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:36.923789 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:36.928876 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:36.929806 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:36.932595 29217 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:36.933811 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:36.934770 29221 raft_consensus.cc:2368] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:36.937461 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:36.937773 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:36.938757 29216 raft_consensus.cc:2410] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:06:36.918398  3369 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.940006 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:36.940362 29143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 2927) took 2024 ms (client timeout 2214 ms). Trace:
W20260709 13:06:36.941835 29143 rpcz_store.cc:269] 0709 13:06:34.917888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.188016 (+270128us) service_pool.cc:224] Handling call
0709 13:06:36.940207 (+1752191us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b54001bc1d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:36.940303 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144706432}
W20260709 13:06:36.944309 29039 leader_election.cc:336] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.214s (SENT)
W20260709 13:06:36.945472 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.946018  3360 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:36.927874  3344 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:36.947228  3263 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:36.947527  3344 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.947860  3263 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.947778  3362 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:36.946645  3360 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:36.924746  3333 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:36.949928  3263 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.948940  3156 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:36.927693  3354 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:36.928591  3265 consensus_meta.cc:287] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.840s	user 0.011s	sys 0.006s
I20260709 13:06:36.951855  3265 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.952987 29070 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:36.954672 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:36.955508 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:36.951241  3360 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.960441 29148 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:36.962051  3312 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:36.919646  2804 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.962643  3312 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:36.963362 29071 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:36.963685  3312 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:36.964897  2804 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:36.940909  3252 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:36.966219 29067 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
W20260709 13:06:36.969756  3221 consensus_meta.cc:287] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.811s	user 0.012s	sys 0.004s
I20260709 13:06:36.970193  3356 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.936802  2939 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:36.971052  2939 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:36.975178  3348 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.403s	user 0.003s	sys 0.000s
I20260709 13:06:36.976598 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:36.976786 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3005) took 1943 ms. Trace:
I20260709 13:06:36.977149  2939 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.977367 29222 rpcz_store.cc:276] 0709 13:06:35.033695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.033815 (+   120us) service_pool.cc:224] Handling call
0709 13:06:35.034057 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.259087 (+1225030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:36.976534 (+717447us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540026c0d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:36.976722 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2586240256}
I20260709 13:06:36.978130  3344 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:36.919646  3324 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:36.988368 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:36.989553 29113 leader_election.cc:400] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:36.990969 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:36.991319 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:36.954888  3265 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.009985 29113 leader_election.cc:403] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:06:37.009678 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:36.936846  2885 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.747s	user 0.012s	sys 0.004s
W20260709 13:06:36.938068  3244 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.004s	sys 0.017s
I20260709 13:06:36.965130  3312 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:37.020968 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.012497 29146 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:37.022303 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:36.989636  2804 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:37.015097 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.016577 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.016778 29220 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:37.028223 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.014708  3324 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:37.021622 29911 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
W20260709 13:06:37.019428  3244 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.004s	sys 0.018s
I20260709 13:06:36.991793  3252 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.034001 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:37.010262  2939 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.035799 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.021530  3335 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:37.018409  2441 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:37.025903  3275 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:37.048844  3357 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:37.049186  3221 raft_consensus.cc:993] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:06:37.049257  3370 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:37.049707  3221 raft_consensus.cc:1081] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Signalling peer 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:06:37.049881  3370 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:37.053512  3370 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:37.023531  3110 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:37.050251  3265 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:37.056339 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.057085 29064 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:37.045964  2939 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.059805  2441 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.060321 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.044126 29039 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:37.030252  3371 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:37.069586 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.034017  3354 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:06:37.020289  2885 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.081421 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:37.081967 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:36.989773  3369 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.086205 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.072803  3265 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:37.064021  3275 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:37.089684 29146 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:37.036616  2804 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.068280  3335 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.114907 29068 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:37.043601  3252 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.122973 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:37.073725  3244 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.117892  3367 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.127274  3367 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.129046  2939 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.129742  3367 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:37.049525  3357 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.038944  3227 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:06:37.132115  3357 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.132741 29145 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:37.133335 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.133970  3244 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.122535 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.112670  3275 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:37.090083  3265 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:37.143980  3335 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.136826  3360 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:37.152177  3265 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:37.152532  3360 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:37.153095  3360 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:37.155900  3347 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:37.156558  3347 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:37.157724  3347 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.159099  3347 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:37.147433 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.160631  3112 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:37.161093  3112 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:37.148730 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.161985 29141 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:06:37.162042  3112 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.163210 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.163894 29141 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:37.164885 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.077665  3371 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.165544 29141 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:37.167363  3371 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.167711 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.168064 29143 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:37.168550 29141 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
W20260709 13:06:37.164876 29140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2780) took 2409 ms (client timeout 1800 ms). Trace:
W20260709 13:06:37.169373 29140 rpcz_store.cc:269] 0709 13:06:34.755584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:34.757434 (+  1850us) service_pool.cc:224] Handling call
0709 13:06:37.164645 (+2407211us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54001cc354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:37.164771 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3261026048}
I20260709 13:06:37.169691 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.170467 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.171542 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.173122 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:37.150802  3351 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.149611  3142 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.168808 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2884) took 1588 ms. Trace:
I20260709 13:06:37.175276  3360 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.174994 29218 rpcz_store.cc:276] 0709 13:06:35.580849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.583238 (+  2389us) service_pool.cc:224] Handling call
0709 13:06:35.583705 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:36.216161 (+632456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.216193 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:36.222046 (+  5853us) log.cc:844] Serialized 26 byte log entry
0709 13:06:36.514269 (+292223us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:06:36.551022 (+ 36753us) log.cc:844] Serialized 14 byte log entry
0709 13:06:36.556593 (+  5571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.163616 (+607023us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b540016c954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:37.163659 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:37.163710 (+    51us) raft_consensus.cc:1685] finished
0709 13:06:37.163800 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.168745 (+  4945us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82641,"prepare.queue_time_us":36546,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":79119,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":24635,"spinlock_wait_cycles":1453510528,"thread_start_us":285638,"threads_started":1,"wal-append.queue_time_us":420374}
I20260709 13:06:37.170861 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.178946 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.180423  2885 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.181131 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.182197 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.182477  3252 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.182780 29221 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:37.182997 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.180420 29214 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:06:37.184044 29214 pending_rounds.cc:85] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 10
I20260709 13:06:37.184259 29214 pending_rounds.cc:107] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:06:37.184500 29214 raft_consensus.cc:2889] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:37.179569 29141 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:37.186082  3203 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.186354  2804 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.187433  3360 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.173678 29140 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:37.186240 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:37.195641 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.191586 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:37.198639 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:37.200889  3306 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:37.201444  3306 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.201803  3306 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:37.199717 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:37.203722 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:37.204670  3354 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:37.205051  3354 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:37.205416  3354 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:37.210644 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.163789  3112 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:37.215821 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:37.206801 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:37.215188  3109 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:06:37.218029 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.221243 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:37.222266 29140 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:37.223340  3320 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:37.223749  3320 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:37.224172  3320 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:37.224565  3151 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:37.229817  3248 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:37.230294  3248 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.230579  3236 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:37.230659  3248 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:37.232319 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1092) took 1005 ms. Trace:
I20260709 13:06:37.233064  3309 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:37.221522 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:37.238135 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.206853  3244 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.232736 29144 rpcz_store.cc:276] 0709 13:06:36.226729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:36.770764 (+544035us) service_pool.cc:224] Handling call
0709 13:06:36.771055 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:36.786637 (+ 15582us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:06:36.786976 (+   339us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.010672 (+223696us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:37.010726 (+    54us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:06:37.010768 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.232065 (+221297us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b54003467d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:37.232116 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.232247 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47262,"prepare.queue_time_us":177,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":75944,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7748,"spinlock_wait_cycles":35337984,"thread_start_us":157385,"threads_started":1,"wal-append.queue_time_us":157781}
W20260709 13:06:37.253407 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.253510 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.254256 29144 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:06:37.256633 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:37.258647 29140 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:37.267813  3307 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:06:37.236740  3109 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:37.269858  3109 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:37.274971 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 2950) took 1742 ms. Trace:
I20260709 13:06:37.275432 29139 rpcz_store.cc:276] 0709 13:06:35.532227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.532658 (+   431us) service_pool.cc:224] Handling call
0709 13:06:36.940775 (+1408117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:36.941124 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:36.941155 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:36.948221 (+  7066us) log.cc:844] Serialized 26 byte log entry
0709 13:06:37.069105 (+120884us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:37.090807 (+ 21702us) log.cc:844] Serialized 14 byte log entry
0709 13:06:37.113855 (+ 23048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.113943 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:37.188895 (+ 74952us) raft_consensus.cc:1685] finished
0709 13:06:37.188949 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:37.274758 (+ 85809us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54001bc1d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:37.274883 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21448,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":52,"raft.queue_time_us":8181,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":214,"spinlock_wait_cycles":3237636480,"thread_start_us":120413,"threads_started":1,"wal-append.queue_time_us":173863}
I20260709 13:06:37.278317  3265 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.279148 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.280754 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.283192 29064 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:37.289558 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:37.289595 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.290381 29141 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:37.291630 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.293522 29142 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:37.258128  3228 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:37.311872 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:37.271853  3378 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 13:06:37.326071 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.303865 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.327284 29908 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:37.313721 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.328537 29065 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:37.313644 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.314297 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.314817 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.303611 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.330403 29071 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:37.310876  3228 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:37.316453  3265 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.271792  3110 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:37.311549  3344 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.320257  3370 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.367274 29143 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:06:37.321449 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:37.314913 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.331327 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.335850 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.378150 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:37.378651 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.337523 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.379554 29066 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:37.380409 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.380959 29066 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:37.381078 29140 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:37.381847 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.382526 29066 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:37.341987 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.344429 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.345293 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.339821 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.384060 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.384913 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.385051 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.385984 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.386308 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.386622 29073 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:06:37.293090  3369 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.352984  3344 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:37.354574 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 49629.6us
I20260709 13:06:37.356112 29148 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:06:37.396631 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.356324 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.356729 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.358837 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.359057 29145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7517b9e93904801886467474f000b2d"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:42054
I20260709 13:06:37.341279  3228 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:37.367832  3370 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.320796 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.368099 29215 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:37.368817 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.369479  3365 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.369833 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.311292  3324 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.372140  3180 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.372291 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.373833 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.374840 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.420461 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.420934 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.422020 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.422557 29217 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:37.390908 29037 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:37.394197 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3002) took 2353 ms. Trace:
I20260709 13:06:37.399197 29145 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:37.401613 29065 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:37.423588 29145 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:37.423678 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:37.417666 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:37.389443 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.426465 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.402853 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:37.423456 29220 rpcz_store.cc:276] 0709 13:06:35.032676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:35.032825 (+   149us) service_pool.cc:224] Handling call
0709 13:06:35.107901 (+ 75076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:37.030338 (+1922437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:37.385821 (+355483us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b540023bc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:37.386109 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4389372800}
I20260709 13:06:37.402984 29141 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:37.403600 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.410213 29073 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:06:37.407444 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.412252 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:37.404857 29072 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:37.427254 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:37.391547  3244 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:37.463488 29221 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
W20260709 13:06:37.447628 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.490803 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 57627.7us
I20260709 13:06:37.450034 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:37.489066 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 50656.5us
I20260709 13:06:37.464596 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.421168 29143 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:37.449241 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:37.429582 29142 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:37.429127 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:37.493140 29066 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
W20260709 13:06:37.493283 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 45927.9us
I20260709 13:06:37.406446  3358 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.471720 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:37.400144  2804 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:37.459782 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:37.490638 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 78482.6us
I20260709 13:06:37.433589 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:37.494266 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 67086.7us
I20260709 13:06:37.499751 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:37.498715 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:37.501279 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:37.507899 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:37.509519 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.513561 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:37.515578 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:37.515882 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 65947.1us
W20260709 13:06:37.517271 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 88241.1us
I20260709 13:06:37.516793 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.520784 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.521685 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.450441  3221 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:37.528985 29145 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.433889  3369 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:37.529016 29214 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:06:37.502513  3140 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:37.534147 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.554960 29145 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.401335  3320 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:37.458768 28964 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 265) took 36725 ms (client timeout 15000 ms). Trace:
I20260709 13:06:37.558748  3320 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.506981  3223 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:37.561115  3223 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:37.561741  3223 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:37.512540  3244 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.533013 29221 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:06:37.530000 29072 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:06:37.507274  3346 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:37.514698  2939 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:37.572930  2939 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.573414  2939 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:37.507274  3265 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:37.462658  3365 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:06:37.417382  3249 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.110s
I20260709 13:06:37.435698  3109 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.494592  3180 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:06:37.437353  3354 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.525141  3370 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:37.426411  3324 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.462185  3307 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:06:37.506810  3236 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:06:37.530894  3367 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:37.528559  3221 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:37.416682  3297 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.116s
I20260709 13:06:37.507803 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:37.535821  3110 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:37.445461  3126 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.544407  3129 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:37.531488  3347 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:06:37.525434  3306 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.516076  3358 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.550938  3140 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:37.446117  3248 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.571080  3346 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.583441  2441 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.618323 29217 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:37.623476 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 235184us
I20260709 13:06:37.610548 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:37.681546 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 71077.9us
I20260709 13:06:37.682482 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:37.636459 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 228154us
I20260709 13:06:37.642504 29213 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:37.618695  3275 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.648363 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:37.657930 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 122849us
I20260709 13:06:37.686777 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.638669  3344 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:37.689208  3356 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:37.689666  3367 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:37.690038  3356 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:37.691196  3367 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:37.691576  3367 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:37.642045  3370 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.692140  3370 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:37.695104  3358 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.653080  3221 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:37.697690  3320 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:37.698032  3354 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.698141  3320 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:37.699201  3244 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.701484  3306 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:37.556877 28964 rpcz_store.cc:269] 0709 13:06:00.733566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.733952 (+   386us) service_pool.cc:224] Handling call
0709 13:06:37.458615 (+36724663us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:29.709871 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:29.710562 (+   691us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:29.710619 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:06:29.710655 (+    36us) tablet.cc:662] Decoding operations
0709 13:06:29.712557 (+  1902us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:29.712659 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:29.712691 (+    32us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:06:29.919884 (+207193us) tablet.cc:701] Row locks acquired
0709 13:06:29.933314 (+ 13430us) write_op.cc:260] PREPARE: finished
0709 13:06:29.933679 (+   365us) write_op.cc:270] Start()
0709 13:06:29.933854 (+   175us) write_op.cc:276] Timestamp: P: 1783602389933587 usec, L: 0
0709 13:06:29.933906 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 13:06:30.038675 (+104769us) log.cc:844] Serialized 7477 byte log entry
0709 13:06:30.942412 (+903737us) op_driver.cc:464] REPLICATION: finished
0709 13:06:31.676095 (+733683us) write_op.cc:301] APPLY: starting
0709 13:06:31.676196 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:31.818737 (+142541us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:31.818877 (+   140us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:32.319167 (+500290us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:32.360306 (+ 41139us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 13:06:32.360377 (+    71us) write_op.cc:312] APPLY: finished
0709 13:06:32.360902 (+   525us) log.cc:844] Serialized 172 byte log entry
0709 13:06:32.847970 (+487068us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:32.903067 (+ 55097us) write_op.cc:454] Released schema lock
0709 13:06:32.926799 (+ 23732us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:32.952802 (+ 26003us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:32.969207 (+ 16405us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2345803,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":661822,"apply.run_cpu_time_us":176362,"apply.run_wall_time_us":1271287,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83547,"num_ops":19,"prepare.queue_time_us":383262,"prepare.run_cpu_time_us":33253,"prepare.run_wall_time_us":626698,"raft.queue_time_us":262027,"raft.run_cpu_time_us":17882,"raft.run_wall_time_us":337711,"replication_time_us":979477,"thread_start_us":1060099,"threads_started":5,"wal-append.queue_time_us":901421,"wal-append.run_cpu_time_us":44199,"wal-append.run_wall_time_us":2170591}]]}
I20260709 13:06:37.704898  3110 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:37.705338  3110 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:37.705737  3110 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:37.707175 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:37.709122  3391 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 13:06:37.709475 29185 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.711089  3310 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:37.711555  3310 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:37.711937  3310 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:37.713675  3275 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.714334  3275 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:37.715142  3275 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:37.716077  3313 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:37.716593  3313 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:37.717046  3367 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.717397  3313 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.718273  3367 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:37.718542  3313 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:37.716132 29218 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:37.721266 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:37.722640  3249 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:37.723057  3249 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:37.727329  3316 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:37.728125  3316 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:37.728477  3316 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:37.728861  3316 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:37.703310  3140 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.733065  3324 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.734181  3109 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:37.735016 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 53858.4us
I20260709 13:06:37.687942 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.688854 29216 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 23: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:37.735049 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:37.740375 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 52525us
I20260709 13:06:37.654726  3228 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.742086 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:37.683501  3346 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:37.742187 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55930.8us
I20260709 13:06:37.744825 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.678712  3098 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:06:37.760003 29111 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.768466 29144 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:37.769613 29112 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.772193 29142 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:06:37.773980 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:37.774307 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.774870 29910 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:37.775699 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:37.741341  2441 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:37.777149 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:37.772509 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.782629 29186 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:37.784052 29112 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.786724 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.787360 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.755963  3332 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:37.757332  3306 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:37.789759 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:37.791260 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:37.794036 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:37.771874  3180 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:37.802534 29186 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:37.780628  3356 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:37.785247  3309 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:37.699224  3320 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.789193  3363 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:37.792263  3097 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:37.807777 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.790794  3208 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:37.795451  3332 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.785717  2939 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.816628 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.834040 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.800978  3326 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:37.834995  3389 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:06:37.808218  3349 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:37.840775  3349 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:37.841306  3349 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:37.813225 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:37.848029 29144 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:06:37.818593  3244 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:37.852440 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44771.4us
I20260709 13:06:37.818920  2441 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:37.856133 29145 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:37.858199 29186 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:37.859696 29148 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:37.832276  3332 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:37.854040  3363 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:37.873000  3228 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.874037  3370 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.876232  3248 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.876739  2939 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.869050 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.848547  3365 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:37.878479 29145 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:37.853732  2951 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:37.880848 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:37.880425 29144 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:37.860055  3097 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:37.883191 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:37.886708  3110 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.887821  3126 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:37.902470 29148 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:37.862385  3320 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:37.907876  3324 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:37.909171 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 51109.5us
I20260709 13:06:37.910439 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.909736 29902 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:37.918893 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:37.920971 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:37.921309 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:37.923748 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:37.913058 29186 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.924494 29216 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:37.926353 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:37.929924  3398 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:37.930474  3398 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:37.930840  3398 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:37.921638 29143 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:37.933586  3124 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.934114  3124 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.924064 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.925019 29216 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:37.936353 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:37.876251  3370 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.944213 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:37.891256  3316 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.944458 29186 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:37.953896 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.938207  3324 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:37.950481 29216 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.938570  3365 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.965632  3365 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.966775  3398 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.940670  3310 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.969815  3310 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.970170  3398 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:37.959066  3370 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:37.971264  3370 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:37.972205  3370 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.975816  3124 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.976190  3098 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:37.884821  3223 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.976658  3098 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:37.977099  3098 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:37.977559 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:37.978173 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:37.978236  3223 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:37.981058  3249 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:37.982522  3249 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:37.985285  3316 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:37.985071  3098 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:37.988844 29216 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:37.990235 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 482515us
I20260709 13:06:37.991236 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:37.992065 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 47734.8us
I20260709 13:06:37.995738 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:37.996503 29907 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:37.996357 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:38.002949 29908 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:38.006548 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.006875 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.007589 29065 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:38.009994  2939 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:37.940023  3221 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.011449 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:38.012298 29073 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
W20260709 13:06:38.014540 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 269345us
I20260709 13:06:38.017491 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:38.018236 29215 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:38.018335 29217 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:06:38.018906 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:37.992041  3326 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:37.913610  3346 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.021517 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:38.024066 29038 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:38.023384 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.023682 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:38.025890 29072 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:38.028215 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:38.028851 29070 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:38.028182 29216 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:38.014825  3098 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.000422  3265 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:38.010823  2939 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.034952 29219 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:38.033083 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:38.014945  3248 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:06:38.035625 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:38.033808 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:38.042313 29218 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:38.038035 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:38.036063 29217 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:38.044925 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.051302 29065 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:38.044023 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:38.053782 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:38.021294  3365 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.055397 29113 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:38.060149 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:38.037945  3167 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.028409  3241 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:38.037451  3265 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:38.051505 29072 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:38.063009  3167 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.065619 29221 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:38.063311  3041 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:38.070441 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:38.024231  3223 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:06:38.021591  3332 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.058101  3244 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:38.058650  2804 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:38.062108  3363 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:38.019330  3349 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.073729  3180 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:06:38.092988 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 59978us
I20260709 13:06:38.072058  3265 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:38.093489 29145 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:38.084329  3244 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.088681  2441 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:38.090142  3363 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:38.101123 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 47407.3us
I20260709 13:06:38.091305  3140 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:38.092931  2804 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:38.109172 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:38.115059 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 264369us
I20260709 13:06:38.121085 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:38.099543  3360 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:38.096908  2885 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:38.100427  3244 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:38.112874 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:38.108729  3363 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:38.098387 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:38.101630 29222 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:38.127895 29112 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:38.099294  3180 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:38.166942 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68685.1us
I20260709 13:06:38.148939  3167 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.150020  2939 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.150599  3409 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:38.154904 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:38.111295  3223 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:38.148557  3172 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:38.158018  3221 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.159446 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:38.160143  3410 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:38.160655  2885 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:38.161347  3140 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:38.164608  3346 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:38.164856  3400 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:06:38.074759 28962 catalog_manager.cc:5697] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 13 to 15, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 15 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:38.169170 29112 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:38.169411 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:38.169800 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:38.196851 29147 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:38.171768 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:38.170202  3349 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.172837  2441 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:38.207813 29213 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:38.210783 29903 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:38.198122 29143 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:38.201737 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.202148  3365 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:38.188886  2939 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:38.200723 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:38.211966 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:38.212011 29072 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:38.180090  3360 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:06:38.181864  3223 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.217700  2885 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.230377  3332 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.219563  3180 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.215888 29147 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:06:38.189569  3172 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:38.250012  3172 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.251135  3126 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:38.251364  3392 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:06:38.232501 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:38.227449  2939 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.253283  2939 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:38.241612  3398 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:38.240425  3221 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:38.239163 29188 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:38.242208  3110 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.231319  2441 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:38.260402 29148 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:06:38.261683 29148 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:38.231174  3403 raft_consensus.cc:2764] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:06:38.252789  3392 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:38.262624  3392 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:38.263749  3392 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.256449 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:38.257541  3349 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:38.257038  3356 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:38.267421 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:38.268079 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:38.274893  3402 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:38.275365 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:38.276263 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:38.277437 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:38.278587 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:38.279086 29217 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 23: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:38.285893  3223 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.286178  3304 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:38.210881  2804 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.270016  3320 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:38.288420  3304 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.275354  3402 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:38.270490  3349 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.271705  3392 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:38.288342  3347 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:38.273087  3363 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.286978  3356 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.304185  3349 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:38.308915  3347 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:38.312871  3393 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:38.322419  3393 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:38.322904  3347 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.323431  3393 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.324715  3393 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:38.313354  3304 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.316181  3265 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.313392  3356 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.317723  3402 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.331753  3402 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:38.320128  3363 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.332621  3349 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.335381  3349 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:38.369489 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 144011us
I20260709 13:06:38.248495 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:38.373308 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 124805us
I20260709 13:06:38.208441 28962 catalog_manager.cc:5697] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 11 to 15, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 15 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:38.377519 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:38.380649 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 87568.1us
W20260709 13:06:38.369294 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 238756us
I20260709 13:06:38.385572 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:38.387370 29218 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:38.387733 29216 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:38.389812 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:38.394553 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:38.397148 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:38.397437 29071 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:06:38.389647 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:38.400311 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:38.407055 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:38.377817 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:38.408113 29069 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:38.411420 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.415259 29066 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:38.415462 29071 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:38.417830 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:38.423933 29907 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:38.424100 28962 catalog_manager.cc:5697] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 12 to 15, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 15 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:38.415347 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:38.419277 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 151804us
I20260709 13:06:38.419584 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:38.439908 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:38.383503 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:38.440375 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.444494 29066 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:38.389539 29072 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
W20260709 13:06:38.445025 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 61882.6us
I20260709 13:06:38.451809  2441 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:38.452759  2441 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:38.454332  2441 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.454995 28962 catalog_manager.cc:5697] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: config changed from index -1 to 10, term changed from 10 to 16, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to <none>, VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 16 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:38.460126 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:38.460289  3140 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:38.461496  3140 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:38.462185  3140 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.461573 28962 catalog_manager.cc:5697] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:38.464701  3167 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:38.465399  3167 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.465646  3265 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.465885  3167 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:38.466811  3409 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:38.318753  3244 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.468715  3346 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:38.469187  3409 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:38.470263  3409 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:38.471048  3244 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.504544  3180 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:06:38.505028  3421 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
W20260709 13:06:38.505101 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 45212.7us
I20260709 13:06:38.505358  3180 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:38.476914 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.477118 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:38.508317 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.506892  3180 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.516193  3410 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:38.518532  3410 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.519270 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:38.521638  3410 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.478646  3126 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:38.523635  3126 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:38.524170  3126 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:38.480479  3411 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:38.527510 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:38.528127  3411 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:38.481319  3252 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:38.529817  3252 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:38.529654  3411 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.531414  3411 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:38.482309  3297 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:38.482537  3307 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:06:38.486101  3346 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:38.486876  3097 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:38.487324  3316 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:38.494078  3402 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:38.502066  3112 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:38.503340  3110 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:38.481107  3248 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:06:38.531365  3252 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.537454 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:38.552634 29139 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:06:38.555307 29070 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:06:38.556602 29073 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:38.557000 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.557560 29142 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: [17.10-21.12]
W20260709 13:06:38.559659 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.541913  3346 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.563683 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.567704 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.574316  3252 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:38.574409 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:38.575366 29142 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:38.577551  3360 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:38.578322  3360 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.579638  3347 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:38.580441  3347 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.582967  3347 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:38.586740  3140 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.588176  3307 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:38.588656  3307 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:38.589952  3307 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.590854 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:38.568569 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.592976  3402 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.594712  3349 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:38.595362  3349 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.597693  3349 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:38.566483  3346 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:38.599027 29215 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:38.601432  3151 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:38.601888  3151 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:38.602769  3360 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.603016  3151 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.612697 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:38.605123  3124 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:38.613489  3124 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:38.612113 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:38.614567  3124 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.616729  3124 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:38.617122 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:38.617331 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 125893us
I20260709 13:06:38.617811 29073 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:38.617852  3110 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:38.618331  3223 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:38.618779  3223 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:38.620900  3223 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.622604  3223 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:38.612111  3316 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.618886  3110 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:38.644093  3110 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:38.645040  3110 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.646324  3110 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:38.588876  3412 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:38.648240  3412 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:38.649397  3412 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.651180  3412 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:38.618808 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.653921 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:38.654357 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:38.655299 29902 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:38.626610  3409 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:38.656404  3409 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.629323  3151 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:38.657063  3221 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:38.629706  3402 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.610502  3167 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.636750 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:38.637590 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:38.659050  3409 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.655107 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:38.626834 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:38.656879 29222 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:38.664752  3112 consensus_queue.cc:1059] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:38.665553 29222 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:06:38.643599  3316 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.611057  3365 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.666512  3363 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:38.666934  3363 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:38.666994  3365 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.667405  3244 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:38.668128  3363 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.669353  3363 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:38.670243 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:38.671991 29072 raft_consensus.cc:2368] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18.
I20260709 13:06:38.591710  3307 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:38.677206 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:38.686764  3347 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:38.687551 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:38.688658  3409 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:38.688838 29216 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:38.689251  3409 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:38.690327  3409 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.688019 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:38.688190 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:38.692569 29219 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:38.692777 29215 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
W20260709 13:06:38.695250 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68541.2us
I20260709 13:06:38.696233  3167 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.700088 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:38.704682  3248 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:38.708982 29216 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
W20260709 13:06:38.709337 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 53463.8us
I20260709 13:06:38.713855 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:38.736299 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:38.736677 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:38.739786  3347 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:38.740404  2939 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.740901  3347 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:38.678556 28962 catalog_manager.cc:5697] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 14 to 16, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 16 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:38.741833  3347 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:38.742722  2939 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.742430 28962 catalog_manager.cc:5697] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 9 to 13, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 13 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:38.749197 29069 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:38.751984  3424 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.100s	user 0.004s	sys 0.000s
I20260709 13:06:38.752259  3324 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:38.752751  3324 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:38.752419  3346 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.753921  3324 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.755648 29216 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:38.759495  3252 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:38.764647 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:38.768915 29219 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:06:38.740742  3365 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.772542 29073 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:06:38.760097 29066 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:38.769294 29907 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:38.797552 29142 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:06:38.743759  3347 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:38.810616 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:38.813612 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:38.797354  3324 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:38.817128 29073 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:38.815245 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:38.819682 29147 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:38.820982 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:38.816354  3252 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.823688 29219 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:38.827046 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:38.816807  3248 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:38.835522 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:38.841382 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:38.841831 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:38.836807 29188 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:38.839143 29215 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:38.816908  3167 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:38.851125 29142 raft_consensus.cc:2368] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:06:38.833905 29112 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:38.842667 29145 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:38.811911  3347 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:38.840675 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:38.830116  3346 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.838176  3180 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:38.871490 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:38.873411 29908 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:06:38.880210 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 49585.5us
I20260709 13:06:38.881292 29875 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:38.833894  3365 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 13:06:38.885733 29221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.009s	sys 0.053s
W20260709 13:06:38.886168 29221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.009s	sys 0.054s
W20260709 13:06:38.886427 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70269.4us
I20260709 13:06:38.886931 29146 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:06:38.887692 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:38.888185  3356 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:38.888447  3422 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:38.888921  3356 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:38.885978  2804 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:38.866222  3244 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:38.898128  2939 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:38.877189  3180 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:38.901566  2939 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.953405  3180 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.954773  2939 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.904629 29213 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:06:38.905144  3306 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:38.959161  3306 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:38.959618  3306 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:38.905150  3316 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:38.875751  3167 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.905696  3228 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:38.961939  3167 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.961615  3228 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.909621 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.964550 29072 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:38.909968  3405 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:38.964865  3126 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.965593  3405 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.967160  3126 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.969012 29213 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:06:38.912843 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:38.920733 29070 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:38.926092  3347 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.971212  3347 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:38.973057  3221 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:38.973686  3221 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:38.902323 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:38.975032 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 87175.2us
I20260709 13:06:38.974809  3221 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.975055  3097 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:38.975889 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:38.976729 29902 raft_consensus.cc:1240] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:38.976137  2804 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.928175 29037 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:38.863621  3248 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.882817  3346 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:38.951221 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:38.977015  3228 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.976133  3097 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.976893  3349 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:06:38.939060 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:38.978752 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 50669us
I20260709 13:06:38.975795  3405 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.979636 29903 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:38.978965  2804 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:38.979758  3346 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:38.980948  3097 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:38.982633  3244 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:38.983067  3244 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:38.983395  3244 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:38.980669  3248 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:38.890226  3356 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:38.980587  3228 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:39.000633 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:38.883423 28962 catalog_manager.cc:5697] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 12 to 13, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 13 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:39.017961 28962 catalog_manager.cc:5697] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:39.001724 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:38.996697 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:39.021600 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 181021us
I20260709 13:06:39.002442  3326 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:39.011150  3252 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:39.024998 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:39.038470 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.031857 29073 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:06:39.042416 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:39.047053 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:39.046365 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:39.037040  3412 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:39.052112 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:39.052217  3412 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:39.052973 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:39.053800  3412 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.054774 29139 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:39.054979  3126 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:06:39.055454 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:39.056296 29068 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:06:39.050334 29902 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:39.024150  3248 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.082998 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:39.088967 29908 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:39.036525  3228 consensus_queue.cc:1059] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 13:06:39.100529  3228 raft_consensus.cc:3055] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:39.101013  3228 raft_consensus.cc:740] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:39.101964  3228 consensus_queue.cc:260] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.102643 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:39.103370  3228 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:39.090796 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:39.096429 29139 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:39.105221 29147 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
W20260709 13:06:39.094252  3411 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.535s	user 0.009s	sys 0.003s
I20260709 13:06:39.113876 29037 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:39.120131 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:39.121398 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:39.127590 29069 raft_consensus.cc:2410] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:06:39.128774 29140 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:06:39.080195  3248 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.130239 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:39.136720 29066 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:39.139554 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:39.141601  3346 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:39.100618 28962 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:39.148180  3124 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:06:39.156338 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:39.162549 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:39.166710 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.158455  3326 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
W20260709 13:06:39.198585 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 150425us
I20260709 13:06:39.198287 29188 leader_election.cc:400] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:06:39.200771 29188 leader_election.cc:403] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:39.186616  3346 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:39.189800  3252 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:39.213107 29066 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:06:39.217724 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:39.222482 29904 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:06:39.220428 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:39.224460 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.204303  3346 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:39.238240  3425 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 13:06:39.238876  3432 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:06:39.245728  3248 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:39.248340 29143 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:06:39.256747 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:39.257876 29144 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:06:39.259908  3252 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:39.260469  3252 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:39.264504 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 45252.1us
I20260709 13:06:39.234261  3167 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:39.268005 28962 catalog_manager.cc:5697] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 18, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 18 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:39.270473 28962 catalog_manager.cc:5697] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 13 to 17, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 17 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:39.272913 29213 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:06:39.275067  3316 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:39.275527  3316 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:39.276352  3431 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:06:39.277031  3316 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:39.278331  3418 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:39.280299  3316 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:39.284209 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 83861.4us
I20260709 13:06:39.286048 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:39.287281  3167 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:39.233381  3346 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:39.285221  3244 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.291764 29141 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:39.287325 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:39.302402 29908 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:39.303637 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:39.223363 29185 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:39.334189 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:39.318099  3167 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:39.341578 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:39.331687  3440 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:39.334605  3349 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:39.347062 29038 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:39.336524  3326 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:06:39.273262  3332 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 13:06:39.357256 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:39.371582 29071 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: [20.10-21.11]
I20260709 13:06:39.365425 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:39.373571 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48452.9us
I20260709 13:06:39.232420  3356 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.110s
I20260709 13:06:39.377374  3140 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:06:39.385876 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:39.328624  3306 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.345633  3412 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:39.392625 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:39.318328  3411 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:06:39.338316  3410 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:06:39.397413  3223 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:06:39.402539  3439 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:39.358232  3349 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:39.346582  3440 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:39.397277  3353 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:39.411412 29067 raft_consensus.cc:2368] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:06:39.352511  3110 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:39.440040 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.377195  3405 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:39.380297  3124 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:39.407481  3167 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:39.451967 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:39.442162  3349 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:39.464063  3124 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:39.464514  3445 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 13:06:39.466913 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 249226us
I20260709 13:06:39.470700  3168 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:06:39.472518  3306 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:39.426671  3409 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:06:39.459767  3405 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:39.492635  3405 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:39.493758  3346 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.440515  3180 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 13:06:39.474342  3439 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:39.506541  3439 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:39.441932  2939 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:06:39.453823  3140 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:39.507681  3140 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:39.508448  2885 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:39.459250  3110 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:39.511286  3110 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:39.512279 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:39.513320  3124 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:39.476639 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 51092.5us
I20260709 13:06:39.443861  3097 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.477094 28962 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 17, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 17 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:39.522228  3097 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:39.521867 28962 catalog_manager.cc:5697] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 12 to 16, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 16 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:39.478482  3393 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:39.525617  3440 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:39.448904  3244 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:39.527421  3440 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:39.496840  3447 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:39.500257 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:39.528597 29215 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:39.501073  3347 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.532312  3347 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:39.533221 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:39.534487  3347 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:39.535661  3306 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:06:39.538431  3441 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.087s	user 0.001s	sys 0.001s
I20260709 13:06:39.542287 29071 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:06:39.542708 29037 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:39.543598 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:39.544901  3353 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:39.545383  3353 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:39.546301  3353 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.560756 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:39.568810 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:39.569658 29904 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:39.523566 28962 catalog_manager.cc:5697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 9 to 14, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 14 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:39.579190  3140 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:39.589422  3140 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:39.594166  3447 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:39.596279  3451 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:39.596812  3451 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:39.597587  3252 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.581677 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:39.601342 29218 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:39.601620 29071 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:06:39.603615 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:39.606151 29067 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:39.597698  3451 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:39.623098  3451 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:39.571636 28962 catalog_manager.cc:5697] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 14 to 17, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 17 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:39.628891 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:39.634191 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:39.630759 29215 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:39.646322 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:39.660771 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:39.661887 29072 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 21: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:39.646356  3126 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:06:39.675318  3126 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:39.675755  3126 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:39.676728  3126 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.678117  3126 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:39.682269  3124 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:39.683107  3124 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:39.692714  3124 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:39.696930 28962 catalog_manager.cc:5697] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 7 to 17, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 17 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:39.699005 29907 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:39.644824  3112 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.700632  3112 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:39.703075  3112 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:39.703702  3347 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:39.705528  3252 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:39.710232 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 114340us
I20260709 13:06:39.710987  3346 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:39.711213 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:39.714176  3326 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:06:39.715127  3326 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:39.691725 29064 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:06:39.724251 29218 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:39.729458 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:39.731685 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:39.741552 29067 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
W20260709 13:06:39.747772 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 344233us
I20260709 13:06:39.748479 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:39.755450 29218 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:06:39.755044 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:39.753736 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:39.764786 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:39.765403 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 54284.4us
I20260709 13:06:39.768225 29185 leader_election.cc:400] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18.
I20260709 13:06:39.768584 29185 leader_election.cc:403] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:06:39.768718 29222 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:39.771291  3316 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18.
I20260709 13:06:39.766381 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:39.775728 29064 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
W20260709 13:06:39.777324 29188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.006s	sys 0.013s
W20260709 13:06:39.778054 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 544524us
W20260709 13:06:39.779476 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:39.699522 28962 catalog_manager.cc:5697] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:39.782258  2804 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:39.782366  3304 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.766281  3180 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:39.730367  3346 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:06:39.781486 29185 leader_election.cc:400] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:06:39.782682  3392 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:39.772485  3316 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:06:39.792217 29141 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:06:39.795364 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:39.790244 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:39.762497  3356 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:06:39.784829  3244 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:39.803875 29222 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:39.803740 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:39.804207 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:39.783288  3112 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:39.774348  3365 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:06:39.807725 29143 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:39.810227 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:39.809914 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:39.788196  3180 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:39.789830  3167 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:39.815169 29072 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:39.817471 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:39.773494  3349 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.825052 29907 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:39.826547 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:39.805809  3244 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:39.847940  3244 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:39.822432 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:39.792057  3110 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:39.827445  3353 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:06:39.840159 28962 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:39.816294  3180 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:39.810685  3392 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.874859  3392 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:39.876516 29037 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:39.803850  3332 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:06:39.882153 29218 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:06:39.880188 29222 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:39.792704  3304 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.890554 29904 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:39.877594  3453 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:39.877936  3110 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.895538  3324 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.818809  3112 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.896456  3450 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:39.904564 29215 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:39.917737 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:39.926422 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:39.890796  3440 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.931121 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:39.941643 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:39.909897  3464 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.946887  3464 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:39.947587  3167 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:39.948194 29143 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:39.949474  3464 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:39.957541  3110 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:39.957772 29038 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:39.957981 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:39.958847 29216 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:39.958971  2804 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:39.965700  3443 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:39.966284  3443 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:39.971779  3405 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:39.973098  3252 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:39.923029  3453 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:39.932539  3324 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.006951  3324 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:40.008824  3453 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.010309  3346 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.010993  3346 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.001999  3440 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.013808  3346 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.013242 28962 catalog_manager.cc:5697] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:39.941284  3297 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:40.015307  3172 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.015821  3172 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.018337  3112 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:40.020325  3450 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.023267  3450 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:40.029251  3349 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:40.034523  3332 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.035193  3332 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.038002  3332 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.014173  3440 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:40.002179  3307 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:40.056936 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 139294us
W20260709 13:06:40.065670 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 132504us
I20260709 13:06:40.072284 29185 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:40.060834 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:40.048658  3332 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:39.999562  3124 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:06:40.047554  3252 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.099047 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:40.086230 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 172257us
I20260709 13:06:40.099781 29071 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:40.086584 29902 raft_consensus.cc:2410] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:06:40.088856 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:40.103312 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:40.057169  3412 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.104446 29068 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:40.112097  3443 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.114120  3443 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:40.115635  3457 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:40.116248 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.116590 29216 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 13:06:40.117003 29216 pending_rounds.cc:85] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 10
W20260709 13:06:40.117048 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 202700us
I20260709 13:06:40.117448 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.118263 29215 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:40.118399 29217 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:06:40.117353 29216 pending_rounds.cc:107] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 13:06:40.119448 29216 raft_consensus.cc:2889] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:40.120354 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.114365 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.121304 29222 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:40.121721 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:40.122355 29213 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:40.123234  3401 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:40.124251  3401 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:40.124594  3401 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:40.125579  3401 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.126171 29216 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:06:40.126623 29216 pending_rounds.cc:85] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 11
I20260709 13:06:40.126899 29216 pending_rounds.cc:107] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:06:40.126942  3401 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:40.127305 29216 raft_consensus.cc:2889] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:40.128188  3393 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.128803  3393 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.130152  3304 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.131346  3393 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:40.098896  3469 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.215s	user 0.000s	sys 0.002s
I20260709 13:06:40.160009 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:40.119753 29069 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:06:40.163280 29069 pending_rounds.cc:85] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 12
I20260709 13:06:40.163622 29069 pending_rounds.cc:107] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:06:40.163909 29069 raft_consensus.cc:2889] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:40.117067 29221 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:40.085677  3349 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:40.166200 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:40.138741 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:40.060305  3307 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.178793 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:40.193907 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55254.9us
I20260709 13:06:40.198689 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:40.015673 28962 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:40.207645 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.221861 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.173396  3392 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:40.234884 29216 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:40.106133  3252 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.108224  3332 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.137637  3412 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:40.245507 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 125757us
W20260709 13:06:40.257349 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 91271.9us
I20260709 13:06:40.261994  3440 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.262789  3440 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.266722  3440 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:40.227373  3457 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.240584  3349 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:40.222395  3477 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:40.241729  3405 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.286033 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.293336 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.284337 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.305038 29141 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:06:40.301198 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 113058us
I20260709 13:06:40.303802 29143 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:40.308712 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:40.314982 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:40.316720 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.320807 29141 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:06:40.323774 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.324524 29143 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:40.278945  3316 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:40.330714 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 78708.9us
I20260709 13:06:40.283181  3349 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.340725 29139 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:40.279292  3457 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.347678 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:40.361769 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:40.373311 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.384320 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:40.308823  3405 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.322903  3244 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.389869 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:40.391237 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:40.356627  3349 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:40.380932 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:40.396078 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.350282  3316 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.376368  2939 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:40.416296 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.440572 29139 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:40.387329  3472 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:40.450285 29148 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:40.453744  3481 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.414997  3332 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:40.396296  3439 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:40.399847  3470 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:40.416321  3480 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:40.452104 29214 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:40.417814  3451 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:40.459607 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 116242us
I20260709 13:06:40.440181  3450 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:40.458168  3472 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:40.557389  3472 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:40.563562 29213 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:40.566601  3480 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.568914  3316 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:40.570220  3480 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:40.416399  3476 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.551940  3457 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:40.502866  3326 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:40.571113  3451 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:40.539395  3485 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:06:40.530097  3252 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:40.469061  3180 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.002s
I20260709 13:06:40.493268  3307 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.584334  3326 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.537439  3244 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.592435  3497 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:40.572513  3476 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.505419  3172 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.490480  3412 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.539350  3491 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.596560 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:40.539134  3489 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:40.595309  3180 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.604552 29113 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:40.600831 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 154409us
I20260709 13:06:40.474148  3365 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.611666 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:40.611840 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 220703us
I20260709 13:06:40.612423 29213 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:40.612888  3412 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:40.614089 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.485790  3453 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:40.620204 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:40.622159 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 179950us
W20260709 13:06:40.602877  3110 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.002s	sys 0.013s
I20260709 13:06:40.625521 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.574376  3481 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.565121  3470 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.640698 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.603642  3489 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.460587  2939 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.599500  3244 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:06:40.623462 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:40.652503 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.609215  3332 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:40.647526 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.645905 29214 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
W20260709 13:06:40.640918  3110 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.002s	sys 0.014s
I20260709 13:06:40.616784  3326 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:40.618808  3463 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:40.655135 29145 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 14.
I20260709 13:06:40.660401 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.671906 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.678453  2939 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:40.679945 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56544us
I20260709 13:06:40.681201  3405 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:40.681849 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.681907  3470 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:06:40.682723 29220 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:40.682893  3470 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:40.682139  3097 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:06:40.690937  3316 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:40.691202  3505 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.691833  3316 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.692353  3505 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.695574  3505 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.695750  3316 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:40.696740  3497 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.646500  3363 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.697419  3510 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.627274  3453 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.700153  3478 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.700549  3500 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.748878  3500 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.751299  3500 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.701738 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.610275  3491 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.704115  3498 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.705463  3481 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:40.706539  3475 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.320s	user 0.004s	sys 0.000s
I20260709 13:06:40.708914  3451 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:40.618254  3365 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.650493  3252 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.661186  3482 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.729753 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.649246  3489 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:40.697638 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:40.671358  3463 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:40.737424  3510 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.746575  3497 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.747789  3478 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.753876 29219 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:40.777215 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:40.778549 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:40.780665 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:40.781896 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:40.780288  3495 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.782821  3500 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.817968 29148 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:40.819973 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.736581  3363 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.789559  3097 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:40.799113  3412 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:06:40.801354  3252 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.781265  3498 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.857573 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.837541 29142 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:40.849278 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:40.933444 29215 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:40.857556  3508 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.114s
I20260709 13:06:40.936611 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.816263  3482 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.881313  3491 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.802155  3477 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:40.873817  3497 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.866762  3510 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:40.840512 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 142957us
W20260709 13:06:40.940070 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 681225us
I20260709 13:06:40.937264 29216 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
W20260709 13:06:40.813016  3167 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.008s	sys 0.021s
I20260709 13:06:40.846203  3326 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:40.941136 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.941764 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:40.812760  3463 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:40.870265  3478 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:40.949260 29213 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
W20260709 13:06:40.950996 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:40.984611 29069 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:40.954423 29215 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:40.963040 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.986156 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:41.006569 29216 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:40.965504 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.954999 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.008162 29143 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
I20260709 13:06:41.066243 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.066968 29219 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:40.871297  3365 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.085501 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.086221 29219 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:40.950196  3326 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.079999 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.954397  3447 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.921182  3223 raft_consensus.cc:2764] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:06:41.084022 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:40.876893  3405 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:40.939049  3482 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:40.881942  3349 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.170262 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.170924 29907 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:40.936648  3500 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.092684 29216 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:06:40.953207  3097 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:40.872414  3453 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.095024  3477 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:40.839437  3495 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:41.097415 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 329977us
I20260709 13:06:41.101533 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.233968 29903 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:06:40.876641  3476 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.103835 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:41.234972 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 131228us
I20260709 13:06:41.236032 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:41.079231  3472 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.001s
I20260709 13:06:41.237706  3472 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.239903  3472 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:41.240069  3504 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:06:41.241303  3223 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:41.241628  3363 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.241797  3223 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.131012  3140 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.242220  3223 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:41.237092 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:41.244511  3097 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:41.244884  3528 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:41.245349  3528 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.245721  3528 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:41.246178  3495 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.247184 29218 raft_consensus.cc:2368] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:41.247769 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.248519 29215 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:41.158746  3252 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.150086 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 1042) took 1430 ms. Trace:
I20260709 13:06:41.160817  3180 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.164106  3520 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.167156 29139 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:06:41.282166 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:41.167706  3405 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.168478 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.091365 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:41.173288 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:41.284896 29145 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 14.
W20260709 13:06:41.285195 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 111988us
I20260709 13:06:41.286002 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:41.216393 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.287034 29908 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:41.223409 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.225333  3464 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.225221  3244 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.225855  3347 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:41.228348  3365 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.230290 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.131003  3326 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:41.289350 29911 raft_consensus.cc:2410] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:06:41.292167 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.319813 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.293889 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:41.292317 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.253672 29902 rpcz_store.cc:276] 0709 13:06:39.719820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.720048 (+   228us) service_pool.cc:224] Handling call
0709 13:06:41.088695 (+1368647us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b5400768c54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:41.149992 (+ 61297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":809016704}
I20260709 13:06:41.295872 29903 raft_consensus.cc:2368] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21.
I20260709 13:06:41.231618  3412 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.320134 29216 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:41.331194 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:41.331616 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:41.332346 29066 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:41.332549  3464 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.340050  3244 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.321784  3470 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:41.343387  3470 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.343976  3470 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:41.353869  3306 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.354709  3306 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.357270  3306 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.360165 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.361667 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.362385 29910 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:41.363076 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.363122 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:41.363705 29910 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 13.
I20260709 13:06:41.363826 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.288957  3495 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.364531  3476 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:41.364959  3476 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.364784  3495 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.365398  3476 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:41.367122  3495 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.368489 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:41.368691 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.343310 29066 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:41.369437 29907 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:41.369800  3494 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:41.370216  3494 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.370569  3494 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:41.288875  3453 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:41.373196  3453 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.294387  3520 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.318261  3180 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.377714  3180 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:41.378768  3486 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.007s	user 0.000s	sys 0.003s
W20260709 13:06:41.382688  3525 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:06:41.383970  3365 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.385380 29186 leader_election.cc:400] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21.
I20260709 13:06:41.385854 29186 leader_election.cc:403] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:06:41.385581  3494 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.242412  3140 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.231863  2804 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.320526 29215 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:41.086652  3489 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.395407  3412 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.410827 29068 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:41.324324  3347 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.413149  3347 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:41.414588  3393 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.414920 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.415288  3393 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.415719 29911 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:41.417687  3393 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.418253  3393 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.418968  3393 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.420185  3304 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.420636 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.252772  3402 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:41.325948 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.327538 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:41.423768 29071 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:41.386369  3365 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.386727  3472 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:41.208557  3447 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.316463 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:41.400476 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2935) took 1279 ms. Trace:
I20260709 13:06:41.401130 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:41.401631  3140 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.401979 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:41.433494 29148 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:41.403843  3494 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.322774 29188 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:41.386783 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:41.318724  3097 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:41.434371 29145 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:41.431619 29217 rpcz_store.cc:276] 0709 13:06:40.121047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.122123 (+  1076us) service_pool.cc:224] Handling call
0709 13:06:40.122476 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:40.122719 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:40.122753 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:40.123274 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:06:40.224361 (+101087us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:40.224406 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.224480 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:41.377897 (+1153417us) spinlock_profiling.cc:243] Waited 45.9 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:41.399585 (+ 21688us) raft_consensus.cc:1685] finished
0709 13:06:41.399609 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:41.400378 (+   769us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105534208,"thread_start_us":97792,"threads_started":1,"wal-append.queue_time_us":154576}
I20260709 13:06:41.387853  3349 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:41.395311  3443 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21.
I20260709 13:06:41.448272 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:41.453034 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:41.466703 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 144021us
W20260709 13:06:41.472484 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 85717.6us
I20260709 13:06:41.467785 29188 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:41.450843  3347 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.481395 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.421557  3402 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:41.500671 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:41.497380 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44481.2us
I20260709 13:06:41.501503 29214 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:41.492353 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.501976 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:41.508543 29220 raft_consensus.cc:2393] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:41.511466 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.512223 29216 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:41.387099  3476 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.500450  3508 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:41.435957  3365 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.450893  3450 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:06:41.514916 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47261.1us
I20260709 13:06:41.400431  2804 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.744962  3500 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.736675 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:41.745971  2804 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.746531  2804 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:41.746855  2804 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.747179  2804 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:41.594883  3347 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.686925 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:41.749128 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62330.9us
I20260709 13:06:41.749784  3347 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.602242  3297 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.524039  3223 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.622509 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.493350  3472 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:41.419643  3528 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.505300  3402 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:41.601112 29146 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.051s	sys 0.128s
I20260709 13:06:41.429937  3304 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.685427 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:41.510563  3451 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.679576 29907 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:41.743155  3140 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:41.751325  3365 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.711172  3306 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:41.772608 29146 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.056s	sys 0.147s
I20260709 13:06:41.819612 29905 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 22: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:41.790719 29903 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:41.822894 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:41.766716  3349 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:41.783541  3140 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.786309  3490 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:41.782842  3450 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:41.770871  3297 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.845541  3140 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:41.842968  3393 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.818938  3451 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:41.865198 29908 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.014s	sys 0.011s
W20260709 13:06:41.866057 29908 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.014s	sys 0.011s
I20260709 13:06:41.850674  3500 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.839432  3349 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.860020  3447 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:41.862212  3528 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.851827  3326 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.876809  3494 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:41.894887  3326 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.895017  3494 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.896346  3450 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.897627  3494 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:41.898092  3523 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.278s	user 0.000s	sys 0.000s
I20260709 13:06:41.900713  3304 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.901533  3353 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:41.901815  3140 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.902180  3353 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.902545  3223 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.904112  3349 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:41.904337  3140 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.905145  3140 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:41.905651  3464 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.906788  3405 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.906965 29071 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
W20260709 13:06:41.909157  3463 consensus_meta.cc:287] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.834s	user 0.009s	sys 0.003s
I20260709 13:06:41.909986  3490 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:41.909763  3463 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.910442  3490 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:41.905848  3140 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.913632 29221 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:41.915174  3140 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.917132  3508 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:41.917522 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2893) took 1577 ms. Trace:
I20260709 13:06:41.917605  3365 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.919042  3451 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.917948 29147 rpcz_store.cc:276] 0709 13:06:40.340799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.342715 (+  1916us) service_pool.cc:224] Handling call
0709 13:06:40.343120 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:40.610765 (+267645us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:06:40.625184 (+ 14419us) log.cc:844] Serialized 14 byte log entry
0709 13:06:40.815427 (+190243us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:40.815519 (+    92us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:40.815585 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.361513 (+545928us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b180046ca20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20522 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:06:41.917267 (+555754us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b540029d454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:41.917316 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:41.917436 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30365,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":2934,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":16187,"spinlock_wait_cycles":652007424,"thread_start_us":141203,"threads_started":1,"wal-append.queue_time_us":689010}
I20260709 13:06:41.925055  3244 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:41.927363  3470 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.001s
I20260709 13:06:41.930562  3297 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.931208  3140 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:41.931802  3140 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.927803  3470 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.933996  3140 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.934121  3470 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.935519  3140 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:41.915855 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:41.888316  3500 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.936101  3140 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.936784 29221 raft_consensus.cc:2368] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20.
I20260709 13:06:41.937986 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.938184  3500 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.938573 29221 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 14.
I20260709 13:06:41.939531 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.940233 29221 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:41.876473  3476 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:41.940954 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:41.941018 29213 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:41.941646 29222 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:41.931797 29037 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:41.938230  3140 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.954267  3140 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:41.954533  3463 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.955211 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:41.954864  3140 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.957983 29070 raft_consensus.cc:2368] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
W20260709 13:06:41.958642 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 484494us
I20260709 13:06:41.959079 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:41.959493  3140 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.959669 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:41.960098  3140 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:41.960183 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:41.960419 29064 raft_consensus.cc:2368] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20.
I20260709 13:06:41.960657  3140 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:41.962776  3140 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.963244  3140 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:41.963425  3489 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:41.964150  3489 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:41.965417 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 3012) took 1278 ms. Trace:
I20260709 13:06:41.965803 29142 rpcz_store.cc:276] 0709 13:06:40.686920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.687100 (+   180us) service_pool.cc:224] Handling call
0709 13:06:41.965226 (+1278126us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b54002fa3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:41.965334 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81522048}
I20260709 13:06:41.956292 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:41.968225 29072 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:41.972721 29907 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:41.969085 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:41.959602 29186 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:41.974740 29905 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:06:41.979722 29902 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 13:06:41.977777 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 475870us
I20260709 13:06:41.995214 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:41.998432 29143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.024s	sys 0.015s
W20260709 13:06:41.954607  3479 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 1.567s	user 0.000s	sys 0.011s
W20260709 13:06:42.004503 29143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.024s	sys 0.015s
I20260709 13:06:42.004562 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.001931 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:42.020174 29072 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:42.020398 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:42.021188 29911 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:06:42.024835 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:42.026691  3473 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:42.027235  3473 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:42.027684  3473 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:42.028782  2804 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:41.980437  3440 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:06:42.004447 29112 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 1.903s (SENT) [suppressed 8 similar messages]
I20260709 13:06:42.038158 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.039002 29072 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:42.034566 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:42.041904 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.042598 29221 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:42.043809 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.044369 29221 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:06:42.023833  3506 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:42.048182 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:42.023247  3349 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:42.000607  3489 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:06:42.051188 29907 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:42.033529  3440 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.048583 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.052436 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:41.963722  3140 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:42.059551 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:42.065522 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.066699 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:42.068293 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:42.050992  3506 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.068068 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:42.071432 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:42.051388  3353 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:42.073282 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.073567 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:42.074640 29147 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:42.057806  3440 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:42.080503 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.102437 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:42.103322 29148 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:42.062925  3470 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:42.110464 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:42.076437  3506 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:42.115110 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.058157  3349 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:42.117527 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:42.126699 29139 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:06:42.081792  3353 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:42.115361 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.130134 29072 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:42.130334 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:42.131629 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:42.132509 29218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 3085) took 1516 ms (client timeout 1818 ms). Trace:
I20260709 13:06:42.132694 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:42.133052 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.134367 29070 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:42.135084 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2887) took 1356 ms. Trace:
I20260709 13:06:42.136256 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:42.137408 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.135496 29068 rpcz_store.cc:276] 0709 13:06:40.778697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.836789 (+ 58092us) service_pool.cc:224] Handling call
0709 13:06:40.837232 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.411438 (+574206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.134844 (+723406us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b5400069ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:42.135004 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317536640}
W20260709 13:06:42.132928 29218 rpcz_store.cc:269] 0709 13:06:40.615598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.619816 (+  4218us) service_pool.cc:224] Handling call
0709 13:06:42.132308 (+1512492us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b54000bd954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:42.132436 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1439728512}
I20260709 13:06:42.143859 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:42.134379 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.125373  3451 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.134917  3140 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:42.116052  3470 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:42.154150 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:42.156276 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.154119 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 1043) took 2131 ms. Trace:
I20260709 13:06:42.157567 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:42.157160 29906 rpcz_store.cc:276] 0709 13:06:40.023121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.023299 (+   178us) service_pool.cc:224] Handling call
0709 13:06:40.023677 (+   378us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:40.024150 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:40.024180 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:40.024790 (+   610us) log.cc:844] Serialized 52 byte log entry
0709 13:06:40.133048 (+108258us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:40.192824 (+ 59776us) log.cc:844] Serialized 14 byte log entry
0709 13:06:40.888357 (+695533us) spinlock_profiling.cc:243] Waited 5.31 ms on lock 0x7b1800399040. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:06:41.141227 (+252870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.141335 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:41.954008 (+812673us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:42.150486 (+196478us) raft_consensus.cc:1685] finished
0709 13:06:42.150516 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.150682 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104725,"prepare.queue_time_us":60249,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":11732,"raft.queue_time_us":755750,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":36709,"spinlock_wait_cycles":114058368,"thread_start_us":360224,"threads_started":2,"wal-append.queue_time_us":286218}
I20260709 13:06:42.170444 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3003) took 1887 ms. Trace:
I20260709 13:06:42.153294  3353 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 13:06:42.152427  2804 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:42.174984  3112 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:42.170888 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 401227us
I20260709 13:06:42.175454 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:42.175976 29113 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:42.176812 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:42.178360 29144 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:42.172190 29146 rpcz_store.cc:276] 0709 13:06:40.283410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.283541 (+   131us) service_pool.cc:224] Handling call
0709 13:06:40.310556 (+ 27015us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:06:40.584552 (+273996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:40.584584 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:06:40.605927 (+ 21343us) log.cc:844] Serialized 78 byte log entry
0709 13:06:40.818596 (+212669us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:41.786404 (+967808us) log.cc:844] Serialized 14 byte log entry
0709 13:06:42.166936 (+380532us) log.cc:844] Serialized 14 byte log entry
0709 13:06:42.170067 (+  3131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.170148 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.170197 (+    49us) raft_consensus.cc:1685] finished
0709 13:06:42.170210 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.170377 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20488,"prepare.queue_time_us":358332,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":255045,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":38008,"thread_start_us":1579721,"threads_started":4,"wal-append.queue_time_us":788290}
I20260709 13:06:42.180708  3402 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:42.183032  3402 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:42.184849  3326 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:42.188103 29904 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:42.190464  3440 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:42.191721 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:42.193495 29144 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:42.170928  3477 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:42.199018  3112 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:42.199213  3474 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.784s	user 0.033s	sys 0.030s
I20260709 13:06:42.200407 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:42.201201  3536 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:06:42.201311  3124 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:42.201603  3440 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:42.202208 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:42.202507 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 2890) took 1258 ms. Trace:
I20260709 13:06:42.202800 29216 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:42.203945 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.204298 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2892) took 2337 ms. Trace:
I20260709 13:06:42.204571 29066 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:42.204702 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.205343 29072 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:42.205448 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.206063 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.206197 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.206561 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:42.206789 29072 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:06:42.206964 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.207217 29066 raft_consensus.cc:2368] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22.
I20260709 13:06:42.207537 29071 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:42.208297 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.201836  3124 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.209353  3124 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:42.209996  3538 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:06:42.209368  3349 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:42.202909 29065 rpcz_store.cc:276] 0709 13:06:40.944257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.944423 (+   166us) service_pool.cc:224] Handling call
0709 13:06:41.006779 (+ 62356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:41.380005 (+373226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:41.380037 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:41.380508 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 13:06:41.817842 (+437334us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:41.817896 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:41.961258 (+143362us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b5400208ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:41.961301 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.202222 (+240921us) raft_consensus.cc:1685] finished
0709 13:06:42.202242 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.202449 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11446,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":32722,"spinlock_wait_cycles":857314560,"thread_start_us":436911,"threads_started":1,"wal-append.queue_time_us":697607}
I20260709 13:06:42.215325  3349 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:42.215626 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:42.215953  3349 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:42.216969 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:42.217718 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.218796  3349 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:42.219664 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.220541 29220 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:06:42.221771 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.221830  3470 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:42.223166 29144 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:06:42.223691 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.226082 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:42.226114 29222 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:42.217713 29221 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:42.228823 29111 leader_election.cc:400] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:42.230935 29111 leader_election.cc:403] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:06:42.232092 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:42.240139 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:42.204676 29073 rpcz_store.cc:276] 0709 13:06:39.867272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.868613 (+  1341us) service_pool.cc:224] Handling call
0709 13:06:39.888925 (+ 20312us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:39.893561 (+  4636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:39.893612 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:39.894418 (+   806us) log.cc:844] Serialized 52 byte log entry
0709 13:06:40.082907 (+188489us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:40.282046 (+199139us) log.cc:844] Serialized 14 byte log entry
0709 13:06:40.282309 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:40.282410 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.153510 (+1871100us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:42.203988 (+ 50478us) raft_consensus.cc:1685] finished
0709 13:06:42.204083 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.204217 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13492,"prepare.queue_time_us":90525,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":20980,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":9642,"spinlock_wait_cycles":64077568,"thread_start_us":261198,"threads_started":2,"wal-append.queue_time_us":490164}
I20260709 13:06:42.161134  3252 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.277956 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.278650 29070 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
W20260709 13:06:42.280346 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 47540.7us
I20260709 13:06:42.234107 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:42.275216  3495 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
W20260709 13:06:42.285941 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 54139.4us
I20260709 13:06:42.286711 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:42.281286 29113 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:42.285233  3508 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:42.293634 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:42.293147 29141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2892) took 1963 ms (client timeout 2175 ms). Trace:
I20260709 13:06:42.288713 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:42.294534 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:42.292987 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 64285.3us
I20260709 13:06:42.296078 29111 leader_election.cc:400] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20.
I20260709 13:06:42.296619 29111 leader_election.cc:403] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:42.294289 29902 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:42.297629 29221 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:42.305927  3508 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:42.294289 29141 rpcz_store.cc:269] 0709 13:06:40.329771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:40.347292 (+ 17521us) service_pool.cc:224] Handling call
0709 13:06:42.292938 (+1945646us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b540027d8d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:42.293088 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4109565184}
I20260709 13:06:42.152940  3451 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:42.309342 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 2904) took 1227 ms. Trace:
I20260709 13:06:42.313937 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:42.313166 29143 rpcz_store.cc:276] 0709 13:06:41.082961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.170831 (+ 87870us) service_pool.cc:224] Handling call
0709 13:06:41.171232 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:41.306609 (+135377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:41.306657 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:41.342404 (+ 35747us) log.cc:844] Serialized 26 byte log entry
0709 13:06:42.005213 (+662809us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:42.005624 (+   411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.289095 (+283471us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54001cc354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:42.289145 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.289200 (+    55us) raft_consensus.cc:1685] finished
0709 13:06:42.289220 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.309253 (+ 20033us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18246,"raft.queue_time_us":18414,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":32008,"spinlock_wait_cycles":267619200,"thread_start_us":662422,"threads_started":1,"wal-append.queue_time_us":807256}
I20260709 13:06:42.279704  3402 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:42.238979  3440 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:42.324517 29037 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:42.315158  3464 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:42.342450 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 1093) took 1242 ms. Trace:
I20260709 13:06:42.287305  3514 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:42.342890 29904 rpcz_store.cc:276] 0709 13:06:41.099998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.146916 (+ 46918us) service_pool.cc:224] Handling call
0709 13:06:42.342277 (+1195361us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54007c01d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:42.342388 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025164160}
I20260709 13:06:42.293004  3529 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:42.350378 29217 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:42.341955  3464 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:42.350754 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.351456 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.352053 29216 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
W20260709 13:06:42.360001 29875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.043s	sys 0.015s
W20260709 13:06:42.374712 29875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.047s	sys 0.017s
W20260709 13:06:42.375098 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 623408us
I20260709 13:06:42.376498 29070 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 22: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:06:42.376823 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 62943.1us
I20260709 13:06:42.318238  3365 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:42.320504  2804 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:06:42.376216 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 57061.1us
I20260709 13:06:42.378259 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:42.394903 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:42.396013 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:42.328442  3353 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:42.407708 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:42.412664 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.238698  3477 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:42.413643 29071 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:42.354058  3529 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:42.415261 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 410813us
I20260709 13:06:42.416374 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.379290  3489 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:42.308810  3252 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:42.424217 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:42.403052 29215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.017s	sys 0.026s
I20260709 13:06:42.374440  3464 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:42.445892 29215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.017s	sys 0.026s
I20260709 13:06:42.447157 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:42.422905 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:42.344424  3402 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.397749  3494 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:42.394421  2804 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:42.454850 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.403789  3498 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:42.402705  3244 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:42.453552 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.343868  3539 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.113s
I20260709 13:06:42.348500  3440 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:42.482491 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:42.428646  3112 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:42.437444  3529 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:42.459039 29065 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:42.436563  3489 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:42.445928  3450 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.014s	sys 0.048s
I20260709 13:06:42.489665 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:42.512720 29218 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:42.381801  3514 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:42.491071 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.451977  3494 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.451514  3412 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:42.458300  2804 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:42.522560 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.440481  3124 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:42.523663 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:42.532286 29114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.017s	sys 0.041s
W20260709 13:06:42.555177 29114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.017s	sys 0.041s
I20260709 13:06:42.458562  3402 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:42.436685  3353 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:42.543043  3545 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:06:42.520627  3244 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:42.512383  3489 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:42.530946  3495 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:42.532928  3494 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:42.548647  3451 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:42.549693  3498 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:06:42.512483  3450 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.014s	sys 0.049s
I20260709 13:06:42.454857  3506 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:42.527073 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:42.552250 29148 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:42.493777  3549 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:42.355294  3490 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:42.583472 29218 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:06:42.557847  3112 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:42.558418  3476 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:42.613127 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 86177.4us
I20260709 13:06:42.603863  3498 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.630978 29220 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:42.629817 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:42.604772  3297 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:42.607128  3499 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:42.635962 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:42.612509  3112 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:42.617801  3476 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:42.607823  3549 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:42.651551 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 242395us
W20260709 13:06:42.645371 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 269424us
I20260709 13:06:42.619246 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:42.627974  3539 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:42.656999 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 1055) took 1355 ms. Trace:
I20260709 13:06:42.578051  3495 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:42.666101 29908 rpcz_store.cc:276] 0709 13:06:41.289685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.289830 (+   145us) service_pool.cc:224] Handling call
0709 13:06:41.329129 (+ 39299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:41.329433 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:06:41.879801 (+550368us) log.cc:844] Serialized 14 byte log entry
0709 13:06:42.644860 (+765059us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:42.644945 (+    85us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:06:42.644999 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.645064 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.645210 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":586876,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":18061,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":15080,"thread_start_us":1295254,"threads_started":3,"wal-append.queue_time_us":678202}
I20260709 13:06:42.617420  3349 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:06:42.635792  3498 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:06:42.677315 29139 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.002s
I20260709 13:06:42.659891  3476 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:42.696019  3473 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.668s	user 0.011s	sys 0.002s
I20260709 13:06:42.703297  3514 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:42.703292  3473 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:42.652251  3542 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:42.706255  3542 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:42.706722  3542 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:42.707448  3490 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:42.665874  3539 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:42.685681  3544 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:06:42.667730  3555 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:06:42.674998  3180 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:42.696638  3508 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:42.696866  3554 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:06:42.698807  3451 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:42.700879  3450 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:42.701735  3520 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:42.702242  3353 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:42.684023  3244 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:42.705605  3495 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:42.708285  3541 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:42.659301  3543 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:42.655448  3297 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:42.709759  3349 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:42.658130  3112 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:42.682283 29139 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
W20260709 13:06:42.666087 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 243268us
I20260709 13:06:42.750234 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:42.713702 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 179422us
I20260709 13:06:42.754578 29185 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:42.666775  3549 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:42.731737 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:42.756748 29213 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:42.757637 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.758435 29213 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:06:42.759191 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.713302  3550 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:42.760212  3550 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:42.761976  3508 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:42.762410  3508 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:42.762583  3550 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:42.762817  3508 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:42.720343  3539 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:42.744503  3453 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:42.718472  3552 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:42.768297  3552 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:42.771515 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:42.773269 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:42.777439 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.778682 29064 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:42.782296 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.783633 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.771538  3552 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:42.787245 29213 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:42.796769 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:42.799096 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:42.797233  3539 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:42.811406  3539 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:42.844398 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 2949) took 1394 ms. Trace:
I20260709 13:06:42.743330  3349 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:42.737998  3541 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:42.831751 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 79833.7us
I20260709 13:06:42.740549  3543 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:43.013172  3543 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:43.012495  3349 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:42.836781 29902 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.001s
I20260709 13:06:43.014276 29902 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:06:42.819196  3440 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:42.749610  3252 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:42.732949  3450 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:42.748858 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 80611.8us
W20260709 13:06:43.018944 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 399772us
I20260709 13:06:42.813537  3363 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20.
I20260709 13:06:42.844967 29215 rpcz_store.cc:276] 0709 13:06:41.450224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.491982 (+ 41758us) service_pool.cc:224] Handling call
0709 13:06:41.515443 (+ 23461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:41.711187 (+195744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:41.711222 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:41.725814 (+ 14592us) log.cc:844] Serialized 26 byte log entry
0709 13:06:42.446364 (+720550us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:42.446433 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.446525 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:42.844088 (+397563us) raft_consensus.cc:1685] finished
0709 13:06:42.844131 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:42.844317 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":205291,"raft.queue_time_us":195625,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":27448,"thread_start_us":619895,"threads_started":1,"wal-append.queue_time_us":747884}
I20260709 13:06:42.825634  3566 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:42.832754  3550 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:42.825999  3353 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:42.842676  3552 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:42.738428  3451 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:42.907254  3544 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:06:42.907120  3304 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:42.912406  3546 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:42.928103  3568 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:06:42.929405  3556 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
W20260709 13:06:42.930289  3558 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
W20260709 13:06:42.931243  3540 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.375s	user 0.000s	sys 0.001s
W20260709 13:06:42.973809  3551 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 13:06:42.980856 29218 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
W20260709 13:06:42.983310  3548 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.333s	user 0.000s	sys 0.001s
I20260709 13:06:42.984714  3297 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:42.987002  3573 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:06:42.991051 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.109719 29217 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:42.913569  3480 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:43.006421  3549 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:42.995736  3571 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:43.016484 29111 leader_election.cc:400] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22.
I20260709 13:06:43.116541 29111 leader_election.cc:403] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:06:42.919902  3464 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.019770 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:43.120872 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 101190us
I20260709 13:06:43.121791 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:43.022064 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:43.023298 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:43.123565 29148 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:43.024451 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.031204 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:43.035660 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.010s	sys 0.054s
I20260709 13:06:43.025316  3393 raft_consensus.cc:2764] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:06:43.124979 29143 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
W20260709 13:06:43.110651  3529 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.003s
I20260709 13:06:43.038559 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.045822 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:43.132005 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 86257us
I20260709 13:06:43.050601 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:43.051015  3473 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.051297 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:43.032768  3349 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:43.136688 29903 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:43.032021  3494 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.092437 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.096055 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.097018 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.097715 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:43.100126 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:43.033869  3552 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:43.039466  3541 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:43.111907 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.028895  3566 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:43.092707  3304 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:43.117165 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 277826us
I20260709 13:06:43.042805  3223 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:43.125164 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.011s	sys 0.054s
I20260709 13:06:43.091840  3124 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:43.110481  3402 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.538s	user 0.014s	sys 0.001s
I20260709 13:06:43.120556 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3100) took 1801 ms. Trace:
I20260709 13:06:43.016083  3506 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.129698 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:43.130479 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 108542us
W20260709 13:06:43.130539 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 114337us
I20260709 13:06:43.160072 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:43.134011 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.161502 29911 raft_consensus.cc:2410] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:06:43.095618  3423 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:43.134748  3539 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:43.116238  3571 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.143019 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:43.127125  3508 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:43.153659 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 706556us
I20260709 13:06:43.101511  3546 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.035601  3353 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.184655  3353 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.163555 29039 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:43.135695  3500 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:43.160666 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:43.165156 29910 raft_consensus.cc:2368] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:06:43.151470 29909 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:43.138011  3537 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:43.102756  3550 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.144249  3140 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:43.131521  3405 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:43.170408 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:43.151013  3552 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:43.172271 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:43.179430 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.224972 29143 raft_consensus.cc:2368] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20.
I20260709 13:06:43.149135  3453 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:43.226099  3453 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:43.178005 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:43.177997 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 76324.8us
I20260709 13:06:43.227701 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:43.228323 29073 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:43.228729 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:43.233292 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.173236 29148 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:43.166224 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3011) took 2512 ms. Trace:
I20260709 13:06:43.181915 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.157054  3566 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:43.129951  3480 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.157816 29219 rpcz_store.cc:276] 0709 13:06:41.319618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.319774 (+   156us) service_pool.cc:224] Handling call
0709 13:06:41.343295 (+ 23521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:41.509397 (+166102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:41.509450 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:41.520668 (+ 11218us) log.cc:844] Serialized 26 byte log entry
0709 13:06:42.086140 (+565472us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:42.503508 (+417368us) log.cc:844] Serialized 14 byte log entry
0709 13:06:42.845923 (+342415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.988495 (+142572us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54003395d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:42.988543 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.113738 (+125195us) raft_consensus.cc:1685] finished
0709 13:06:43.120339 (+  6601us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.120493 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51350,"prepare.queue_time_us":543610,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":3953,"raft.queue_time_us":92867,"raft.run_cpu_time_us":9079,"raft.run_wall_time_us":222534,"spinlock_wait_cycles":343953664,"thread_start_us":1065510,"threads_started":3,"wal-append.queue_time_us":756813}
I20260709 13:06:43.203713 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.166282  3252 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:43.137413  3529 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.207660 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:43.231891 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.176354  3423 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.235683 29140 rpcz_store.cc:276] 0709 13:06:40.653700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:41.643884 (+990184us) service_pool.cc:224] Handling call
0709 13:06:41.644261 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:41.650120 (+  5859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:41.650166 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:41.650739 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 13:06:42.161868 (+511129us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:42.161922 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.162010 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.165996 (+1003986us) raft_consensus.cc:1685] finished
0709 13:06:43.166017 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.166155 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5512,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":8921,"thread_start_us":408357,"threads_started":1,"wal-append.queue_time_us":865048}
I20260709 13:06:43.241496 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:43.185921  3506 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:43.246650 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 68743.5us
I20260709 13:06:43.256350 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:43.267457 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59893.4us
I20260709 13:06:43.264163 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:43.250581 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 87127.4us
I20260709 13:06:43.161634  3223 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.252128 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.181208  3571 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:43.226994  3581 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22.
I20260709 13:06:43.224237  3180 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:43.164817  3402 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.261124 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.253387  3589 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:43.269100 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:43.266388 29113 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:43.253499  3252 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:43.238108  3500 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:43.256451  3520 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:43.243991  3586 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:43.295697 29907 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:43.281378 29905 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:06:43.283336 29911 raft_consensus.cc:2368] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23.
I20260709 13:06:43.250932  3451 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:43.259380  3405 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:43.220566  3537 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:43.311158  3583 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 13:06:43.306115  3489 consensus_meta.cc:287] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.726s	user 0.013s	sys 0.001s
I20260709 13:06:43.312357  3489 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:43.314350  2804 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.755s	user 0.005s	sys 0.005s
I20260709 13:06:43.314785  2804 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.314996 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.316129  3453 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.317348  3223 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.318346  3464 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.319072  3464 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:43.319473  3464 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:43.319588  3549 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.319941  3464 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:43.321908  3402 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.323303  3546 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.276767  3180 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:43.323690 29217 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:43.324249  3180 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:43.325763  3494 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.326531  3494 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:43.327885 29073 raft_consensus.cc:2368] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20.
I20260709 13:06:43.297778  3347 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.328862  3347 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.279285  3589 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.330226  3589 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:43.332273  3450 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:43.333901  3508 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:43.334309  3529 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.337195 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.337396  3480 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:43.337879 29072 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:43.338737  3500 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.342535 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.343122 29217 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:43.344000 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.344631 29220 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:43.280891  3591 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:43.347054 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.348245 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.348802 29216 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:06:43.349774  3476 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.653s	user 0.011s	sys 0.000s
I20260709 13:06:43.350183  3476 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.352578  3561 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:06:43.353040  3498 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.668s	user 0.004s	sys 0.007s
I20260709 13:06:43.353096  3561 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.353703  3561 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:43.357678 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:43.357208 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:43.286363  3252 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:43.371361 29905 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:43.353601  3498 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.366412 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.374989 29213 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:43.273341  3506 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.287570  3520 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:43.361996  3490 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.372809 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.379451 29216 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:06:43.278479  3112 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.380734 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.355347  3347 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.372043  3529 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:43.368832  3480 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.388662 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:43.400815 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 131785us
I20260709 13:06:43.418200 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:43.376870 29217 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:43.412658 29146 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:43.414841 29213 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
W20260709 13:06:43.418633 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 150373us
I20260709 13:06:43.387219  3520 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:43.405699  3542 consensus_meta.cc:287] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.001s
I20260709 13:06:43.384928  3140 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:43.432631 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.438848 29902 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
W20260709 13:06:43.440176 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 98904.5us
I20260709 13:06:43.443073 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:43.444502 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.447536 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:43.448179 29072 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:43.451030  3566 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.451538  3508 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:43.452071  3508 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.454334  3508 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
W20260709 13:06:43.455348  3585 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.282s	user 0.001s	sys 0.001s
I20260709 13:06:43.460397 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:43.461117 29910 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:43.427248  3539 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:43.430353  3476 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.423758  3546 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:43.418279  3529 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:43.432742  3412 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:43.461638  3537 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.464128  3584 raft_consensus.cc:2764] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:06:43.433936  3480 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:43.500861  3365 consensus_meta.cc:287] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.101s	user 0.013s	sys 0.000s
I20260709 13:06:43.501792  3365 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
W20260709 13:06:43.502799 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 72925.1us
I20260709 13:06:43.406522  3490 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.506315  3490 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.306788  3586 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.509395  3586 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:43.480068 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:43.545091 29112 leader_election.cc:400] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:06:43.545491 29112 leader_election.cc:403] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:06:43.460402 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.542212 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:43.515615 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.476902  3546 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:43.548755  3495 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:43.549885  3546 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.552278 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.553545 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:43.554653 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.556272 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:43.539023 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 120919us
I20260709 13:06:43.558071 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:43.555796 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:43.571049 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 66810.6us
I20260709 13:06:43.578675 29902 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:06:43.587653 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.588653 29145 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
W20260709 13:06:43.594029 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 248934us
I20260709 13:06:43.511384  3477 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.602698 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.610270 29070 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
W20260709 13:06:43.611100 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 53145.6us
I20260709 13:06:43.604254 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.605012 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.611881 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:43.464532  3574 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
W20260709 13:06:43.606462 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 340121us
I20260709 13:06:43.615031 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:43.596889 29072 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:43.613049 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:43.615164 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.616004 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:43.619803 29038 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:43.623786 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.626224 29213 raft_consensus.cc:2368] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22.
I20260709 13:06:43.628901 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.629631 29213 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:43.518013  3580 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:43.617063 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:43.632238 29113 leader_election.cc:400] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20.
I20260709 13:06:43.632594 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:43.618778 29146 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:43.487402  2804 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.503727  3480 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.478334  3506 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:43.599890  3447 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:43.523211  3304 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.648075 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.648236 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.648893 29216 raft_consensus.cc:2368] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15.
I20260709 13:06:43.633266 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:43.482426  3529 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:43.490025  3112 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.555250  3326 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.636487 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.485929  3498 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.480803  3423 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.659758 29147 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:43.638017 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 429613us
I20260709 13:06:43.633438 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:43.606237  3363 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:06:43.483397  3412 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.646914 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:43.643291 29185 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:43.512362  3464 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.511642  3180 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.653728 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.676699 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:43.656545 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:43.678293 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.678550 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:43.643563 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:43.658442  3593 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:43.658601  3566 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.676864 29143 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:43.662134 29148 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 23
W20260709 13:06:43.682081 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 66185.4us
I20260709 13:06:43.664546 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:43.682934 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:43.666642 29910 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:43.661322 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.482328  3541 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.511492  3589 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.705662  3591 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:43.705849  3589 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.706557  3450 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:43.707136  3450 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:43.707235  3598 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:43.707760  3598 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.708156  3450 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.714049  3464 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.714854  3297 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:43.715422  3297 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:43.715857 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.716288  3297 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.717288 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.719614  3564 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:43.720152  3564 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:43.720983  3564 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.723348 29146 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:43.669556 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.447069  3542 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.655813  3575 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:43.729897 29068 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:43.680590 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 47413us
I20260709 13:06:43.650214  3580 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:43.688935 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 1058) took 1251 ms. Trace:
W20260709 13:06:43.726492 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 57367.9us
I20260709 13:06:43.730988 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.648885  3472 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:43.732412 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.731215 29904 rpcz_store.cc:276] 0709 13:06:42.438024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:42.438158 (+   134us) service_pool.cc:224] Handling call
0709 13:06:42.439030 (+   872us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:42.451014 (+ 11984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:42.451050 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:42.457463 (+  6413us) log.cc:844] Serialized 26 byte log entry
0709 13:06:42.665491 (+208028us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:42.665579 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:42.665676 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:43.688613 (+1022937us) raft_consensus.cc:1685] finished
0709 13:06:43.688644 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:43.688853 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1061,"raft.queue_time_us":3877,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":6077,"thread_start_us":198176,"threads_started":1,"wal-append.queue_time_us":385305}
I20260709 13:06:43.733711 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.683142 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:43.692306 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45530.9us
I20260709 13:06:43.639287  3561 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.697753  3180 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.698348 29140 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: []
W20260709 13:06:43.700030  3349 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.000s
I20260709 13:06:43.744838 29144 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:43.701623  3480 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:43.732453  3555 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:06:43.734232 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.735803 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:43.671190  3405 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:43.727061  3539 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:43.518311  3140 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.663049  3495 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:43.733042  3575 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:43.774686 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 91622.7us
I20260709 13:06:43.733373 29147 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:06:43.749342 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:43.703936  3541 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.749981 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:43.775552 29874 leader_election.cc:400] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15.
I20260709 13:06:43.751030 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.776027 29874 leader_election.cc:403] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:43.753953 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:43.776180 29066 raft_consensus.cc:2368] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
I20260709 13:06:43.759605 29071 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:43.762658 29148 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
W20260709 13:06:43.759677 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 95215.7us
I20260709 13:06:43.668437  3326 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.774561 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:43.688870  3477 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:06:43.700639 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 57125us
I20260709 13:06:43.791508 29186 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:43.734313 29145 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:43.778775 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:43.788945 29113 leader_election.cc:400] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:43.796706 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:43.798293 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.799383 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.800165 29213 raft_consensus.cc:2368] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:06:43.801900 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.802747 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:43.804059 29904 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
W20260709 13:06:43.808437 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 54586.1us
I20260709 13:06:43.810439 29113 leader_election.cc:400] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16.
I20260709 13:06:43.817922 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.805176 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.820701 29875 leader_election.cc:400] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20.
I20260709 13:06:43.821076 29875 leader_election.cc:403] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:06:43.818987 29145 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:43.834599 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.740710  3167 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:06:43.835304 29146 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:43.836390 29221 raft_consensus.cc:2368] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23.
I20260709 13:06:43.747726  3472 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.789345  3575 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:43.791949  3589 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 14. Current term is 15.
I20260709 13:06:43.790759  3477 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.840477 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:43.841239 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:43.841531 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.790874  3180 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:43.842034 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.850929 29148 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:43.791219  3423 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:43.863978 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.864430  3423 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:43.864934  3423 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:43.865329 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.865553 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.866385 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.866532 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.866993 29140 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:43.867570 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.839767 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.867750 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:43.847561 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 185893us
I20260709 13:06:43.894002 29038 leader_election.cc:400] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22.
I20260709 13:06:43.794479  3365 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:43.831331  3405 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:43.853739  3595 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.181s
I20260709 13:06:43.855067 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.855962 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.858031  3598 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.860198  3533 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:43.834038  3547 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:43.869061  3542 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.869735 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:43.933071  3603 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:06:43.933707 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.934459 29220 raft_consensus.cc:2368] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15.
I20260709 13:06:43.935209 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.872217  3550 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.873413  3412 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.938135 29142 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:43.938508 29068 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:43.929929 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:43.878139  3180 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:43.940187  3180 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:43.875299 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.876328  3326 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:43.868892  3560 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:43.841287 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:43.868098  3604 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20.
I20260709 13:06:43.797914  3489 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.868259 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:43.848239 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 48574.4us
I20260709 13:06:43.958832 29219 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:43.894891 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:43.929785  3579 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:06:43.894940 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.961334 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:43.962273 29216 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:43.897186 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 56814.3us
I20260709 13:06:43.883100 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:43.902101 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.964010 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:43.902657 29140 raft_consensus.cc:2368] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:06:43.904075 29038 leader_election.cc:403] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:06:43.757429  3571 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.977330  3571 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:43.978816  3561 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:43.817658  3480 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:43.979826  3604 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:43.980607  3604 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:43.981146  3604 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:43.982283  3605 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 13:06:43.984133  3552 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.743s	user 0.006s	sys 0.005s
I20260709 13:06:43.984658  3552 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.910171 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:43.908758 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:43.986259 29145 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:43.837358  3297 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:43.996548 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:43.997419 29903 raft_consensus.cc:2393] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:43.944430  3405 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.998417 29186 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:43.999483  3480 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:44.000041  3480 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:43.922186 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.001256 29905 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:43.928390 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:43.928400 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:44.004388 29221 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:06:44.009675 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 81380.5us
I20260709 13:06:44.010064 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:43.879081  3477 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:43.833359  3580 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:43.995115  3575 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:44.021682 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 127839us
I20260709 13:06:44.021926  3541 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:44.023273  3575 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:06:43.823257  3252 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:43.950717 29111 leader_election.cc:400] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
I20260709 13:06:44.025755 29111 leader_election.cc:403] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:06:44.025856  3252 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:44.026540 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 76891.3us
I20260709 13:06:44.026706  3365 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:44.027288 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:43.951793 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.028625  3495 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:44.029136 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:43.952960 29217 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
W20260709 13:06:43.929026  3543 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.915s	user 0.004s	sys 0.009s
I20260709 13:06:43.956092 29071 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:44.031395  3543 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.034704  3543 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:44.035009  3597 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:43.958869  3167 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 13:06:44.035812  3597 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:44.037264  3597 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:44.000622  3480 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:43.980810 29222 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:43.988345 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.040798 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:44.041661 29907 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:43.990156 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.990725 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:43.991123 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.045827 29908 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:43.993989 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:44.047364 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.011065 29188 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.003883 29186 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:44.048741 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44993.6us
I20260709 13:06:43.947394 29113 leader_election.cc:400] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23.
I20260709 13:06:44.049652 29113 leader_election.cc:403] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:06:44.050163 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 103032us
W20260709 13:06:44.030851 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 147887us
I20260709 13:06:43.911523 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:44.051502 29139 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:44.051291 29113 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.051069 29185 leader_election.cc:400] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20.
I20260709 13:06:44.052727 29185 leader_election.cc:403] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:06:44.057126 29902 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:43.968183  3533 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:44.057528 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:44.066823 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55890.9us
I20260709 13:06:44.052259 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:44.072309 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:44.079108 29068 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:44.072645 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:44.073789 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:44.081727 29217 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:44.073503 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:44.082499 29069 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:44.075881 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:44.076457 29072 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:44.074940 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.075265 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.069000 29071 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:44.089771 29113 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.069002 29145 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:44.090278 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:44.090741 29905 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:44.074371  3365 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
I20260709 13:06:44.078079  3560 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:44.093279  3560 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.094185  3495 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:44.071883  3520 consensus_meta.cc:287] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.000s
I20260709 13:06:44.096987  3520 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:44.086589  3514 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:44.097868  3514 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:44.098227  3514 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:44.099201  3520 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:44.100250 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:44.100176  3611 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.084s
I20260709 13:06:44.101433  3611 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:44.101780  3611 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:44.102613  3611 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.105144 29146 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:06:44.092419  3608 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:44.110447 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:44.110567 29221 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:06:44.051031  3571 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:44.113801 29065 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:44.114297 29070 raft_consensus.cc:1240] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:44.112706  3596 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.199s
I20260709 13:06:44.117647  3596 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:44.117517  3591 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:06:44.118053  3596 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:44.118552 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:44.070164  3625 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:06:44.115031 29871 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.130020 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:44.063076  3549 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23.
I20260709 13:06:44.095726  3560 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:44.120028 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.124769 29068 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:44.062619  3550 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:06:44.118554  3610 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:06:44.077880  3140 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:44.135128  3365 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:44.153416  3140 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.119338  3608 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.158010  3552 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:44.158385  3608 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:44.149323  3569 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:44.142850  3363 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:44.160739  3564 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:44.136755 29216 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:44.137087 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:44.149505 29112 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:44.164027 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:44.138934 29039 leader_election.cc:400] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:06:44.019577  3494 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.094s
I20260709 13:06:44.063740  3326 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:44.118956  3473 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:44.072814  3472 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:44.140573 29113 leader_election.cc:400] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15.
I20260709 13:06:44.184412 29113 leader_election.cc:403] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:06:44.087329  3499 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.131125 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.164849 29039 leader_election.cc:403] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:06:44.184859 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:44.185624 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46833.5us
I20260709 13:06:44.186028 29066 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:44.186712 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:44.142884 29148 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:44.189563 29148 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:44.113293  3450 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.170939 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:44.077837  3547 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20.
I20260709 13:06:44.176780 29214 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:44.177755 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:44.130687  3495 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:44.141309  3611 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:44.195080 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:44.197468 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:44.199651 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 59191.2us
I20260709 13:06:44.155844  3140 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:44.225988  3595 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:44.128815  3596 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:44.227264  3480 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.228344  3596 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:44.228988 29214 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:06:44.200475 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:44.203470 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:44.128489  3477 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.205960 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:44.189091  3423 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.179261  3549 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:44.150348  3533 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:44.210523 29221 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
W20260709 13:06:44.217458  3529 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.000s
I20260709 13:06:44.221352 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:44.166481  3439 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.200460 29113 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.229804  3480 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:44.237656 29219 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:44.241945 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:44.262658 29148 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:44.266949 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:44.197542  3560 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:06:44.270746 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:44.233711  3489 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.232597  3326 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:44.274917 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 143864us
I20260709 13:06:44.276770 29143 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:44.277148 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.244902  3495 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:44.239537  3598 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 14. Current term is 15.
I20260709 13:06:44.281057 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:44.281275 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:44.281973 29148 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:44.284392 29147 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:44.283772 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 83389.4us
I20260709 13:06:44.285550 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:44.285676 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.289131 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:44.279598 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.291530 29066 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:06:44.292413 29068 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:44.232019  3450 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:44.299146 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.300928 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.301261 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:44.301625 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 59819us
I20260709 13:06:44.320645 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.321563 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.298862  3447 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:44.256856  3529 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.278103  3549 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:44.325258 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:44.325366 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:44.298375  3537 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:44.265344  3439 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:44.327406 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.298004  3508 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:44.300426  3494 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:44.339181 29216 raft_consensus.cc:2368] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:06:44.303326  3480 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.312089  3561 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:44.306687  3541 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22.
W20260709 13:06:44.348014 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 45683.2us
I20260709 13:06:44.332801 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:44.318286  3140 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:06:44.416620  3529 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:44.334335 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.323356  3472 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:44.326155  3566 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:44.417692  3472 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:44.329890  3550 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:44.418146  3472 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:44.418362  3550 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:44.418854  3550 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:44.421582  3514 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:44.422518 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1162) took 1253 ms. Trace:
I20260709 13:06:44.422681  3547 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:44.423074  3547 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:44.422945 29139 rpcz_store.cc:276] 0709 13:06:43.169490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.169613 (+   123us) service_pool.cc:224] Handling call
0709 13:06:43.190357 (+ 20744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.052010 (+861653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.422283 (+370273us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b5400041354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:44.422456 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1979972864}
I20260709 13:06:44.423539  3547 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:44.336949  3463 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:44.424250  3463 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:44.425184  3463 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.390720  3549 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.348307  3495 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:44.366394  3537 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:44.432025  3450 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:44.432934  3537 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:44.434298  3623 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.413s	user 0.002s	sys 0.000s
W20260709 13:06:44.404522  3628 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:06:44.437337 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.438067 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.438521 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:44.439215 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.336206  3552 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:44.423708  3514 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:44.440897  3552 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:44.442613  3552 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.340345  3571 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:44.444968  3571 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:44.445523  3571 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:44.440451 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.438774 29910 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 22: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:44.332616  3451 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:44.412715 29909 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:44.327867  3584 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:44.473539 29112 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.338409  3326 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:44.413503 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 64300.6us
W20260709 13:06:44.414064  3631 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.002s
I20260709 13:06:44.415230  3477 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:44.415028  3604 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.322625  3180 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:44.447784 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 113526us
I20260709 13:06:44.529250 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:44.438454 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.450006 29221 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:06:44.450482 29218 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:44.450661 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:44.431597 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:44.453739 29910 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:44.464584 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.533723 29905 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 15: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:44.465394 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.466670 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:44.466856 29148 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:06:44.467661 29908 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:06:44.468290 29140 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:44.468672 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.536643 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:44.467715 29147 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:44.469834 29902 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:44.487040  3439 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:44.488330 29141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 1163) took 1318 ms (client timeout 1736 ms). Trace:
I20260709 13:06:44.486413 29220 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:06:44.473783  3584 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:44.511972 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 179218us
I20260709 13:06:44.512494 29112 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.524443 29073 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
W20260709 13:06:44.544615 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 79307.8us
I20260709 13:06:44.469174 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:44.549137 29145 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:44.537838 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 106316us
W20260709 13:06:44.538798 29141 rpcz_store.cc:269] 0709 13:06:43.169828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.172022 (+  2194us) service_pool.cc:224] Handling call
0709 13:06:44.488111 (+1316089us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400342454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:44.488267 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2761991552}
I20260709 13:06:44.554215 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:44.558661  3450 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.560026  3539 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:44.560602  3539 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:44.539783  3393 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:06:44.565961 29069 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:44.571664 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:44.572443 29064 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:44.574942 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:44.577605  3167 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:44.541563  3591 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:06:44.551965  3589 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:06:44.581337 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:44.552000  3480 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:44.582919  3480 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:44.583307 29071 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:44.583415  3480 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:44.555759  3477 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:44.581444  3589 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:44.586233  3589 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:44.587666  3589 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.589782  3589 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:44.559640  3495 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:44.595780  3304 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:44.596453  3304 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:44.598783  3347 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:44.588145 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:44.546063  3453 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
W20260709 13:06:44.599879  3629 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:06:44.613976  3180 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:44.600337 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.601460  3450 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:44.615646  3450 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:44.615453  3590 consensus_queue.cc:1059] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:06:44.616613  3450 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:44.620012 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 63206.2us
W20260709 13:06:44.621237 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.624806 29903 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:06:44.624886 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 83147.8us
I20260709 13:06:44.538229  3405 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.631109 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:44.543411  3584 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:44.538704  3600 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.539448  3575 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:44.634275 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:44.634701 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:44.635617 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:44.636049 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:44.642612 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:44.628774 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:44.700773 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:44.702042 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:44.703157 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 74510us
I20260709 13:06:44.700773 29147 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
W20260709 13:06:44.654470 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 322286us
I20260709 13:06:44.656554 29066 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:06:44.657001 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:44.601830  3423 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:44.631042  3529 raft_consensus.cc:3055] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:06:44.607481  3514 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:44.611348  3550 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:44.664801  3571 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:44.666278 29146 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:06:44.666612 29142 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:06:44.743017 29142 pending_rounds.cc:85] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 9
I20260709 13:06:44.743305 29142 pending_rounds.cc:107] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:06:44.743562 29142 raft_consensus.cc:2889] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:44.668104  2804 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:44.606539  3547 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:44.726977 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70151.5us
I20260709 13:06:44.542631  3528 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.126s
I20260709 13:06:44.731570 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:44.728166 29139 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:06:44.745172 29072 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:44.725025 29140 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:44.747301  3494 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:06:44.726125 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 83603.9us
I20260709 13:06:44.702554  3349 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.728305  3451 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:44.667352  3611 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.081s
I20260709 13:06:44.725742  3529 raft_consensus.cc:740] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:44.645041  3546 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.100s
W20260709 13:06:44.772169 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:44.741578  3564 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:44.748872  3514 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:44.768909  3451 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:44.773523  3599 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.007s	user 0.015s	sys 0.024s
W20260709 13:06:44.790154 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 147682us
I20260709 13:06:44.799033  3495 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:06:44.812889 29902 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:06:44.817711 29911 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:06:44.759382  3598 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:06:44.774566  3592 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:06:44.772712  3349 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:44.771895  3560 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 13:06:44.790786  3580 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.000s	sys 0.016s
W20260709 13:06:44.796826  3599 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.016s	sys 0.024s
I20260709 13:06:44.794667  3451 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:44.787879  3223 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:06:44.793485  3528 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:06:44.829695  3500 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:44.840709 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 66077.8us
W20260709 13:06:44.834013  3580 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.000s	sys 0.017s
I20260709 13:06:44.872354 29188 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.872941 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:44.830024  3140 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:06:44.830140  3590 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.034s
I20260709 13:06:44.830389  3353 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
W20260709 13:06:44.882128 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 130085us
I20260709 13:06:44.890413 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.891165 29904 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:44.895246 29112 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.896443 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:44.898607 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:44.899528 29908 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 22: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:06:44.958812 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 79097.6us
I20260709 13:06:44.960140 29188 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:44.893617  3494 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:44.883174  3450 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:44.864447  3500 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:44.969471 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 69339.5us
I20260709 13:06:44.886512  3528 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:06:44.979979 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 87102us
I20260709 13:06:44.836884  3529 consensus_queue.cc:260] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.011686 29186 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:44.988842 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:44.995415  3641 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:06:44.988925  3541 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:45.005034  3596 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.885175  3480 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:44.888330  3646 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.154s
I20260709 13:06:44.989049  3470 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:44.982184  3499 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:06:45.059717  3644 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:06:45.060029  3642 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:06:45.060174  3447 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:44.989511  3566 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.075s
I20260709 13:06:45.061779  3653 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:44.888914  3514 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:44.965734  3549 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.090s
I20260709 13:06:45.065444  3480 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:45.051877  3541 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.099452  3541 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:45.038794  3575 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:06:45.056386  3596 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.102964  3596 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:44.991053  3550 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:45.067886  3627 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.637s	user 0.010s	sys 0.010s
I20260709 13:06:45.058184  3571 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:45.080698  3584 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.079497  3470 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:45.105188 29222 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:45.105369 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:45.105967  3647 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.105s	user 0.002s	sys 0.000s
I20260709 13:06:45.106302  3584 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:45.107568  3564 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:06:45.108203  3547 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:45.108439  3653 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.110467  3349 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:45.110852  3653 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:45.072669  3637 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:06:45.107796 29222 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:45.072069  3643 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:06:45.081511  3546 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:45.069324  3167 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:45.069794  3633 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.635s	user 0.006s	sys 0.003s
I20260709 13:06:45.084249 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:45.074434 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:45.121783 29222 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:45.087488 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:45.076331 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2921) took 1418 ms. Trace:
I20260709 13:06:45.123208 29143 rpcz_store.cc:276] 0709 13:06:43.657450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.682592 (+ 25142us) service_pool.cc:224] Handling call
0709 13:06:45.076127 (+1393535us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b54001bc6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:45.076256 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1351338240}
I20260709 13:06:45.085404 29038 leader_election.cc:400] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:06:45.072120  3598 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:45.079845 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 68297.6us
I20260709 13:06:45.129274 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:45.075559  3638 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 13:06:45.079458  3423 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:06:45.074453  3472 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.000s
W20260709 13:06:45.071051  3632 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.628s	user 0.008s	sys 0.000s
I20260709 13:06:45.120909  3596 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:06:45.058892 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 70121.4us
I20260709 13:06:45.143534 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:45.143785 29222 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:06:45.077983 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:45.124794 29220 raft_consensus.cc:2368] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
I20260709 13:06:45.126925 29069 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:45.084653 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 96610.9us
I20260709 13:06:45.120163  3546 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:45.069108  3477 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:45.108237  3564 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:45.071889  3650 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.079s
I20260709 13:06:45.010911  3506 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 13:06:45.069950  3500 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:45.147145 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 403233us
I20260709 13:06:45.147483 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:45.083549  3304 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.170862  3304 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:45.171665  3657 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:45.172255  3657 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.174926  3657 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:45.161847 29903 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:06:45.176134  3564 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:45.176623  3528 consensus_queue.cc:1175] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a936878f59c840a6bd176a1d2d7347b1" responder_term: 18 status { last_received { term: 17 index: 15 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:45.081524  3494 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.150609  3611 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:45.179886 29148 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:45.169257 29218 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:06:45.140992  3598 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.177405  3564 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:45.161836 29220 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:45.167120  3646 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:06:45.149024  3596 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:45.181639 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:45.141798  3423 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:45.138226  3591 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:06:45.200626 29218 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:45.202280  3646 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:45.202754  3646 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:45.202951 29909 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 1108) took 1934 ms. Trace:
I20260709 13:06:45.203855  3646 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.205345  3646 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:45.206027  3593 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:45.206455  3593 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:45.208263  3575 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:45.156327  3477 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:45.196857  3546 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.227587  3656 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:45.228214  3656 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.228556  3596 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.230057  3596 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:45.230705  3656 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:45.231552  3500 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:45.231328  3656 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:45.195097 29220 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:06:45.207285  3593 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.235973  3593 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:45.211180  3480 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:45.200695  3591 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:45.240319  3591 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:45.142829  3451 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.218029 29902 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:45.203418 29909 rpcz_store.cc:276] 0709 13:06:43.268282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.269494 (+  1212us) service_pool.cc:224] Handling call
0709 13:06:43.524702 (+255208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.457546 (+932844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.904187 (+446641us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b5400023d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:45.202748 (+298561us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400023d50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:45.202889 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2445880576}
I20260709 13:06:45.226670  3550 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:45.145135  3565 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 13:06:45.148483  3472 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.232101  3500 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.225369  3423 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:45.247336 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 54070.8us
W20260709 13:06:45.247884 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.251482 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:45.252807 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:45.250890 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:45.260327 29220 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:06:45.265933 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:45.267135 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.270351 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:45.271174 29219 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:45.271293 29071 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:45.273245 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:45.275173 29218 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:45.241185  3591 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.286144 29071 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:45.288949 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:45.289716 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:45.292845 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:45.293282 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:45.303975  3589 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:45.307593  3451 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:45.311503  3539 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.313019  3539 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:45.257933  3550 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.259090  3566 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:06:45.295342  3660 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:06:45.326993  3472 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:45.328383 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.330927 29113 leader_election.cc:400] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
I20260709 13:06:45.368263 29113 leader_election.cc:403] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:06:45.359690 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:45.371388 29067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2933) took 1819 ms (client timeout 1989 ms). Trace:
I20260709 13:06:45.375329 29903 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:06:45.375394 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:45.333639  3604 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:45.298036  3651 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:45.381613 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:45.336174 29911 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:45.336367  3600 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:45.337241  3566 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:45.337608  3499 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:06:45.289029  3571 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:06:45.289089  3423 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:45.339453 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.351660  3550 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:45.354607  3592 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:45.355376  3353 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:45.361455  3595 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:06:45.372326 29067 rpcz_store.cc:269] 0709 13:06:43.551621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:43.614754 (+ 63133us) service_pool.cc:224] Handling call
0709 13:06:45.366046 (+1751292us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54001d9054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:45.371252 (+  5206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2916852608}
I20260709 13:06:45.378285 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:45.380059  3597 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:45.384806 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:45.389907 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:45.396672 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:45.437631 29145 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:45.399219 29216 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:06:45.439026 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:45.439813 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:45.441350 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:45.443828 29068 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:45.405524 29218 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:06:45.412019 29904 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:06:45.402920 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:45.395713  3566 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:45.411700  3593 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:06:45.418609  3591 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:45.419875 29215 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:45.453934 29221 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:45.420835 29220 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:45.395960  3508 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.383117  3180 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:45.457612 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 54483.6us
I20260709 13:06:45.426247 29903 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:06:45.426810 29146 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:45.459244 29903 pending_rounds.cc:85] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 13
I20260709 13:06:45.429965  3353 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:45.430711  3560 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:45.431770  3543 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:45.422554 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:45.431892  3592 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:45.459650 29903 pending_rounds.cc:107] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:06:45.472919 29903 raft_consensus.cc:2889] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:45.463048 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:45.464056 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:45.485633 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:45.498669 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.513756  3349 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:45.523576 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:45.526216  3595 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:45.476984  3592 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:45.473766  3571 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:45.474049  3347 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.469779  3543 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:45.488394  3651 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:45.460137  3140 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
W20260709 13:06:45.547336 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 98130.7us
I20260709 13:06:45.461024  3584 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
W20260709 13:06:45.551144 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 92592.8us
I20260709 13:06:45.518855  3660 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.470854  3508 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.487691  3520 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:06:45.568835 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:45.463088  3580 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:06:45.566874 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:45.513448  3575 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:06:45.543238  3470 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:45.517292  3597 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:06:45.548512  3349 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:45.593814 29147 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:45.556604  3353 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:45.662276 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:45.668818 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 70841.2us
I20260709 13:06:45.671514 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:45.672163 29215 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:45.674690 29147 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:06:45.566111  3347 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.553663  3571 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.686830 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:45.699885 29215 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:45.592362  3470 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:45.571110  3651 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.542979  3566 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.663493  3593 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:45.669932  3480 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:45.724403 29215 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:06:45.596298  3365 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.102s
I20260709 13:06:45.718760 29147 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:06:45.739035 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.583786  3660 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:45.746344 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.749351 29215 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:06:45.751783 29147 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:45.732710  3550 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:45.779527 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:45.781052 29904 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:06:45.739133  3676 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:45.658468  3592 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:45.775164  3668 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
W20260709 13:06:45.776068  3674 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:06:45.782816  3667 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.333s	user 0.002s	sys 0.000s
I20260709 13:06:45.762004  3447 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:45.710692  3658 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.094s
I20260709 13:06:45.676602  3353 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.745049  3677 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:45.792320  3180 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
W20260709 13:06:45.808419  3666 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.281s	user 0.001s	sys 0.001s
I20260709 13:06:45.768538  3598 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:45.826363  3508 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:45.793955  3669 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:06:45.828538 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:45.829314  3651 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:45.830873 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:45.760720  3349 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.797361  3550 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.737607  3593 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.792102  3470 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.832576 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:45.843888 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:45.850466 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 180949us
I20260709 13:06:45.844221 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:45.855381 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:45.857801 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:45.880448 29222 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:06:45.881723 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:45.831856  3347 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:45.832872  3571 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:45.861730  3673 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.002s
I20260709 13:06:45.888576 29067 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:45.835018  3660 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:45.892477 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:45.795733  3676 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.898598  3140 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:45.900483  3140 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.900823  3580 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:45.901484  3580 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.902741  3140 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:45.844259  3447 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.905293  3593 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:45.905844 29141 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:45.906451  3447 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:45.886147  3678 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.099s	user 0.000s	sys 0.002s
I20260709 13:06:45.903611  3580 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:45.907418  3550 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:45.914126 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:45.860028  3677 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:46.059783 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:46.075867 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.916582 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:45.916541 29144 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:46.001672 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:46.033111 29215 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
W20260709 13:06:46.072306 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 161266us
W20260709 13:06:46.001188 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.921446 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:46.124830 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 125135us
I20260709 13:06:46.108134 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 1034) took 1874 ms. Trace:
W20260709 13:06:46.193979 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 279934us
I20260709 13:06:45.992408 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:46.197093 29213 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:06:45.978665 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:46.129303 29113 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:45.893023  3470 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:46.026948  3539 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.713s	user 0.005s	sys 0.009s
I20260709 13:06:46.210171 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:46.204217  3349 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:45.926348  3447 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:46.095254  3529 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:46.217432 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:46.241586 29911 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:46.167409  3598 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
W20260709 13:06:46.029975  3591 consensus_meta.cc:287] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.003s
I20260709 13:06:46.034258  3682 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:46.220377 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:45.977840  3472 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:45.980937  3580 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.241s
I20260709 13:06:46.166671  3537 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:46.098937  3353 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.025691  3676 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:46.132570  3393 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.117393  3500 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:46.026372  3579 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:46.222858 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 93642.7us
I20260709 13:06:46.112390  3180 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:46.174499  3593 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:46.231683 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:46.197633 29065 rpcz_store.cc:276] 0709 13:06:44.234117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:44.234548 (+   431us) service_pool.cc:224] Handling call
0709 13:06:44.900465 (+665917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:44.913955 (+ 13490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:44.913986 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:44.914030 (+    44us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:06:44.914060 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:44.914126 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:46.107772 (+1193646us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b54002ac450. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:46.108012 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13086,"raft.queue_time_us":14171,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":22436,"spinlock_wait_cycles":1530726144}
I20260709 13:06:46.019359  3453 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:45.926397  3560 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.296s
I20260709 13:06:46.259637  3541 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:46.266534 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1193) took 1174 ms. Trace:
W20260709 13:06:46.267587  3671 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.598s	user 0.002s	sys 0.002s
I20260709 13:06:46.333222  3453 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:46.277225  3677 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:46.282811  3598 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:46.349305  3453 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.350019  3598 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:46.351238  3453 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:46.351399  3598 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.229041  3520 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:46.356649  3520 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.357322  3520 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.318274  3537 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.243351  3682 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.242360  3561 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.360621  3561 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.362033  3682 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:46.240998  3506 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:46.294024  3472 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:45.994095  3477 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:45.938205  3679 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.305s
I20260709 13:06:46.364732  3472 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:46.325848 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:46.366509 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:46.332470 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 100900us
I20260709 13:06:46.367511 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:46.307650 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 376512us
I20260709 13:06:46.334164  3575 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:46.378005 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3008) took 1114 ms. Trace:
I20260709 13:06:46.332176  3579 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:46.337478  3698 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.094s	user 0.000s	sys 0.005s
I20260709 13:06:46.317283  3393 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.335062  3541 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:46.360991  3537 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:46.402241 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.353176  3598 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:46.336596  3590 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:46.377982 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:46.336251  3180 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.379504 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:46.324885  3611 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:46.327178  3500 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.381363 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:46.378931 29222 rpcz_store.cc:276] 0709 13:06:45.263999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.264271 (+   272us) service_pool.cc:224] Handling call
0709 13:06:45.264707 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:45.884497 (+619790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.377556 (+493059us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b540033a4d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:46.377909 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1379317120}
I20260709 13:06:46.335273 29214 rpcz_store.cc:276] 0709 13:06:45.092712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.092863 (+   151us) service_pool.cc:224] Handling call
0709 13:06:45.093214 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:45.093596 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:06:45.096563 (+  2967us) log.cc:844] Serialized 14 byte log entry
0709 13:06:45.215034 (+118471us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:46.257738 (+1042704us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b2c00127298. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:06:46.258365 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 13:06:46.258564 (+   199us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:46.258617 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.258719 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:46.265980 (+  7261us) raft_consensus.cc:1685] finished
0709 13:06:46.266156 (+   176us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:46.266441 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46190,"prepare.queue_time_us":3118,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":58473,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":6571,"spinlock_wait_cycles":62316800,"thread_start_us":86514,"threads_started":1,"wal-append.queue_time_us":173699}
I20260709 13:06:46.399376 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:46.401481 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:46.371397 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.406445 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:46.407848 29909 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:46.415130 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:46.417284 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:46.418417 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:46.413039 29142 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:06:46.421236  3180 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:46.421797  3393 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:46.421897  3700 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:46.422950 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:46.424036  3693 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:46.424391 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46512.9us
I20260709 13:06:46.424645  3693 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.426600  3584 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:46.431617  3584 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:46.428313  3547 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:46.422698  3700 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.432385  3595 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.432041  3541 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.399930  3506 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:06:46.334173  3593 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.451684  3347 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:46.452601  3593 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.415140  3590 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.402817  3651 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:46.453881  3561 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:46.454221  3651 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.454663  3651 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:46.428382 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:46.444864 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:46.441740 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:46.440521 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:46.445307 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:46.450527  3541 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:46.452306  3347 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.448848 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:46.421875  3180 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:46.466019 29143 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 15: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:46.443017  3584 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:46.472420  3693 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:46.477125  3590 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.480401 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:46.482101  3584 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.483901 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:46.484869 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:46.485095  3584 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:46.487197  3693 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:46.487748  3693 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:46.488296  3693 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:46.495394  3716 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:06:46.435714  3537 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:46.516775 29069 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:46.532341 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:46.532927 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:46.535146 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.536193  3707 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:46.538535 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 157949us
I20260709 13:06:46.436121  3500 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.435889  3611 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.496439  3180 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:46.553465 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:46.521957  3520 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:46.462533  3595 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.580768 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:46.557580 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:46.600718 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:46.573189 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:46.625226 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.584790 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:46.678359  3707 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.631729 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:46.680742  3347 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:46.681372  3580 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:46.681810  3580 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:46.682220  3580 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:06:46.682462  3702 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:06:46.570781 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:46.684031 29067 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:46.691779 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:46.691962 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 60333.9us
I20260709 13:06:46.684536  3453 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.684608  3579 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.712595 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:46.685140  3714 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:06:46.688879  3700 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.718901 29910 raft_consensus.cc:2410] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
W20260709 13:06:46.690145  3703 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:06:46.692528 29073 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:46.722721 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:46.723382 29146 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:46.705911 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:46.534180  3537 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.720046 29065 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:06:46.720014 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:46.744184 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.680208  3500 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.634446  3252 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:46.733170  3592 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:46.743724 29911 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:46.692608  3646 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.612627  3571 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:46.611621  3590 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.628751  3682 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:46.706950  3180 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:46.716771  3596 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:06:46.745401  3579 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:46.715682  3453 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:46.810482 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 98478.8us
I20260709 13:06:46.761595  3597 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:46.823025 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 50169.3us
I20260709 13:06:46.823807 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:46.773842  3650 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:46.803859  3529 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.535s	user 0.012s	sys 0.000s
I20260709 13:06:46.824656  3543 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.833055 29908 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 20: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:46.840330 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 1134) took 1167 ms. Trace:
I20260709 13:06:46.842871 29067 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:46.778235  3539 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:46.840391 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:46.844023 29904 rpcz_store.cc:276] 0709 13:06:45.672627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.672849 (+   222us) service_pool.cc:224] Handling call
0709 13:06:45.673279 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:45.781546 (+108267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:46.840089 (+1058543us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54005bc554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:46.840272 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247012224}
I20260709 13:06:46.844559 29902 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:06:46.797757  3590 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.797204  3646 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.769245  3252 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.849877  3528 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.857071  3528 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.799892  3571 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.860500 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:46.821882  3682 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.846892  3543 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:46.869515 29908 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:46.852250  3529 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:46.820195  3579 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.835999  3597 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.840740  3650 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.866911  3711 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:46.854023  3539 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.877545  3529 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:46.905995 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.894364  3529 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:46.903326  3528 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.924276  3480 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:46.926703 29215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:46.924563  3447 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:46.924355  3658 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:46.933739  2804 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:46.939240 29215 raft_consensus.cc:606] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:46.942947 29215 raft_consensus.cc:614] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:46.891034  3711 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.895534  3693 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:46.955202 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:46.939493  3611 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.945537  3447 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.961544  3453 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.969103  3447 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:46.944989  2804 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:46.969771  3571 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:46.972354  2804 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.943234  3537 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.979169 29071 raft_consensus.cc:2393] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 22: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:46.947996  3707 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:46.945880  3480 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:46.945609  3658 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:46.986774  3721 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.139s	user 0.002s	sys 0.000s
I20260709 13:06:47.000541  3646 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:47.035023 29215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "813e480732fd4061837e784d4d768b1f"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:47.036033 29215 raft_consensus.cc:606] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.036587 29215 raft_consensus.cc:614] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:47.021327  3579 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:47.034373 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 185634us
W20260709 13:06:47.041289  3718 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.158s	user 0.002s	sys 0.000s
I20260709 13:06:47.009070  3590 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:47.055575 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.032161  3537 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:47.060299 29215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:47.066092 29215 raft_consensus.cc:606] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:47.066824 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.054399  2804 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.074795 29904 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:47.056573  3597 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.059058  3541 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.067581 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:47.070039  3575 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:47.061159  3539 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.081159  3658 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:47.068799  3537 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.070663  3700 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:47.069911  3704 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.094074 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:47.076627  3650 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.107362 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:47.084187  3470 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:47.099154 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.099642 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.113965 29139 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:47.073719  3590 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:47.117285 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 402094us
I20260709 13:06:47.159193 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:47.143311 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 73700.5us
I20260709 13:06:47.165556 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.149732  3650 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.151980 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.153012 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:47.153874 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:47.195720 29906 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:47.193908  3463 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:47.200240  3646 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:47.200927  3646 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.201417  3646 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:47.201710 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:47.208329 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.208814 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.209733 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.210083 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.210352 29068 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:47.211205 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.212440 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.216367  3571 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.216995  3571 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.217518 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.208758 29070 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:06:47.078788  3677 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:06:47.156144 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:47.067982  2804 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.223217  3658 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:47.223462  2804 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.224053  3683 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.224615  3683 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.225172  2804 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:06:47.225636  2804 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:47.226045  3537 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.226675  2804 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.228129  3707 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:47.228300  3590 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.229123  3707 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.232465 29904 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:06:47.232744  3480 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.235179  3590 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.236208  3717 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:47.237038  3717 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.238382  3180 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.240136  3717 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.241114 29068 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: []
W20260709 13:06:47.245643  3728 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
W20260709 13:06:47.248675  3726 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:06:47.187036  3650 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.252104  3223 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.252537 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3015) took 1397 ms. Trace:
I20260709 13:06:47.252882  3223 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:47.254734 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.252902 29218 rpcz_store.cc:276] 0709 13:06:45.855805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.098024 (+242219us) service_pool.cc:224] Handling call
0709 13:06:46.098528 (+   504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:46.363988 (+265460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:46.364019 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:46.364555 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:06:46.472692 (+108137us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:06:46.473782 (+  1090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.252031 (+778249us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b540016c954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:47.252080 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:47.252125 (+    45us) raft_consensus.cc:1685] finished
0709 13:06:47.252138 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:47.252474 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1383,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":89,"spinlock_wait_cycles":609557376,"thread_start_us":107751,"threads_started":1,"wal-append.queue_time_us":104911}
W20260709 13:06:47.186407 29037 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 2.058s (SENT) [suppressed 6 similar messages]
I20260709 13:06:47.089403  3541 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.261171 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:47.263168 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.263908 29071 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:06:47.264745 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.116618  3579 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.187687 29147 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:47.191583 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.156631  3470 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:47.185267  3727 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:06:47.152649  3711 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.119659  3682 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.111616  3600 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:47.193085 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.194507 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.156584  3693 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.159161  3729 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.117759  3597 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.075s
I20260709 13:06:47.297001  3707 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.272985  3571 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:47.297322  3729 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.265318 29071 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:47.298971 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.299574 29071 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:47.112142  3704 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.310428 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.187252  3549 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:47.280534 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:47.282739  3677 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:47.282153 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 90647.3us
I20260709 13:06:47.287744 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:47.255432  3720 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:06:47.326323 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.105803  3700 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.312824  3549 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:47.329473  3546 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:47.329236  3549 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.329919  3700 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.228121  2804 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:47.330420  3549 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:47.253547  3650 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.301277 29069 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:47.262869 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:47.333871 29068 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:47.335045 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.305385 29221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:47.337365 29221 raft_consensus.cc:606] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.337755 29221 raft_consensus.cc:614] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:47.307231 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.339135 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.308935 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.340606 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.341202 29217 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:47.341990 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.342522 29217 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:47.343241 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.311221 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.281319  3590 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.153789  3722 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:06:47.223814  3658 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.322741 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.299733  3729 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.285351  3480 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:47.363096 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.312033  3506 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:47.313903  3682 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:47.285233  3717 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:47.337114 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.306363 29216 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:47.350612 29215 raft_consensus.cc:2393] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:47.321887 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.375106 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.313577  3547 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:47.350138 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 197212us
I20260709 13:06:47.287969  3470 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.285048  3579 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.381698 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.384980 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.386173 29213 raft_consensus.cc:2410] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:06:47.353523  3677 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.390053 29072 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:47.393245  3677 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.393870 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.395076 29147 raft_consensus.cc:2410] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:06:47.398162 29112 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:47.399385  3717 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:47.400101  3717 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:47.400656  3717 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:47.377779 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:47.384826 29220 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:06:47.361337  3724 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:47.407721 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70719.8us
W20260709 13:06:47.370844  3731 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.113s	user 0.000s	sys 0.004s
W20260709 13:06:47.411737 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:47.414852 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 124946us
I20260709 13:06:47.417771 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.419696 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:47.420949 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 234561us
I20260709 13:06:47.421504 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:47.426616 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:47.427314 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:47.441197 29906 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:06:47.428643 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 148197us
I20260709 13:06:47.429577 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:47.431965 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:47.433965  3734 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:47.361258  3590 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.412034  3508 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:47.437031 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.450780 29215 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:47.451879 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.375109  3682 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.444319 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:47.372294  3506 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.456067 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.444484 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:47.412134  3365 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:47.436291 29213 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:06:47.414194  3738 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:06:47.447675 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.452564 29215 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:06:47.371026  3480 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:47.468451  3580 consensus_meta.cc:287] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.786s	user 0.014s	sys 0.000s
I20260709 13:06:47.450451  3508 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:47.472398  3470 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.458607  3658 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:47.428941  3180 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.404572  3547 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.459512  3704 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.523141 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:47.615300 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:47.480458  3592 consensus_meta.cc:287] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.712s	user 0.008s	sys 0.006s
I20260709 13:06:47.524847  3711 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:06:47.525565 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 69580.3us
I20260709 13:06:47.448398  3724 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:47.476130  3677 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:47.469785  3722 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:47.423743  3739 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:47.547596 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.680672 29213 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 23: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:47.451372 29037 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:47.681661 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 230427us
W20260709 13:06:47.548225 29144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 2957) took 2186 ms (client timeout 1628 ms). Trace:
W20260709 13:06:47.562594 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 115035us
I20260709 13:06:47.682332 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.568074  3650 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:47.569154  3575 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:47.484045  3733 log.cc:927] Time spent T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:06:47.606568 29219 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
W20260709 13:06:47.580521 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 136076us
W20260709 13:06:47.593713 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.617851 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.618690 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.619520 29140 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:47.702014 29143 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:47.620258 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.621695 29215 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:47.630399 29142 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:47.482407  3682 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:47.556123  3651 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.101s	user 0.000s	sys 0.015s
W20260709 13:06:47.556730  3598 consensus_meta.cc:287] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.146s	user 0.005s	sys 0.010s
I20260709 13:06:47.548071  3716 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:47.688450 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:47.682571 29144 rpcz_store.cc:269] 0709 13:06:45.360346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.375073 (+ 14727us) service_pool.cc:224] Handling call
0709 13:06:47.510160 (+2135087us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b540029d454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:47.546651 (+ 36491us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1119704960}
I20260709 13:06:47.682341 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:47.694137 29218 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 13.
I20260709 13:06:47.692936 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.730221 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.730388 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.730718 29146 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:47.730806 29213 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:47.690558 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.570076  3600 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:47.487717  3595 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.711470 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.686270  3724 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.748327  3724 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:47.656608  3508 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:47.720594 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.751834 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.725198 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.680315  3739 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.629252  3541 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.753257 29217 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:47.693835  3736 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.754273 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.754091  3736 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.755555 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.756443  3579 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.756664 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.757138  3686 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:47.680912  3722 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:47.733177 29145 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:47.693765  3537 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:47.760303  3722 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:47.727241 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:47.761621 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:47.756479 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 66026.9us
I20260709 13:06:47.763214 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:47.763545 29902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 1088) took 1906 ms (client timeout 2132 ms). Trace:
I20260709 13:06:47.735587 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:47.731706 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.727900 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:47.741397 29147 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:47.757294 29217 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:06:47.760514  3547 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:47.770588 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.741998 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:47.764482 29902 rpcz_store.cc:269] 0709 13:06:45.857217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.857342 (+   125us) service_pool.cc:224] Handling call
0709 13:06:47.763351 (+1906009us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b54004141d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:47.763485 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2209739136}
I20260709 13:06:47.772437 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.705101  3439 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:47.605839  3580 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.614056  3717 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:47.768678 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.771234 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.773499 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.774704 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 2972) took 1263 ms. Trace:
I20260709 13:06:47.775836 29065 rpcz_store.cc:276] 0709 13:06:46.511494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.515124 (+  3630us) service_pool.cc:224] Handling call
0709 13:06:46.515466 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:46.743157 (+227691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:47.770776 (+1027619us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54002ad354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:47.774644 (+  3868us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469705088}
I20260709 13:06:47.775048 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.778272 29213 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:47.779603 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.780433 29213 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:47.781549 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:47.732862  3707 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.716640  3180 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:47.725803  3223 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.814333 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.784236 29217 raft_consensus.cc:2410] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:06:47.815162 29144 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:47.705772  3575 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.764144  3365 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:47.766533  3470 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:47.854367  3470 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:47.855012  3546 consensus_meta.cc:287] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.002s
I20260709 13:06:47.810956 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.811149 29221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:47.856415 29221 raft_consensus.cc:606] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:47.732685  3651 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.858691 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:47.859557  3651 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.753511  3506 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:47.764201  3600 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:47.821048 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 2979) took 1115 ms. Trace:
I20260709 13:06:47.870048 29067 rpcz_store.cc:276] 0709 13:06:46.705493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.705652 (+   159us) service_pool.cc:224] Handling call
0709 13:06:47.820831 (+1115179us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54000686d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:47.820973 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271222656}
I20260709 13:06:47.809336  3363 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:47.789117  3439 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.826488  3683 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:47.826737 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:47.830866  3365 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.774135  3514 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:47.741683  3716 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.835983  3347 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:47.837453  3529 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.001s
I20260709 13:06:47.753512  3739 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:47.838078  3223 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:47.916709  3223 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.917269  3223 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:47.815814  3584 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:47.920015  3584 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.922780  3584 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:47.840869  3652 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:47.808586  3547 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:47.842002  3575 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
W20260709 13:06:47.842074 29145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 3068) took 1443 ms (client timeout 1667 ms). Trace:
W20260709 13:06:47.925931 29145 rpcz_store.cc:269] 0709 13:06:46.398938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.399112 (+   174us) service_pool.cc:224] Handling call
0709 13:06:47.841890 (+1442778us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54003d36d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:47.842010 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3036615040}
I20260709 13:06:47.821547  3707 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.929160  3707 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.844710  3717 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.930323 29140 raft_consensus.cc:2376] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19. Re-sending same reply.
I20260709 13:06:47.933888 29146 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:06:47.845371  3252 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.820338  3180 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.848481  3580 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.849462  3480 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:47.852448  3590 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.860169 29141 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:06:47.860857 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:47.703789  3658 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:06:47.855839 29903 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
W20260709 13:06:47.862165 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 151711us
I20260709 13:06:48.014390 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:47.862697 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 141952us
W20260709 13:06:47.863217 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 180946us
I20260709 13:06:47.782879  3604 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:06:47.864612  3646 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.663s	user 0.010s	sys 0.003s
I20260709 13:06:47.866703 29215 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:48.046427 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.047497 29145 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:47.863777  3677 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:06:47.894285 29871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.010s	sys 0.018s
W20260709 13:06:48.057866 29871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.010s	sys 0.018s
I20260709 13:06:47.898913  3365 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:47.899272  3514 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.869553  3600 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:47.897430  3363 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:47.912874  3716 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:47.913690  3347 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.108670  3477 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:48.110648  3347 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.914674  3529 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.113456  3529 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:47.924489 29909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 1089) took 1994 ms (client timeout 2166 ms). Trace:
W20260709 13:06:48.114125 29909 rpcz_store.cc:269] 0709 13:06:45.930569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.930757 (+   188us) service_pool.cc:224] Handling call
0709 13:06:47.924297 (+1993540us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54005e92d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:47.924421 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474565504}
I20260709 13:06:47.868278  3470 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:47.948252 29213 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:47.949741 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:47.991430 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:48.116151  3600 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:06:48.116899 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 166468us
I20260709 13:06:48.019255 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:48.058888 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:48.058396 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 776046us
W20260709 13:06:48.119417 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 103587us
I20260709 13:06:48.120208 29038 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:48.064435  3514 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:48.121400 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:48.021833 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.122794 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.123124  3541 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:48.123601  3541 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.124027  3541 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:48.060600  2804 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.124293 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.124749  2804 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.064694 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.021749  3393 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.066561  3543 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:48.067761 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:48.045676  3252 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:48.073881  3745 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:48.076409 29214 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:48.147010 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.078035 29072 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:48.079273 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:48.059314  3677 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.081874 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:48.089455  3439 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:48.090842 29903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 1082) took 2111 ms (client timeout 2058 ms). Trace:
I20260709 13:06:48.148931  3547 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:48.107669  3566 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:48.119437 29067 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
W20260709 13:06:48.126739 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.242453 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.083567 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3073) took 1673 ms. Trace:
I20260709 13:06:48.140738 29073 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:48.141634 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.012354  3722 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.019029  3580 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.142743 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.143210 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:48.244503 29140 raft_consensus.cc:2393] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:48.243505 29141 rpcz_store.cc:276] 0709 13:06:46.410091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:46.410327 (+   236us) service_pool.cc:224] Handling call
0709 13:06:46.410750 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.011753 (+1601003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.083224 (+ 71471us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540029ed54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:48.083493 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3332751616}
W20260709 13:06:48.262858 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 119745us
I20260709 13:06:48.144627 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.145440 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.080595 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:48.264811 29066 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
W20260709 13:06:48.151150 29903 rpcz_store.cc:269] 0709 13:06:45.979822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:45.992095 (+ 12273us) service_pool.cc:224] Handling call
0709 13:06:48.090575 (+2098480us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54004e61d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:48.090717 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3810536704}
I20260709 13:06:48.102236 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 1147) took 1008 ms. Trace:
I20260709 13:06:48.062201  3736 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.143496  3652 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:48.139607  3529 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:48.233393 29148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.024s	sys 0.051s
I20260709 13:06:48.142771  3453 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:48.145336  3494 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.059844  3646 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:48.233762 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.328330 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.056653  3604 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.328673 29910 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:48.329079  3604 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:48.329699  3646 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:48.279510  3439 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.330741  3439 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.331270 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 1148) took 1178 ms. Trace:
I20260709 13:06:48.332666 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:48.333258  3439 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:48.331679 29910 rpcz_store.cc:276] 0709 13:06:47.152714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.155878 (+  3164us) service_pool.cc:224] Handling call
0709 13:06:48.331061 (+1175183us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54005a4e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:48.331205 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2545956608}
I20260709 13:06:48.149132  3683 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:48.233934 29875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.022s	sys 0.065s
W20260709 13:06:48.336182 29875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.023s	sys 0.065s
W20260709 13:06:48.336544 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 635838us
W20260709 13:06:48.336719  3508 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.586s	user 0.008s	sys 0.003s
I20260709 13:06:48.337145  3508 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.153370  3677 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:48.239027  2804 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.127362 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.243708 29871 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:48.341430 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 97840.4us
I20260709 13:06:48.265949 29904 rpcz_store.cc:276] 0709 13:06:47.093515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.107025 (+ 13510us) service_pool.cc:224] Handling call
0709 13:06:48.102012 (+994987us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54004b50d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:48.102153 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285636864}
I20260709 13:06:48.264318 29909 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:06:48.345031  3729 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:48.345520  3729 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.345903 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:48.346194  3729 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:06:48.348367  3711 consensus_meta.cc:287] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.000s
I20260709 13:06:48.348794  3711 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:48.349308  3600 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.145869  3393 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.352735 29911 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:48.279067 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.355387  3393 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:48.278335 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:48.286867 29148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.024s	sys 0.052s
I20260709 13:06:48.238096  3477 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:48.357789  3477 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.358258 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.288514 29143 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
W20260709 13:06:48.360702 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:48.292560 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.365213 29142 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:48.349969 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.293861  3363 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.294186 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.366638 29139 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:48.296258 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.297290 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.271551 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:48.304198  3704 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:48.289639  3453 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:48.368465  3704 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:48.368851  3453 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:48.369450  3704 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.371070  3704 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:48.316344 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:48.315871 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:48.316660 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:48.372781 29065 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:48.319494 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:48.345500  3480 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.279412  3252 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:48.317665  3529 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.316816 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:48.145278  3695 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:48.146888  3745 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:48.347534  2804 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:48.349475  3652 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:48.355865  3724 consensus_meta.cc:287] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.000s
I20260709 13:06:48.359351  3494 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.293617 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.367928 29147 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:48.302799  3506 raft_consensus.cc:993] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:06:48.374040 29066 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:48.320688  3180 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.375581 29037 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:48.381173 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.430433 29911 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:48.433362 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:48.276847  3580 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:48.282333 29874 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.016s	sys 0.053s
I20260709 13:06:48.380254  3508 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:48.436015 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 60502.4us
W20260709 13:06:48.435266 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.015s	sys 0.035s
W20260709 13:06:48.437063 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.015s	sys 0.035s
I20260709 13:06:48.437727  2804 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:48.385200 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.383399 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.438115  2804 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:48.438515  2804 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:48.435206 29874 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.017s	sys 0.053s
W20260709 13:06:48.439674 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 664649us
I20260709 13:06:48.440585 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:48.392521 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 121068us
I20260709 13:06:48.441721 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:48.347384  3722 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.398478 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.398640 29905 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:48.398221 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.445427 29140 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:48.327666  3365 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.405251 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:06:48.488232 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 46608.4us
I20260709 13:06:48.489014  3695 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:48.489586  3695 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.489995  3695 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:48.489902 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:48.490990  3595 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:48.492110 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:48.492831  3592 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:48.493221  3592 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:48.493835  3592 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:48.430125  3658 raft_consensus.cc:1081] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Signalling peer 730186a8c70f43f2a57d80163bbc37d2 to start an election
I20260709 13:06:48.387436  3477 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:48.387637 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 70978.3us
I20260709 13:06:48.398763  3470 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:48.511499 29072 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:48.438637 29904 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:48.438441 29906 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:48.446553 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:48.401065 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.513546 29903 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:48.444106 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:48.453828  3738 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:48.455415 29146 raft_consensus.cc:2410] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
W20260709 13:06:48.431344  3700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.025s	sys 0.047s
I20260709 13:06:48.387142  3480 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.467070 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:48.469852 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:48.473495  3600 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:48.405635  3252 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:48.473801 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:48.406534  3223 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.475955 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:48.540529 29213 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:48.478655 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:48.541041 29214 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:48.480654 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:48.541497 29186 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.025s	sys 0.041s
W20260709 13:06:48.541837 29186 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.025s	sys 0.041s
I20260709 13:06:48.482040  3494 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:48.482857 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:48.497846 29139 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:48.417626 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:48.568751 29141 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:48.493232 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:48.462423  2804 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:48.521173 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 74703.7us
I20260709 13:06:48.577230 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:48.578293  3738 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:48.578410 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:48.578758  3738 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.579208  3738 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:48.579362 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:48.511894 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:48.580535 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68720.3us
I20260709 13:06:48.581645 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:48.582259  3584 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:48.583220  3470 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:48.584208  3579 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:48.584859  3579 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.584990  3547 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:48.585880  3579 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:48.493216  3716 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:48.497257  3639 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.011s	sys 0.019s
W20260709 13:06:48.542104 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 532258us
I20260709 13:06:48.543684 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.589349  3528 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:48.548729 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:48.589820 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:48.588572 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:48.591498  3590 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:48.591900  3590 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.592016 29186 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:48.520715  3700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.025s	sys 0.047s
I20260709 13:06:48.588440  3716 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.592360  3590 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:06:48.588660  3639 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.012s	sys 0.019s
I20260709 13:06:48.594954  3716 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.565430 29221 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:48.592123  3347 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:48.595659  3347 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:48.596469  3347 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:48.600158 29186 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:48.600301  3595 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:48.601014  3595 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:48.601418  3595 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:48.603013  3575 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:48.600173  3547 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:06:48.605244 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 112080us
I20260709 13:06:48.605576 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:48.560012 29905 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:06:48.406255  3739 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.427611  3724 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:48.620107 29216 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.024s	sys 0.036s
W20260709 13:06:48.630808 29216 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.024s	sys 0.037s
I20260709 13:06:48.623930 29141 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:48.400214  3683 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.435401  3363 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.093s
I20260709 13:06:48.564858  3252 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.640136  3252 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:48.645227  3547 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:48.655404 29145 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:48.495216  3707 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.135s
I20260709 13:06:48.539494  3753 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:48.634092  3716 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:48.662492 29908 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:48.664045 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:48.708755 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44839.9us
I20260709 13:06:48.710101 29909 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
W20260709 13:06:48.710426 29215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.011s	sys 0.011s
W20260709 13:06:48.710716 29215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.011s	sys 0.011s
W20260709 13:06:48.713840 29217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.011s	sys 0.013s
W20260709 13:06:48.715735 29217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.011s	sys 0.013s
I20260709 13:06:48.731036  2804 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:48.632175  3700 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 13:06:48.732715  3700 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:48.733160  3700 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:48.734125  3700 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.753265 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:48.753983 29215 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:48.754456 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:48.754951 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:48.755277 29222 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:48.759773 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.760943 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:48.762372 29139 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:48.763756 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:48.764096 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:48.758998 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:48.755594 29214 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:48.765952 29144 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:48.760456 29909 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:48.765277 29908 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:48.766510 29907 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 13:06:48.779845  3595 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:48.667655  3541 consensus_meta.cc:287] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.543s	user 0.006s	sys 0.002s
I20260709 13:06:48.735625  3700 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:48.656549  3707 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:48.788154 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:48.669373  3596 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.752074  3543 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:48.797163 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 1150) took 1353 ms. Trace:
I20260709 13:06:48.820779 29908 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:06:48.830210 29907 rpcz_store.cc:276] 0709 13:06:47.443909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.444893 (+   984us) service_pool.cc:224] Handling call
0709 13:06:47.445236 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.766992 (+1321756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.796774 (+ 29782us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54005bc554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:48.797072 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3038109056}
I20260709 13:06:48.747063  3453 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.655864  3677 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.784742  3528 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:48.662580  3590 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.512576  3745 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.894577  3470 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:48.900525  3543 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:48.906213  3529 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:48.916680  3514 consensus_meta.cc:287] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.795s	user 0.010s	sys 0.004s
I20260709 13:06:48.969045  3514 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.877777  3596 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.930263  3611 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:48.927747  3365 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.962256  3528 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:48.967440  3480 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:48.981815  3528 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:48.973420  3580 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.978726  3652 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.814100  3707 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:48.939745  3223 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.968152  3579 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.967116  3470 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.982848  3739 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.985085  3724 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:49.002126  3760 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:06:48.988165  3590 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:48.978986  3611 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:48.998225  3514 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:48.986608  3695 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:49.016242  3757 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
W20260709 13:06:49.049779  3759 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.128s	user 0.001s	sys 0.001s
I20260709 13:06:49.010991  3543 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:49.104423  3704 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.733s	user 0.015s	sys 0.000s
I20260709 13:06:49.078722  3677 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:49.106545  3470 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:49.113570  3611 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:49.114158  3683 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:49.118607  3604 consensus_meta.cc:287] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.789s	user 0.005s	sys 0.005s
I20260709 13:06:49.129518  3579 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:49.132225  3729 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.786s	user 0.005s	sys 0.008s
I20260709 13:06:49.134397  3695 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:49.145480  3453 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.147359  3595 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.006814  3347 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:48.811651  3541 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:49.171108  3592 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.677s	user 0.014s	sys 0.000s
I20260709 13:06:49.185667  3729 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.193822  3729 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.151401  3707 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.234632  3707 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:06:49.201038  3738 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.000s
I20260709 13:06:49.237985  3738 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.246531  3738 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:49.297327 29039 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.054s	user 0.006s	sys 0.041s
W20260709 13:06:49.298101 29039 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.055s	user 0.006s	sys 0.041s
W20260709 13:06:49.298723 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 1.05618e+06us
I20260709 13:06:49.323995 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3163) took 1889 ms. Trace:
I20260709 13:06:49.324806 29217 rpcz_store.cc:276] 0709 13:06:47.433794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.866362 (+432568us) service_pool.cc:224] Handling call
0709 13:06:47.866755 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:48.120294 (+253539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:48.120323 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:48.120798 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:06:48.716961 (+596163us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:06:48.717001 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:48.746069 (+ 29068us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54002f62d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:48.746116 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:49.322903 (+576787us) raft_consensus.cc:1685] finished
0709 13:06:49.322984 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.323628 (+   644us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":375,"raft.queue_time_us":19399,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6470,"spinlock_wait_cycles":582218624,"thread_start_us":595784,"threads_started":1,"wal-append.queue_time_us":878684}
W20260709 13:06:49.209147  3252 consensus_meta.cc:287] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.569s	user 0.013s	sys 0.000s
I20260709 13:06:49.196043  3347 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:49.198558  3541 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.208104  3592 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:49.449753  3252 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.177723  3604 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.452323  3695 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:49.453563  3695 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:49.455214  3604 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:49.357576 29871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.000s	sys 0.012s
W20260709 13:06:49.456156 29871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.000s	sys 0.012s
I20260709 13:06:49.457214 29902 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:49.458868 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54902 (request call id 1099) took 1336 ms. Trace:
I20260709 13:06:49.289136 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:49.325884 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:49.433502 29188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.002s	sys 0.010s
I20260709 13:06:49.434010  3477 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:49.436754 29111 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.038s	sys 0.044s
I20260709 13:06:49.460352 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:49.463528  3780 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:49.465790 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:49.465601  3780 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.466864 29903 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:49.469048 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:49.470316 29905 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:49.471695  3677 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:49.472483  3541 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.472716  3677 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:49.473361  3541 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:49.478667  3543 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:49.479687  3543 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:49.480252  3695 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.482311  3543 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.483742  3541 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.501119  3772 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:49.502193  3772 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:49.434053  3363 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:49.505496  3363 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:49.512087 29188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.002s	sys 0.011s
W20260709 13:06:49.514289 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 908990us
I20260709 13:06:49.516342  3477 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:49.516525 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.022s	sys 0.017s
W20260709 13:06:49.517961 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.022s	sys 0.018s
W20260709 13:06:49.519115 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 997675us
I20260709 13:06:49.459309 29902 rpcz_store.cc:276] 0709 13:06:48.122642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.124067 (+  1425us) service_pool.cc:224] Handling call
0709 13:06:49.458479 (+1334412us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54005a32d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:49.458743 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3062608640}
W20260709 13:06:49.522408 29213 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.008s	sys 0.004s
W20260709 13:06:49.527189 29111 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.038s	sys 0.045s
I20260709 13:06:49.534583 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:49.536504  3541 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:49.475773  3677 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:49.540473 29213 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.008s	sys 0.004s
W20260709 13:06:49.544657 29186 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.007s	sys 0.014s
I20260709 13:06:49.553020  3580 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:49.544709 29038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.027s	sys 0.020s
W20260709 13:06:49.562016 29186 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.007s	sys 0.014s
W20260709 13:06:49.563079 29038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.027s	sys 0.020s
W20260709 13:06:49.563232 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 944763us
I20260709 13:06:49.564682  3541 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:49.565495  3772 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.567401  3772 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:49.567489  3695 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:49.565999  3541 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:49.568486  3695 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:49.571018  3695 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.571750 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:49.572628 29068 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:49.573755 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:49.575209  3541 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.564373  3580 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.577382 29066 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:49.581080  3780 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.586684  3724 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:49.587759  3724 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:49.590384  3724 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.591809 29186 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:49.593366  3514 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:49.593864  3514 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.594266  3514 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:49.595218 29186 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:49.596541  3773 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:49.596988  3773 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.597193  3363 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:49.597379  3773 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:49.553781  3592 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.600600 29186 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:49.601995  3683 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:49.602550  3683 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:49.602986  3683 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:49.607095  3580 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:49.607892  3477 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:49.612054 29112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.002s	sys 0.010s
W20260709 13:06:49.612589 29112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.054s	user 0.002s	sys 0.010s
I20260709 13:06:49.614562 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:49.615789 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:49.616674 29066 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:49.616832 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:49.617913 29073 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:49.620410 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:49.621269 29066 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:06:49.620438 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:49.622771 29073 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
W20260709 13:06:49.623970 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 1.06559e+06us
I20260709 13:06:49.625681 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:49.625751 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:49.627221 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:49.627928 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:49.638280 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:49.639094 29222 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:49.642732  3738 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:49.637255  3596 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:49.643949  3729 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:49.644605  3729 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:49.646164  3729 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.651748  3738 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:49.653378  3738 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.659219  3745 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:49.661115 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:49.661218 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:49.661964 29220 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:06:49.647665 29065 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:49.665184 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:49.665380 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:49.590431 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46751.5us
I20260709 13:06:49.533017  3252 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:49.668720 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:49.677217  3180 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:49.678203  3180 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.681486  3180 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:49.685196 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 148402us
I20260709 13:06:49.601954 29186 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:49.699997 29907 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:49.722997 29068 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
W20260709 13:06:49.718649 29221 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.004s	sys 0.008s
W20260709 13:06:49.716162 29216 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.013s	sys 0.028s
I20260709 13:06:49.568222  3772 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:49.729743 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 128023us
W20260709 13:06:49.730196 29216 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.097s	user 0.013s	sys 0.029s
I20260709 13:06:49.732890 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:49.733150 29221 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.005s	sys 0.009s
I20260709 13:06:49.729223 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:49.735255  3658 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.740302 29065 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 24: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:49.730420  3252 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:49.752017  3494 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:49.752341  3347 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:49.752048 29073 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.766063  3658 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:49.766654  3494 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.768203 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3046) took 1014 ms. Trace:
I20260709 13:06:49.769089  3683 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.756541  3252 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:49.771096  3252 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:49.769485  3494 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:49.773860 29222 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:06:49.775982 29220 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:49.769711  3514 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.761510  3347 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.798633 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:49.801492 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:49.802572 29215 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:06:49.768603 29219 rpcz_store.cc:276] 0709 13:06:48.753699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.755426 (+  1727us) service_pool.cc:224] Handling call
0709 13:06:48.791102 (+ 35676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.791304 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:49.513709 (+722405us) log.cc:844] Serialized 14 byte log entry
0709 13:06:49.704173 (+190464us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:49.722800 (+ 18627us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:49.735645 (+ 12845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.742981 (+  7336us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.768039 (+ 25058us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":535391,"prepare.queue_time_us":651775,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":58,"raft.queue_time_us":6443,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":2514,"thread_start_us":276004,"threads_started":2}
I20260709 13:06:49.808496  3683 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:49.809320 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:49.809862 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.801549  3772 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.813200  3658 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:49.817845 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:49.819736 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 1.37568e+06us
I20260709 13:06:49.820866 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:49.825618  3514 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:49.829075  3700 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 1.003s	user 0.011s	sys 0.008s
I20260709 13:06:49.830181 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1200) took 2392 ms. Trace:
I20260709 13:06:49.830608 29216 rpcz_store.cc:276] 0709 13:06:47.437747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.021019 (+583272us) service_pool.cc:224] Handling call
0709 13:06:48.021331 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.021546 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:06:48.061896 (+ 40350us) log.cc:844] Serialized 14 byte log entry
0709 13:06:49.771690 (+1709794us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:49.783372 (+ 11682us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:06:49.829835 (+ 46463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.829943 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.830123 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1563,"prepare.queue_time_us":40636,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":14773,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":1667571,"threads_started":2,"wal-append.queue_time_us":906967}
I20260709 13:06:49.832281 29217 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:06:49.846896  3707 consensus_meta.cc:287] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.611s	user 0.004s	sys 0.014s
I20260709 13:06:49.850116  3773 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.853983 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:49.854383 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:49.854987 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:49.855703 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:49.874022 29143 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.875835 28963 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 13 to 15, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 15 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.877831 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:49.878722 29067 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:06:49.878822 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:49.878417 28963 catalog_manager.cc:5697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 6 to 18, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 18 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.879679 29064 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:49.880290 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:49.880643 29065 pending_rounds.cc:85] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 15
I20260709 13:06:49.881001 29065 pending_rounds.cc:107] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 13:06:49.881326 29065 raft_consensus.cc:2889] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:49.881281 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:49.882500 29071 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:49.883725 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:49.884657 29069 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:49.860790 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:49.888087  3745 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:49.888744  3745 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.889163 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:49.891073 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:49.891422  3745 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:49.891899 29903 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:06:49.892880 29143 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
W20260709 13:06:49.863979 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:49.898030 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:49.869683 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:49.898770 29903 raft_consensus.cc:2368] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:06:49.899117 29217 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:49.899755 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:49.900539 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:49.901444 29909 raft_consensus.cc:2368] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17.
I20260709 13:06:49.799714 29214 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:49.859844 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:49.902693 29146 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:49.892463 28963 catalog_manager.cc:5697] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 11 to 12, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 12 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.903993 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:49.904281 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:49.904750 29220 raft_consensus.cc:3060] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:06:49.905268 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:49.906905 28963 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:49.908313 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:49.891957 29907 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 24: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:49.861673 29073 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:49.893900 29037 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:49.894017 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:49.880646 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:49.902032  3781 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
I20260709 13:06:49.915477 29220 raft_consensus.cc:2468] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
W20260709 13:06:49.917320 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 96622.1us
I20260709 13:06:49.917423 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:49.862258  3347 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:49.920276  3494 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:49.922740 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:49.923875 29144 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:06:49.924598 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:49.925191 29216 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.927393 29148 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:49.928445 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:49.930603 29219 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:49.918388  3604 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.937803 29147 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:49.908262  3780 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:49.941237 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:49.942957 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:49.930369  3700 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:49.948736 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:49.935410  3596 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:49.950771 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3042) took 1198 ms. Trace:
I20260709 13:06:49.952303 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:49.967518 29142 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:49.971024  3700 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:49.976290 29113 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:49.982301 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:49.982553 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:49.983130 29146 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 15.
I20260709 13:06:49.985558 28963 catalog_manager.cc:5697] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:06:49.979496  3780 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:49.991649 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:49.961989 29213 rpcz_store.cc:276] 0709 13:06:48.752493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.752782 (+   289us) service_pool.cc:224] Handling call
0709 13:06:48.787567 (+ 34785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.787780 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:06:49.562440 (+774660us) log.cc:844] Serialized 14 byte log entry
0709 13:06:49.920007 (+357567us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:49.929212 (+  9205us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:06:49.929277 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:49.937488 (+  8211us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:49.950698 (+ 13210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":901,"prepare.queue_time_us":654671,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":54606,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":8424,"thread_start_us":950217,"threads_started":2}
I20260709 13:06:49.954768  3494 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:49.945063  3604 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.005347 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:50.007370 29144 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:50.022492 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:49.997112  3596 raft_consensus.cc:2764] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:06:49.957823  3580 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:06:50.040588 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:50.041720 29186 leader_election.cc:400] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:06:50.049075 29186 leader_election.cc:403] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:50.049420 29147 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
W20260709 13:06:50.064576  3528 consensus_meta.cc:287] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.966s	user 0.015s	sys 0.007s
I20260709 13:06:50.065115  3528 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.067410  3528 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.068135  3773 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:50.072104  3771 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.072989  3771 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.078264  3758 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:50.078763 29874 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:50.079582 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 130259us
I20260709 13:06:50.080672  3780 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.081440  3780 raft_consensus.cc:2804] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:06:50.025646  3729 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:50.082413  3729 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.085085  3729 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.086089 29902 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:06:50.089485 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 158595us
I20260709 13:06:50.090431 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:50.091529 29215 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:50.079831 29142 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:50.091856 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:50.104454 29148 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
W20260709 13:06:50.107963 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 113568us
I20260709 13:06:50.059973  3596 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:50.112941  3596 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:50.083060  3604 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:50.113971 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.113389  3596 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:50.112036 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:50.118525 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.118819 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.119771 29213 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:50.018810  3700 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.122125 29186 leader_election.cc:400] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17.
I20260709 13:06:50.122496 29186 leader_election.cc:403] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:06:50.079095  3758 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:50.124537  3499 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:50.057210  3393 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:50.095240  3771 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:50.092094  3658 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:50.135149 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 289728us
W20260709 13:06:50.149390 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 266729us
I20260709 13:06:50.126425  3780 raft_consensus.cc:697] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 24 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:50.134148  3499 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:50.155148 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:50.148360 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:50.146431  3477 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:50.160254  3499 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:50.183367 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1212) took 1425 ms. Trace:
I20260709 13:06:50.187616 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:50.194000 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 68246us
I20260709 13:06:50.183506  3477 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:50.185838  3180 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:50.203980 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3040) took 1402 ms. Trace:
I20260709 13:06:50.204342  3776 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
W20260709 13:06:50.205299 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 113555us
W20260709 13:06:50.207679 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 161336us
I20260709 13:06:50.195322  3771 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:50.190448  3597 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:50.160436  3393 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.210536 29064 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
W20260709 13:06:50.220126 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 113854us
I20260709 13:06:50.200976  3738 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:50.206881 29218 rpcz_store.cc:276] 0709 13:06:48.751532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.752384 (+   852us) service_pool.cc:224] Handling call
0709 13:06:48.789078 (+ 36694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.789267 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:49.464780 (+675513us) log.cc:844] Serialized 14 byte log entry
0709 13:06:50.112537 (+647757us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:50.122390 (+  9853us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:50.133944 (+ 11554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.138134 (+  4190us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.154329 (+ 16195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50558,"prepare.queue_time_us":653553,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":53602,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":452375,"threads_started":2,"wal-append.queue_time_us":631843}
I20260709 13:06:50.230206 29217 raft_consensus.cc:2410] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:06:50.185369 29221 rpcz_store.cc:276] 0709 13:06:48.757519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.757630 (+   111us) service_pool.cc:224] Handling call
0709 13:06:48.757886 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:48.758091 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:06:49.737056 (+978965us) log.cc:844] Serialized 14 byte log entry
0709 13:06:50.160371 (+423315us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:50.181039 (+ 20668us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:06:50.182717 (+  1678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:50.182928 (+   211us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:50.183246 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":683966,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":460,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":16197,"thread_start_us":1303708,"threads_started":2,"wal-append.queue_time_us":405795}
I20260709 13:06:50.235042 29215 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:50.208763  3477 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:50.238440  3494 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.252035 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:50.245500  3580 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:50.245993 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:50.248873 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 121601us
I20260709 13:06:50.246970 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:50.261304 29071 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:50.250897 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:50.197052  3658 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.265246 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.269510 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:50.270268 29068 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:50.271229 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:50.272094 29068 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:06:50.270269 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:50.274544 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:50.275889 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:50.236588  3604 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:50.280831 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:50.267500  3776 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:50.281550  3780 consensus_queue.cc:237] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.224934  3758 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.288089 29064 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:50.288554  3580 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.310786  3776 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.311446  3580 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:50.291322 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:50.312958 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:50.315860  3683 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:50.316327  3683 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.316705  3683 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:50.316926 29065 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:06:50.318316  3180 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:06:50.318907  3252 consensus_meta.cc:287] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.000s
I20260709 13:06:50.319301  3252 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.305640 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:50.316840 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:50.325554 29068 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:06:50.273485  3738 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:50.309396  3393 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:50.322062 29112 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:50.322589  3494 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:50.323040  3724 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:06:50.298908 29216 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:50.328292 29902 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:50.275288  3771 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.361014 29067 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:50.355705  3604 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.364723  3604 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.365594 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.365851 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.366472 29220 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:50.366531 29216 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:50.362237  3771 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.370332 29071 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:50.368281  3363 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:50.332592  3514 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17.
W20260709 13:06:50.374469  3788 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 13:06:50.375149 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:50.376179  3738 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.377506  3596 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.378153  3477 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:50.360589  3783 log.cc:927] Time spent T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.209s	user 0.001s	sys 0.001s
I20260709 13:06:50.332388  3494 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.384084 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.385000 29217 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
W20260709 13:06:50.386690 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 69944.7us
I20260709 13:06:50.386839 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.387565 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:50.384187  3494 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:50.388411 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:50.389122 29908 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:06:50.390448 29221 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:50.393833 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.395121  3363 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:06:50.395190  3683 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.396288  3776 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:50.395691  3363 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:50.397902  3683 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:06:50.368402  3707 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:50.398142  3252 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.397771  3363 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.399060 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.399454  3658 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:50.399803 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:50.400789 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44544.6us
I20260709 13:06:50.400918 29216 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:50.406682 29217 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:50.333381  3180 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:50.402858 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 88009.6us
I20260709 13:06:50.410259  3798 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:50.420003  3798 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.402134  3597 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:50.422029  3798 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:50.423029  3597 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.424132 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.425057 29217 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:50.379904  3596 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:50.427260 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.428764  3499 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.430984  3499 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.432945 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:50.434728  3363 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:50.435124  3771 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:50.435508 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:50.436367  3771 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:50.436812  3771 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.437228  3771 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:50.439644  3798 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:50.445495  3347 consensus_meta.cc:287] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.000s
I20260709 13:06:50.445257 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:50.453940 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:50.452732 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 92819.4us
I20260709 13:06:50.454829 29217 raft_consensus.cc:2393] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 24: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:50.415388  3477 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:50.454289 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 66844us
I20260709 13:06:50.457005 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.456475  3347 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.459458 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:50.459630 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 52095.3us
I20260709 13:06:50.460358 29905 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 20: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:50.462189 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:50.466293 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:50.468961 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.468750 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.470108 29112 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:50.470618 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:50.468166 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:50.471688 29214 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:50.472663 29142 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:06:50.476183 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.477795 29146 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:50.478945  3776 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:50.463806  3180 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.481966  3180 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.481788 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.484390 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.484997 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:50.485260 29147 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:50.486006 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.486747 29216 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:50.488095 29213 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:50.491957 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:50.494963  3347 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:50.495549 28963 catalog_manager.cc:5697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 13 to 16, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 16 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.499673 28963 catalog_manager.cc:5697] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 16 to 17, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 17 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:50.501086 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 48358.6us
I20260709 13:06:50.500412 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.454406  3798 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.515273 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:50.516340 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 71170.5us
I20260709 13:06:50.517894 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.519116 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.520236 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.518031 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.521658 29214 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:50.521791 29216 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 23: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:50.516242  3494 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.520005  3363 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:50.522956  3595 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:50.523358  3595 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.523711  3595 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:50.524317  3494 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:50.525139 29148 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:50.526928 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.528666 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.529476 29219 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:50.530792  3771 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:50.532300 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70470.2us
I20260709 13:06:50.532994  3771 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:50.533339 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:50.534426 29871 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:50.535133 29875 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:50.536120  3528 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:50.536576  3528 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:50.536981  3528 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:50.537379  3729 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:06:50.538362  3729 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:50.521713  3646 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:50.539750  3646 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:06:50.541234 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 82354.2us
I20260709 13:06:50.541106  3646 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.542164 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:50.526422  3724 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:50.544862  3724 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:50.545225  3771 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:50.546018  3771 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.547781 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:50.545338  3724 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:50.535583 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:50.559792 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:50.560851  3700 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:50.561300  3700 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:50.561628  3700 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:50.564236 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.566510  3772 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:50.566967  3772 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.567322  3772 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:50.548488  3771 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:50.558879 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.552429  3724 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:50.557818  3780 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:50.586812  3780 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:50.587862 29219 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:50.589524  3738 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:50.590341  3738 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.592567 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.593202 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.593817 29146 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:50.594842 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.595592 29144 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:50.596266 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.597092 29143 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:06:50.593840 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.598810  3738 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.600530 29148 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:06:50.555998  3745 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:50.587971  3780 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.601267 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:50.604318 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:50.605120 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 51630.3us
I20260709 13:06:50.605887 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:50.578306  3528 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.606842  3347 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:50.607224  3347 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:50.607599  3347 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:50.608017  3528 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:50.607882  3595 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.608767  3771 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:50.614748 28962 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 242) took 30517 ms (client timeout 15000 ms). Trace:
I20260709 13:06:50.616796 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:50.616688  3724 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.618163 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:50.605118 29140 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:50.604413  3745 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:50.621836 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:50.604339 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:50.619666 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:50.630846 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 60803.3us
I20260709 13:06:50.630743  3738 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:50.634734  3738 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:50.635146  3738 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:50.623955  3745 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:50.648828 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 68239.4us
I20260709 13:06:50.630033  3528 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:50.653728  3595 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.654963 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:50.656016 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.657334 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.658115 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.658970 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.659772 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.661841 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.662636 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.663298 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.664049 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.667806 29142 raft_consensus.cc:2368] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:50.667824 29143 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:50.668251 29146 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:50.669356 29148 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:50.648332  3604 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:50.685499  3604 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:50.671280 29147 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:50.672255  3595 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:50.686645  3604 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.673216  3724 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.677066 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.678627 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.689409 29143 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:06:50.690260  3347 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.681469 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.692497 29219 raft_consensus.cc:2368] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:50.693804 29113 leader_election.cc:400] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:50.694082 29113 leader_election.cc:403] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:06:50.684306  3528 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:50.670797 29144 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:50.694953  3528 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:50.696025  3528 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.697096  3347 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:50.697250 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.698501 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.684778 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:50.699121 29146 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:50.700104  3729 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:50.700536  3729 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.700896  3729 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:50.702603 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.703267 29139 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:06:50.704293 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.699975 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.705106 29142 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
W20260709 13:06:50.615283 28962 rpcz_store.cc:269] 0709 13:06:20.102834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:20.196783 (+ 93949us) service_pool.cc:224] Handling call
0709 13:06:50.614475 (+30417692us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:41.159352 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:41.160424 (+  1072us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:41.160511 (+    87us) write_op.cc:435] Acquired schema lock
0709 13:06:41.160553 (+    42us) tablet.cc:662] Decoding operations
0709 13:06:41.162975 (+  2422us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:41.163092 (+   117us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:41.163132 (+    40us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:06:41.788405 (+625273us) tablet.cc:701] Row locks acquired
0709 13:06:41.815015 (+ 26610us) write_op.cc:260] PREPARE: finished
0709 13:06:41.815220 (+   205us) write_op.cc:270] Start()
0709 13:06:41.815398 (+   178us) write_op.cc:276] Timestamp: P: 1783602401815145 usec, L: 0
0709 13:06:41.815438 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:06:41.816387 (+   949us) log.cc:844] Serialized 8291 byte log entry
0709 13:06:43.725971 (+1909584us) op_driver.cc:464] REPLICATION: finished
0709 13:06:44.484833 (+758862us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f7833a030ac 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f783a543ef0
0709 13:06:46.433022 (+1948189us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b180036e5c0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:06:46.433207 (+   185us) write_op.cc:301] APPLY: starting
0709 13:06:46.433306 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:46.687776 (+254470us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:46.687819 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:47.853783 (+1165964us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:47.854325 (+   542us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:06:47.854452 (+   127us) write_op.cc:312] APPLY: finished
0709 13:06:47.855312 (+   860us) log.cc:844] Serialized 188 byte log entry
0709 13:06:48.594203 (+738891us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:48.594409 (+   206us) write_op.cc:454] Released schema lock
0709 13:06:48.594618 (+   209us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:48.595059 (+   441us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:48.595093 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4110000,"reported_tablets":49,"spinlock_wait_cycles":44416,"child_traces":[["op",{"apply.queue_time_us":1105746,"apply.run_cpu_time_us":70814,"apply.run_wall_time_us":2161812,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91907,"num_ops":21,"prepare.queue_time_us":474273,"prepare.run_cpu_time_us":39828,"prepare.run_wall_time_us":1078837,"raft.queue_time_us":474312,"raft.run_cpu_time_us":41866,"raft.run_wall_time_us":1109410,"replication_time_us":1910364,"spinlock_wait_cycles":33262592,"thread_start_us":2650946,"threads_started":5,"wal-append.queue_time_us":1688401,"wal-append.run_cpu_time_us":188837,"wal-append.run_wall_time_us":3355775}]]}
I20260709 13:06:50.712462  3700 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.706630  3724 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:50.706981 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.714622 29144 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:50.706631 29141 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:06:50.707261  3780 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:50.708307 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.717067 29148 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:50.709370 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:50.708987 28963 catalog_manager.cc:5697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.710084  3393 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:50.691830  3738 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.705392 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.732023 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.733580  3738 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.733103 29141 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:50.734990 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.736388 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.738801  3347 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:50.739414  3347 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.738921 28963 catalog_manager.cc:5697] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 9 to 13, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 13 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.749935  3347 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:50.753773 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.754928 29146 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
W20260709 13:06:50.761313 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 111708us
I20260709 13:06:50.761658 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.765609 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.766340 29141 raft_consensus.cc:2376] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22. Re-sending same reply.
I20260709 13:06:50.767501 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.772430 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.773329 29148 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:50.788223 29148 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:50.748406  3646 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:50.730149  3729 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.730803  3772 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:50.805016 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 95734.8us
I20260709 13:06:50.803901 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.808058 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.808808 29148 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:50.809056 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:50.809674 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:50.806734 29141 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:50.813269  3803 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:50.813195 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.814181 29142 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:50.820492 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:50.827981 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:50.827834 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:50.829111 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.829865 29148 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:50.830765 29905 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:50.831002  3700 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:50.837507 29148 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:50.840070 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:50.842713 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:50.843963 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 211923us
I20260709 13:06:50.759564 28963 catalog_manager.cc:5697] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:06:50.852780  3745 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.860947 29142 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:50.866787 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:50.867581 29909 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 21: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:50.870858  3772 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.872166  3729 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:50.877840  3745 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:50.880218 29905 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:06:50.890618 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:50.896781 29142 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:50.897238 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:50.899206 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:50.900157  3597 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:50.900828  3597 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:50.901269  3772 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:50.901717 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.902089 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:50.902876 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.902333 29146 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:50.903456  3772 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:50.903895  3772 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.904277  3772 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:50.903461 29140 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:50.893822 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:50.906306 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:50.907868 29909 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:06:50.911130 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.924182 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:50.927446 29909 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:06:50.930671 29144 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:06:50.931100 29144 pending_rounds.cc:85] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 14
I20260709 13:06:50.931353 29144 pending_rounds.cc:107] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:06:50.931633 29144 raft_consensus.cc:2889] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:50.932449 29146 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:06:50.933061 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.934490 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.935148 29143 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:06:50.933985  3604 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:50.936094  3347 consensus_queue.cc:1059] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:50.937245  3347 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:50.937613  3347 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:50.938861 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:50.939605 29067 raft_consensus.cc:2368] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
I20260709 13:06:50.940310 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:50.938346  3780 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:06:50.943416 29072 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:50.942018  3683 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:06:50.944072  3780 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:50.944494  3780 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:50.944705  3683 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:50.945042 29142 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:50.945168  3683 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:50.936287 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:50.945432  3780 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.942078  3773 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:50.947661  3773 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.948781 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:50.950055  3738 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:50.947832  3683 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:50.947120  3780 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:50.964017 29908 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:50.964767 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 48897.4us
I20260709 13:06:50.967408 29221 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:06:50.968184 29038 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:50.968374 29039 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.667s (SENT)
I20260709 13:06:50.968842 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:50.969753 29143 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:50.957470  3745 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:50.950579  3738 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:50.979261 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:50.986200  3724 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:50.938455  3347 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.976367  3683 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:50.964850  3252 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 13:06:50.991864 29140 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:51.008718 29218 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:06:51.011216 29220 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:06:51.019359 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:51.015834  3773 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:50.961375  3604 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:50.997383  3817 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:50.996855  3347 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:51.000097  3252 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:51.026593  3646 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.012s
I20260709 13:06:51.038910  3813 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:06:51.049531 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:51.039446  3814 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:06:51.053239  3604 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:50.992864  3810 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:06:51.011662  3180 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 13:06:51.011816  3724 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.039s
I20260709 13:06:51.014757  3772 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:51.064945 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.069769 29220 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:06:51.072892 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.066148 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.073832 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.076531 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.076581 29073 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:06:51.075847 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.077502 29068 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 22: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:51.077503 29071 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:06:51.078649 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.080404 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.081658 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.063179  3729 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:06:51.083141 29066 raft_consensus.cc:1275] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:06:51.065441  3745 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.084211  3745 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:51.074740  3738 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:51.086580  3745 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:51.086712 29064 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:51.063066  3580 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:51.031427  3776 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:06:51.073865 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.102689 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.090332 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:51.060963  3817 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:51.110664 29214 raft_consensus.cc:2410] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 24 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:06:51.112761 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.113408 29070 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:51.118333 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:51.119208 29219 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:51.114804 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:51.117508 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:51.126322 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:51.127576 29068 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:06:51.129808 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:51.073582  3803 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 13:06:51.135751  3771 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:51.114734  3494 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:06:51.164935  3771 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.172652  3771 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:51.154645  3745 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:51.177114  3499 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:06:51.177596  3499 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:51.178527  3499 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:51.181495  3817 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:51.188200  3514 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:51.188658 29140 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:06:51.188776  3514 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:51.202883 29070 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:06:51.206720  3252 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:51.206737  3824 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:51.220371 29215 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:51.222779 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 92791.2us
I20260709 13:06:51.223866 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:51.179850  3499 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:51.230208 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.199644 29071 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:06:51.236626  3514 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:51.253011 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.253579 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.253860 29070 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:51.254916 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.255117 29068 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 13:06:51.255846 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.177147  3772 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:51.268373 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.269739 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.278041 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.278595 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:51.279263 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 53784us
I20260709 13:06:51.281795 29215 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:51.283221 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.285261 29073 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:51.285562 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.286331 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.286749 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.287087 29073 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:51.283442  3707 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:51.287882 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.291282 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.288154 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.296126 29070 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:51.291523 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.297605 29068 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 14: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:51.284204 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:51.299736 29140 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:06:51.300004 29066 raft_consensus.cc:2368] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:51.300323 29069 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:06:51.296638 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.308477 29073 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:06:51.309931 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.310828 29073 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:51.315558 29067 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:51.316730 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.317806 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:51.312043 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 161979us
I20260709 13:06:51.319254 29073 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:51.319502 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.320317 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.321784 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.322218 29070 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:51.322383 29871 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.227135  3832 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:51.318118 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.331435 29140 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:51.334964 29140 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:06:51.338188 29140 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:42054
I20260709 13:06:51.339457 29871 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.339519 29140 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:51.340641 29140 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:51.343286  3738 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:51.351657 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.352586 29141 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:51.353972 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:51.354545 29141 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:51.362609 29141 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:51.362746  3738 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:51.364660 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.364940 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:51.365463 29148 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:51.367714 29148 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:51.367877 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:51.368701 29145 raft_consensus.cc:2368] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17.
I20260709 13:06:51.370724 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.371290 29145 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:06:51.372895 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:51.373600 29145 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:51.376382 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 75685.4us
W20260709 13:06:51.377424 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46451.1us
W20260709 13:06:51.378432 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.379604  3831 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:51.377548 29039 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:51.381561 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.382956 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.349365 29140 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:51.386186 29145 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 17. Ops: []
I20260709 13:06:51.389338 29073 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:51.390379 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.393550 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.394390 29148 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:06:51.398563 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.399544 29142 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:51.401158 29148 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:51.315836  3494 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:51.330060  3514 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:51.331225  3832 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:51.335443  3824 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:51.407032 29140 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:51.293133  3707 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:51.408825 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:51.323666  3528 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.083s
I20260709 13:06:51.410076 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:51.411101 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:51.406534  3646 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:06:51.468822 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.469812 29148 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:51.471249 29141 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:06:51.472600 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 133266us
I20260709 13:06:51.478650 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.479020 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.484652 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.491307 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.492766 29142 raft_consensus.cc:2410] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:06:51.429386  3828 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:51.495069 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:51.496150 29066 raft_consensus.cc:2368] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:51.428431  3832 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:51.497277 29148 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:51.488659 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 76899.4us
I20260709 13:06:51.502373 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.500658 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.422170  3494 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:51.513738 29072 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:51.513487 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:51.517949 29140 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:51.515552 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.516440 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.519208 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.520291 29141 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:51.520737 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.522809 29066 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:51.523785  3836 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:06:51.527227  3836 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:51.531445 29140 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:51.533027  3595 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:51.520056  3393 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:51.495903  3780 consensus_queue.cc:1048] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 13:06:51.511360  3831 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:06:51.537091  3646 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:51.537971  3776 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:51.546506 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.548832 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:51.553057 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:51.576562 29142 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:51.554217 29065 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:51.562223 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:51.565068 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.571370 29219 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:51.572484 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:51.581625 29148 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:51.585520 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.587067 29140 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:51.588001 29214 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:51.589161 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:51.589918 29220 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:51.594303 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.538627  3816 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:51.603479 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:51.604447 29140 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:06:51.605273 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 227798us
I20260709 13:06:51.612996 29039 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:51.621160 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:51.543288  3595 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:51.628890 29215 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:06:51.632692  3514 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:51.634523  3842 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:51.636454 29214 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:51.573503  3776 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:51.646771 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 125250us
I20260709 13:06:51.615406 29146 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
W20260709 13:06:51.656486 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 43646.3us
I20260709 13:06:51.640192  3514 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:51.657886 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:51.661301 29188 leader_election.cc:400] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17.
I20260709 13:06:51.662881 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:51.653936 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 176508us
I20260709 13:06:51.664476 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:51.666498 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 164199us
I20260709 13:06:51.683175 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.683887 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:51.687230 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:51.694883 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:51.697636 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:51.692616 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:51.700299 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:51.701100 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:51.708611 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:51.701278 29214 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:51.712967 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:51.699820  3595 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:51.691260  3840 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:51.714869  3646 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:06:51.715107  3840 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:51.715191  3595 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:51.715880  3840 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:51.720098 29905 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:51.721819  3838 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:06:51.724720  3494 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:51.714210  3825 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:51.726112 29215 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
W20260709 13:06:51.727656 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:51.685667  3707 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:51.731673 29220 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:51.725777  3828 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:51.726049  3817 raft_consensus.cc:2764] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:06:51.755168  3841 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:51.692461  3816 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:06:51.762902 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:51.741285  3580 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:51.734665  3825 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:51.702356  3836 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:06:51.742663  3646 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:51.765594 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:51.768831 29111 leader_election.cc:400] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
I20260709 13:06:51.773133 29111 leader_election.cc:403] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:51.758939  3828 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:51.778007  3844 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:51.779007  3847 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:51.760573  3841 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:51.781936  3771 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.609s	user 0.010s	sys 0.000s
I20260709 13:06:51.766875  3827 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:51.770741  3825 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:51.767581  3597 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:51.824865 29909 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:06:51.827188 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 89512.8us
I20260709 13:06:51.769354  3580 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:51.738157  3776 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:51.831790 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 134285us
I20260709 13:06:51.862318 29185 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:51.833153 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 147022us
I20260709 13:06:51.823989  3828 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:51.868335  3849 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:06:51.860417 29909 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:51.825896  3595 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:51.826125  3817 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:51.884503 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 171666us
I20260709 13:06:51.874521  3597 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:51.882360 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 184035us
I20260709 13:06:51.825716  3841 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:51.886343 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:51.887351 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 203555us
I20260709 13:06:51.876502  3580 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:51.897845 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:51.904996 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 136297us
I20260709 13:06:51.906118 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.881870  3844 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:51.831066  3771 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:51.909560 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:51.892784  3597 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:51.874970  3836 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:06:51.915786 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.932183 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:51.933104  3862 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:51.933280 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:51.913537  3844 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.934741  3862 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:51.934945  3844 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:51.935258  3862 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:51.939370  3836 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:51.940059  3836 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:51.940989  3836 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:51.924059  3847 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:51.944095 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.943979  3646 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:51.945250  3846 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:51.945576 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:51.926887 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:51.927618  3180 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:51.946703  3180 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:51.928089  3499 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:06:51.945658  3846 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:51.924414  3771 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:51.947649 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 52052.4us
W20260709 13:06:52.065105 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 119657us
I20260709 13:06:52.066027 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.133188 29216 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:06:52.134720 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 76480.5us
I20260709 13:06:51.943259  3393 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:52.161971  3853 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.162712  3845 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
I20260709 13:06:52.163172  3839 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.265s
W20260709 13:06:52.164073 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:52.165050  3858 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.256s	user 0.000s	sys 0.000s
I20260709 13:06:51.956334  3494 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:51.976915  3846 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:52.204591 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 257522us
I20260709 13:06:52.163759  3843 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:06:52.225944 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 159979us
I20260709 13:06:52.245638 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.252040 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:52.255319 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 320781us
W20260709 13:06:52.277235 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.244145  3843 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:52.291033  3845 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:52.294484  3856 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:52.296166 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 530613us
W20260709 13:06:52.300455  3834 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:06:52.301064  3861 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:52.312211  3852 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:52.203583  3840 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.250439  3862 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.258247  3828 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:52.322225 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70446.5us
W20260709 13:06:52.319664  3824 consensus_meta.cc:287] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.886s	user 0.010s	sys 0.000s
I20260709 13:06:52.337368 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:52.324182  3843 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:52.350091  3840 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:52.224579  3853 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.351626  3870 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:52.368327 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.370625 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.376708 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.344969  3780 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:52.381340 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:52.387512 29111 leader_election.cc:400] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:52.390349 29111 leader_election.cc:403] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:06:52.371328  3870 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:52.403088 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.370893  3824 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:52.374686  3798 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:52.416762 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.344947  3852 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.376966  3845 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:52.382687  3707 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:52.401576  3180 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:06:52.475664 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 146329us
I20260709 13:06:52.476358 29902 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
W20260709 13:06:52.478348  3597 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.003s
I20260709 13:06:52.473590  3827 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:52.400835  3780 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:52.500475 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 254947us
I20260709 13:06:52.405203  3580 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.475124  3771 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:52.480140  3646 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.480755  3798 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:52.524439  3862 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.507053  3597 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.523105 29222 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:52.517743  3580 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:52.549664 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:52.507622 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 186503us
I20260709 13:06:52.489357  3499 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:52.554915 29067 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:06:52.490265  3180 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:52.503378  3827 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:52.537418  3832 consensus_meta.cc:287] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.023s	user 0.004s	sys 0.008s
I20260709 13:06:52.523170  3780 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:52.521270  3852 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:52.556886 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 61415.5us
I20260709 13:06:52.558223 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:52.516796  3850 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:52.596534  3707 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:52.596923  3499 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.597263  3875 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:52.611464 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 61912.1us
W20260709 13:06:52.595203  3844 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.000s
W20260709 13:06:52.594125 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 177460us
I20260709 13:06:52.610428 29146 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:06:52.604007 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:52.613924 29111 leader_election.cc:400] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:52.591881  3827 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:52.594730  3828 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:52.499156  3870 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:52.614909  3825 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.743s	user 0.004s	sys 0.019s
I20260709 13:06:52.538057  3646 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.553099  3862 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.628980  3870 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:52.616111  3499 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.631063 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:52.633984  3846 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.650928  3180 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.652240  3869 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:52.653227  3846 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:52.614102  3707 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.655581  3707 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:52.618781  3875 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.657918  3875 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:52.636577  3798 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:52.626376  3852 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:06:52.659502  3866 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:52.660104  3866 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:52.661029  3866 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:52.662118  3841 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.747s	user 0.003s	sys 0.007s
I20260709 13:06:52.662730  3841 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.651185 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:52.622200  3580 raft_consensus.cc:2764] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
I20260709 13:06:52.593406  3832 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.663713  3597 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:52.649828 29907 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:52.667253 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:52.681744 29146 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:52.668532 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:52.669085  3803 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:06:52.671132  3499 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:52.671319 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:52.671515  3773 raft_consensus.cc:3055] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:52.672254 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:52.673933  3845 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:52.674335 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3025) took 1292 ms. Trace:
I20260709 13:06:52.674722  3798 consensus_queue.cc:1243] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 20.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:06:52.687657 29147 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:52.696705  3884 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:52.687108  3773 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:52.698921  3862 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:52.617627  3844 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.681313  3494 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:52.711738 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 91170.8us
I20260709 13:06:52.712237 29218 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:06:52.713843 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:52.714118 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:52.715150 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:52.715473 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:52.715735 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:52.715875 29069 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:52.693888  3845 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:52.699347  3773 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:52.711083 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.716144 29073 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:52.622233  3850 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:06:52.702562  3844 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:52.716485 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:52.694090 29145 rpcz_store.cc:276] 0709 13:06:51.377113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.377227 (+   114us) service_pool.cc:224] Handling call
0709 13:06:51.377505 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:51.394745 (+ 17240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.668716 (+1273971us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b54001bc6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:52.668888 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19439872}
I20260709 13:06:52.732244 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3219) took 1180 ms. Trace:
I20260709 13:06:52.732802 29218 rpcz_store.cc:276] 0709 13:06:51.554587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.555073 (+   486us) service_pool.cc:224] Handling call
0709 13:06:51.555350 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:52.722438 (+1167088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.731860 (+  9422us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003395d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:52.732139 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2659963648}
W20260709 13:06:52.728986 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 77858.8us
I20260709 13:06:52.734344 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:52.736274  3845 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.736719  3843 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.737320  3841 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:52.737761  3845 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:52.739068  3843 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:52.740171  3853 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:52.739607 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:52.741385  3817 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:52.741941  3817 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:52.743086  3646 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:52.742897  3817 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.744612 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3019) took 1378 ms. Trace:
W20260709 13:06:52.744650  3888 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.099s	user 0.002s	sys 0.000s
I20260709 13:06:52.745026 29143 rpcz_store.cc:276] 0709 13:06:51.366456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.366557 (+   101us) service_pool.cc:224] Handling call
0709 13:06:51.366813 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:51.585937 (+219124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:51.585972 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:51.586019 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:51.586048 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:52.744361 (+1158313us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54003a48d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:52.744412 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:52.744551 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428665216}
I20260709 13:06:52.647375  3825 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.694250  3707 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.720974  3499 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.735246 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:52.754716 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:52.752162 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:52.750480 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 130859us
I20260709 13:06:52.760619 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.758061 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:52.764279 29215 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:06:52.762835 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:52.757298  3595 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:06:52.776736 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.775349 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:52.784037 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.756660  3798 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.757262  3841 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:52.756811  3843 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:52.765895  3803 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.775844  3861 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:52.831041  3881 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:52.757339  3707 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.833700  3707 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:52.762033  3499 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.786890  3853 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:52.837051  3853 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:52.837464  3853 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:52.837913 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:52.773442  3814 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:06:52.813771  3832 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:52.808394  3798 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.817775  3863 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:52.842564  3863 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:52.842648  3827 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.843544  3863 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.844470  3724 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:52.828981  3843 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.844861  3724 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:52.845693  3724 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.846582  3843 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:52.847105  3724 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:52.830348 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46460.2us
I20260709 13:06:52.830010  3494 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.849632 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:52.850782 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:52.850909 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.851533 29065 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:52.851946  3832 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:52.852643  3832 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:52.854180  3872 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.639s	user 0.011s	sys 0.000s
I20260709 13:06:52.856206  3863 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:52.860082  3499 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:52.864116 29213 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:52.800719  3597 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:52.865880  3597 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:52.866313  3597 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:52.801637  3875 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:52.854236 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.868408  3875 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.869590 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.870733  3875 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:52.806244  3887 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.872051  3887 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:52.873791  3853 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:52.874084  3347 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:52.874323  3887 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:52.786240  3869 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:52.828087  3893 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:06:52.879393 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:52.878749 29874 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:52.880887 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:52.883281 29143 raft_consensus.cc:2376] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21. Re-sending same reply.
I20260709 13:06:52.881755 29144 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:06:52.798357  3828 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:06:52.883745  3849 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:06:52.890852 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:52.891076 29144 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:06:52.891038 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 128273us
W20260709 13:06:52.871896 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 130483us
I20260709 13:06:52.889451 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:52.892828 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:52.898368 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:52.906548 29139 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 16: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:52.892226 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:52.893873 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:52.910290 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:52.911154 29143 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:52.910475 29218 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:52.914222 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:52.916435 29217 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:52.916982 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:52.918879 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:52.882053  3862 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:52.884300  3595 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:52.919814 29213 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:52.920766 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:52.924786 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:52.965081 29217 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:52.907079  3875 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.906766  3499 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:52.912086  3738 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:52.984624 29218 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:52.985663 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:52.831067  3803 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:53.001564 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:52.923072  3580 consensus_queue.cc:1059] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.084s
W20260709 13:06:53.006121  3851 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.102s	user 0.024s	sys 0.044s
I20260709 13:06:53.004019 29143 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
W20260709 13:06:53.009157 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 116390us
I20260709 13:06:52.969408  3840 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:53.006549  3851 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.223s	user 0.024s	sys 0.044s
I20260709 13:06:52.989120  3738 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.989406  3866 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:53.033459  3347 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:52.964584  3885 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 13:06:52.927794  3707 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:06:52.994642  3862 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:53.047039 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 137517us
I20260709 13:06:53.025494  3828 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:53.054857 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:53.028389  3846 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:53.076388 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 183792us
I20260709 13:06:52.989137  3875 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:53.077858 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 51770.8us
I20260709 13:06:53.078176 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:52.993995  3499 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.082343 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:53.037593  3738 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:53.082626 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:53.086143 29217 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:06:53.022325  3595 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:06:53.113843 29214 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:06:53.105754  3847 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:53.119669 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:53.122200 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:53.122640 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:53.045603  3866 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:53.126546 29113 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:53.103907  3825 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:53.123186 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:53.137311 29222 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:53.138536 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:53.138814 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:53.139012 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:53.106097  3700 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:53.045302  3840 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.148738 29220 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:53.108371  3846 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:53.118817  3841 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:53.124531  3849 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.159595 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:53.121665  3885 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:53.134042  3890 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:53.133667  3815 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.240s
I20260709 13:06:53.142429  3700 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:53.146438  3847 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:53.158445  3595 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:53.168162  3853 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:53.202754 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.172207  3846 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:53.207351 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55654.8us
I20260709 13:06:53.187462  3904 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:53.238705 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 156415us
I20260709 13:06:53.201829  3908 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.227854 29145 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:06:53.223173  3890 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:53.201280  3851 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.246627 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:53.240278 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:53.269133 29147 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:53.192817  3850 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:53.205973  3909 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:53.194759  3885 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:53.220839 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:53.243745  3853 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:06:53.263185 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 124281us
I20260709 13:06:53.178516  3849 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.241832 29214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:53.293358 29214 raft_consensus.cc:606] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:53.293824 29214 raft_consensus.cc:614] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:53.294391 29066 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:53.229707  3595 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.297986 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:53.320494 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:53.326303 29069 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:06:53.301911 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.327751 29067 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:53.329260 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:53.330229 29070 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:53.198274  3862 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:53.333920 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:53.286294 29909 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.335163 29909 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:53.261981  3890 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:53.272298  3803 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:53.339514 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.297420 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:53.271399  3850 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.344303 29113 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:53.282322  3909 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.242815  3904 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.284832  3851 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:53.381747 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.399665 29908 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:53.413986 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:53.360986 29064 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:06:53.395731 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:53.381652 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 49887.8us
I20260709 13:06:53.443058 29070 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:06:53.334926 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:53.358178 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:53.452172 29065 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:53.453040 29069 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:06:53.442880 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:53.272203  3700 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.356271 29068 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:53.457382 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:53.281162  3841 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:53.485363 29069 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
W20260709 13:06:53.487026 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 144823us
I20260709 13:06:53.462512 29067 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
W20260709 13:06:53.489504 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 268781us
W20260709 13:06:53.487273 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 247205us
I20260709 13:06:53.474242 29214 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:53.343375  3875 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:53.443006  3850 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:53.392139  3499 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:53.492295  3856 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 1.160s	user 0.015s	sys 0.000s
I20260709 13:06:53.353528  3890 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:53.497036 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:53.496532 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:53.497328 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:53.412598  3596 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:53.495908 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:53.505934 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:53.712993 29064 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:53.714236 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:53.437588  3846 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:53.385529  3828 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:53.511574 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:53.460041  3866 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:53.272070  3908 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.585772  3646 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:06:53.579064 29219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:53.749719 29219 raft_consensus.cc:606] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:53.563331  3738 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.545980  3856 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:53.576927  3798 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:53.553320  3514 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:06:53.540742 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.505424  3875 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.596146 29218 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:53.584708  3852 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:53.532078  3884 raft_consensus.cc:3055] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:53.570112  3180 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:06:53.573997  3827 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:53.705992 29142 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:53.547168 29068 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:53.834671  3798 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:53.537963  3913 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:06:53.846431  3798 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.846912 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:53.536939  3907 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:53.847998  3907 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.848668  3798 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:53.538661  3899 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
I20260709 13:06:53.577409  3911 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.854403  3738 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:53.537456  3916 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:53.497037  3920 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:53.855293  3920 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.855690  3920 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:53.585175  3700 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:53.564491  3902 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.365s	user 0.001s	sys 0.001s
I20260709 13:06:53.594055  3840 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:53.536100  3923 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:06:53.540232  3595 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:53.541738  3910 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:06:53.522162 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:53.877029 29214 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:53.581205  3851 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:53.549489 29067 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:53.707044 29909 raft_consensus.cc:2410] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:06:53.540701  3912 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:53.524328  3915 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:06:53.526326  3904 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:53.521308  3919 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.522783 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:53.359123  3849 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:53.451884  3909 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:53.520710 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:53.595136 29066 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:53.521226 29147 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:06:53.708469 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:53.701261 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:53.538897 29072 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:53.533332 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:53.896561 29217 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:53.551963  3776 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:53.531169 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:53.705428 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:53.710753 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:53.742416 29144 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:53.746882 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:53.885699  3912 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:53.734680 29112 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 2.136s (SENT) [suppressed 37 similar messages]
I20260709 13:06:53.902870 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:53.703769  3724 consensus_meta.cc:287] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.856s	user 0.010s	sys 0.006s
I20260709 13:06:53.748624 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:53.715816  3863 consensus_meta.cc:287] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.843s	user 0.005s	sys 0.008s
W20260709 13:06:53.750993 29871 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.015s	sys 0.007s
W20260709 13:06:53.910823 29871 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.015s	sys 0.007s
I20260709 13:06:53.913846 29871 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:53.701274  3853 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:53.704247  3813 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.353387  3847 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:53.787861 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:53.921206 29219 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:53.921211 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:53.705761  3596 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.924355 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:53.744943  3828 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:53.800732 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 67699.6us
I20260709 13:06:53.800846  3906 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:53.811636 29221 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:53.738107  3926 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:53.775260 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 274207us
I20260709 13:06:53.817225 29903 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:53.743398  3815 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:53.796053  3845 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.051s	user 0.015s	sys 0.000s
W20260709 13:06:53.827621 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:53.831044 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:53.777485  3929 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:53.749387  3646 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:53.728188  3850 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:53.812774  3597 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.946s	user 0.010s	sys 0.004s
W20260709 13:06:53.838964  3891 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:06:53.839422  3917 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.849797 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:53.573280  3903 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:06:53.854118  3911 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.522871 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:53.744755  3846 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.880170 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:53.834575  3180 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:53.825039  3852 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:53.829381  3884 raft_consensus.cc:740] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:53.889468 29910 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:53.909246 29145 raft_consensus.cc:2368] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:06:53.883766  3738 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:53.916411 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 205760us
I20260709 13:06:53.919265 29064 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 13:06:53.910511 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 386979us
I20260709 13:06:53.787286 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:53.922892 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:53.934373 29144 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:53.935509  3849 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:54.005774 29219 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:06:53.938863  3939 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.940536  3862 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:54.009605 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:53.941752 29066 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:06:54.013095 29215 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:06:53.942389 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:53.942950 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:53.953053 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:53.954665 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:53.955476 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:54.016294 29908 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:53.837357  3875 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.011083 29111 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:53.957202 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:53.941520  3724 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:54.025024  3724 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:54.025694  3724 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:54.027555 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 72976us
I20260709 13:06:54.028605 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:53.948280  3596 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:53.957942  3912 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:53.896780  3919 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:53.986159 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:53.986313 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:53.986605 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:53.965407  3906 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.967696  3815 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.965286  3926 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.955379  3874 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:06:53.925606  3853 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.987776  3879 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.017974  3863 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.010766 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:53.942021  3813 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.032617 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:54.072232 29145 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:54.032863 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:54.033799 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:53.980741  3597 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:54.035789 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 78708.6us
W20260709 13:06:54.038733 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 304086us
I20260709 13:06:54.082235 29904 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:06:54.041731 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:54.050555 29911 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:06:53.960165  3893 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:06:54.061986 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:54.084870 29148 raft_consensus.cc:2410] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 22 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
W20260709 13:06:54.081208 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 70517.5us
I20260709 13:06:54.070431 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.092350 29215 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:06:54.038934  3876 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:54.075214 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:54.035219 29185 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:54.045784 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:53.983610  3929 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.080152  3841 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:06:54.103178 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 68085.1us
I20260709 13:06:54.018803  3738 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.083295 29147 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:54.083871  3595 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:06:54.102070 29039 leader_election.cc:400] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
W20260709 13:06:54.101047 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 234241us
I20260709 13:06:54.104391 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:54.104923 29215 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:54.104983 29143 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:54.034461  3862 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:53.997826  3917 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.115393  3936 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:54.115783  3892 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:54.115875  3940 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:54.116441 29148 raft_consensus.cc:1240] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: [16.15-16.15]
W20260709 13:06:54.116617  3941 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.105s	user 0.001s	sys 0.001s
I20260709 13:06:54.117046 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:54.028265  3939 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.117748  3876 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:54.075708  3863 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.118192  3827 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:06:54.120645 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.072080  3879 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.096010  3514 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:06:54.117552  3943 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:06:54.063040  3180 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.107760  3580 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:54.075743  3646 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.105448  3852 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.092017 29219 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
W20260709 13:06:54.089546 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 51458us
W20260709 13:06:54.232640 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 151694us
I20260709 13:06:54.144847 29147 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:06:54.099309  3884 consensus_queue.cc:260] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.147890 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:54.235632 29069 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:54.291859 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:54.146585 29064 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
W20260709 13:06:54.257898 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 140936us
I20260709 13:06:54.102741  3907 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.273175 29144 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:06:54.286429  3876 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:54.304979 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 72205us
W20260709 13:06:54.310719 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 162961us
W20260709 13:06:54.288751 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.121309  3845 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:54.233946 29910 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:54.161736  3853 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.237157  3885 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:54.317589 29147 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:06:54.238090  3874 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:06:54.326488 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.318048 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:54.320147 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.326318  3846 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.353845 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3062) took 1145 ms. Trace:
I20260709 13:06:54.284459  3707 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.237550  3862 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.210335  3828 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.236481  3933 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:54.301883  3646 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:54.239583  3597 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.306380  3827 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:54.304594  3936 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:54.144471  3908 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.390893 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:54.349457  3595 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.116s
I20260709 13:06:54.316922  3852 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:54.442224  3948 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.266s	user 0.000s	sys 0.001s
I20260709 13:06:54.441534  3876 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:54.441615  3949 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:06:54.446280 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 1178) took 1049 ms. Trace:
I20260709 13:06:54.317447  3907 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:54.497570 29903 rpcz_store.cc:276] 0709 13:06:53.396671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.396802 (+   131us) service_pool.cc:224] Handling call
0709 13:06:54.446033 (+1049231us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54003fdc54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:54.446209 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":834654464}
I20260709 13:06:54.231158  3911 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:54.239720  3942 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:54.203048  3954 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:54.344581  3780 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:54.452054 29141 rpcz_store.cc:276] 0709 13:06:53.208680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.208834 (+   154us) service_pool.cc:224] Handling call
0709 13:06:53.245813 (+ 36979us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.287411 (+ 41598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.287463 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.340492 (+ 53029us) log.cc:844] Serialized 26 byte log entry
0709 13:06:53.775233 (+434741us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:53.784371 (+  9138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.784498 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.353449 (+568951us) raft_consensus.cc:1685] finished
0709 13:06:54.353534 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.353752 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27130,"raft.queue_time_us":130707,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":44923,"thread_start_us":375076,"threads_started":1,"wal-append.queue_time_us":486958}
I20260709 13:06:54.510238 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3228) took 1622 ms. Trace:
I20260709 13:06:54.510660 29221 rpcz_store.cc:276] 0709 13:06:52.887795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.888208 (+   413us) service_pool.cc:224] Handling call
0709 13:06:52.888781 (+   573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.958631 (+1069850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.509991 (+551360us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54002f5dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:54.510178 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2121858944}
I20260709 13:06:54.461090 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.463198 29218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:54.463267 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:54.311810  3824 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.135s
I20260709 13:06:54.447134  3850 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.471194 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:54.423554 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.473438 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.474807  3862 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:06:54.476133 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3051) took 1183 ms. Trace:
I20260709 13:06:54.478639 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:54.482035  3776 consensus_meta.cc:287] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.573s	user 0.021s	sys 0.014s
I20260709 13:06:54.460132  3707 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.483456  3936 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.484117  3815 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:54.484759  3926 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:54.484900 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:54.533949 29147 raft_consensus.cc:2410] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:06:54.486415 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 1180) took 1072 ms. Trace:
W20260709 13:06:54.487183  3959 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:06:54.536787  3957 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.097s
I20260709 13:06:54.537261 29147 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:06:54.488049  3947 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.374s	user 0.000s	sys 0.000s
I20260709 13:06:54.539201 29147 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:54.491070  3724 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.540143 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 1083) took 1186 ms. Trace:
I20260709 13:06:54.540602 29072 rpcz_store.cc:276] 0709 13:06:53.354231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.899645 (+545414us) service_pool.cc:224] Handling call
0709 13:06:53.900051 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.900404 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.900438 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.900928 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:06:53.974466 (+ 73538us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:54.018089 (+ 43623us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.018387 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.018462 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.539906 (+521444us) raft_consensus.cc:1685] finished
0709 13:06:54.539945 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.540079 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":174961,"prepare.queue_time_us":43792,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":287,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":5966,"spinlock_wait_cycles":11252480,"thread_start_us":73161,"threads_started":1,"wal-append.queue_time_us":132801}
I20260709 13:06:54.542209  3724 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.542936 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:54.543619 29072 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:06:54.544303  3707 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:54.544713 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:54.545083  3347 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.545603  3347 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.545902 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:54.546514 29072 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:54.547583 29072 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:54.492364  3951 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.548208  3951 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.534356 29904 rpcz_store.cc:276] 0709 13:06:53.414383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.504163 (+ 89780us) service_pool.cc:224] Handling call
0709 13:06:54.486126 (+981963us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b540063d654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:54.486347 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433339776}
I20260709 13:06:54.550683  3951 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:54.492872  3961 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:54.493944 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:54.551596 29143 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:54.551482  3961 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.496241  3945 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.192s
I20260709 13:06:54.162545  3892 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.504400  3874 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:54.557119  3874 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.511901 29215 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 14.
I20260709 13:06:54.512136 29218 raft_consensus.cc:606] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:54.512296 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 49126.5us
I20260709 13:06:54.559439 29218 raft_consensus.cc:614] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:54.522894  3952 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.219s
I20260709 13:06:54.505885  3942 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.524865 29073 rpcz_store.cc:276] 0709 13:06:53.285864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.287112 (+  1248us) service_pool.cc:224] Handling call
0709 13:06:53.287478 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:53.287717 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:53.743823 (+456106us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.239544 (+495721us) spinlock_profiling.cc:243] Waited 7.99 ms on lock 0x7b18003ec740. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f783a5053c7 00007f7836b179a7 00007f7836b270bf 00007f7836b27038
0709 13:06:54.467626 (+228082us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.469160 (+  1534us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:54.469234 (+    74us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:54.469301 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.469402 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.469589 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16390,"prepare.queue_time_us":813111,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":79,"raft.queue_time_us":11441,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":25336,"spinlock_wait_cycles":26538112,"thread_start_us":900531,"threads_started":3,"wal-append.queue_time_us":351429}
I20260709 13:06:54.566834 29073 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:54.554194 29143 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:54.509616  3954 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:54.562765 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 244785us
I20260709 13:06:54.521059  3847 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:54.519735  3907 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:54.468771  3933 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.574469  3961 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.576040  3892 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:54.576877  3958 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:06:54.577649 29215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:54.578565 29072 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 15. Current term is 16. Ops: []
W20260709 13:06:54.577657  3946 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.373s	user 0.000s	sys 0.001s
I20260709 13:06:54.589221 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 1082) took 1264 ms. Trace:
I20260709 13:06:54.516398  3780 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.589690 29067 rpcz_store.cc:276] 0709 13:06:53.325163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.882028 (+556865us) service_pool.cc:224] Handling call
0709 13:06:53.882454 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.882949 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.882993 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.892584 (+  9591us) log.cc:844] Serialized 26 byte log entry
0709 13:06:54.106896 (+214312us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:54.301351 (+194455us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.301601 (+   250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.301675 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.588980 (+287305us) raft_consensus.cc:1685] finished
0709 13:06:54.589004 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.589144 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":195455,"prepare.queue_time_us":198063,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":19191,"raft.queue_time_us":3250,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":13895,"thread_start_us":147414,"threads_started":1,"wal-append.queue_time_us":289808}
I20260709 13:06:54.573769  3847 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.569835  3942 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:54.572037  3863 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.606381 29073 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:06:54.570161  3954 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:54.611661 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.613488 29038 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:54.608597  3862 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:54.572193  3875 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.704698 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:54.627525 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 57312us
I20260709 13:06:54.721478 29220 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:54.628321 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:54.698137 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 1080) took 1399 ms. Trace:
I20260709 13:06:54.637324 29214 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:54.682663 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:54.640578 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:54.644397  3944 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.627s	user 0.004s	sys 0.000s
I20260709 13:06:54.661233  3845 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.658174 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:54.677829 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:54.687808 29909 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 1172) took 3279 ms (client timeout 2136 ms). Trace:
I20260709 13:06:54.685498 29065 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:06:54.748097  3825 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:54.677644  3738 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:54.633723 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.525409  3827 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.749572 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3069) took 1152 ms. Trace:
I20260709 13:06:54.750092  3827 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:54.749974 29139 rpcz_store.cc:276] 0709 13:06:53.597450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.597586 (+   136us) service_pool.cc:224] Handling call
0709 13:06:53.597960 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.598167 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.598190 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.700164 (+101974us) log.cc:844] Serialized 26 byte log entry
0709 13:06:53.821548 (+121384us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:53.821605 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:53.821696 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.643502 (+821806us) spinlock_profiling.cc:243] Waited 9.51 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:54.749293 (+105791us) raft_consensus.cc:1685] finished
0709 13:06:54.749315 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.749521 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21870464,"thread_start_us":120906,"threads_started":1,"wal-append.queue_time_us":260335}
I20260709 13:06:54.752266 29139 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:06:54.706791 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:54.708041 29146 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 16. Current term is 17. Ops: []
W20260709 13:06:54.755205 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.755368 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:54.633669 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.706321 29073 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:06:54.709867 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:54.607694  3847 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:54.592490  3917 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.632946 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.716758 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:54.716953 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:54.629845  3939 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:54.765214 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55451.2us
I20260709 13:06:54.766199 29902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 1175) took 1363 ms. Trace:
I20260709 13:06:54.766703 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:06:54.720925 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.723024  3803 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:54.710817  3595 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:54.724799  3772 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.725615  3810 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:06:54.725796 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 97570.8us
I20260709 13:06:54.778337 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:54.779222 29038 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:54.726536  3906 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.761543  3845 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:54.728112 29070 rpcz_store.cc:276] 0709 13:06:53.299031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.580690 (+281659us) service_pool.cc:224] Handling call
0709 13:06:53.581117 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:53.968532 (+387415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.968559 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:53.978580 (+ 10021us) log.cc:844] Serialized 26 byte log entry
0709 13:06:54.381054 (+402474us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:54.465789 (+ 84735us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.469924 (+  4135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.647377 (+177453us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b1800078640. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1
0709 13:06:54.650461 (+  3084us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b1800078640. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:06:54.653381 (+  2920us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b1800078640. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419
0709 13:06:54.653902 (+   521us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400187254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:54.653967 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.682551 (+ 28584us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b1800078640. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:06:54.697919 (+ 15368us) raft_consensus.cc:1685] finished
0709 13:06:54.697944 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.698073 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40637,"prepare.queue_time_us":87645,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":5217,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":948394112,"thread_start_us":347194,"threads_started":1,"wal-append.queue_time_us":461209}
I20260709 13:06:54.732998 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:54.717718  3949 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:06:54.757298 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 3025) took 1987 ms. Trace:
W20260709 13:06:54.746493 29909 rpcz_store.cc:269] 0709 13:06:51.408007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:51.408125 (+   118us) service_pool.cc:224] Handling call
0709 13:06:53.285864 (+1877739us) spinlock_profiling.cc:243] Waited 8.02 ms on lock 0x7b54007a0920. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a8b06f 00007f7836b03f60 00007f7836b057de 00007f7836b1127c 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2
0709 13:06:54.687478 (+1401614us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b54007a0154. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:54.687682 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839522560}
W20260709 13:06:54.769299 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.736008  3968 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.766637 29902 rpcz_store.cc:276] 0709 13:06:53.393651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.393782 (+   131us) service_pool.cc:224] Handling call
0709 13:06:53.394202 (+   420us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:53.413200 (+ 18998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:53.413246 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:53.860425 (+447179us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b2c00012928. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:06:53.861263 (+   838us) log.cc:844] Serialized 52 byte log entry
0709 13:06:54.231897 (+370634us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:06:54.231967 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.232032 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.737792 (+505760us) raft_consensus.cc:1685] finished
0709 13:06:54.737823 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.757474 (+ 19651us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132172,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"spinlock_wait_cycles":39858944,"thread_start_us":370029,"threads_started":1,"wal-append.queue_time_us":442452}
I20260709 13:06:54.745512  3494 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.776360 29874 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:54.710780  3893 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:06:54.794142 29215 raft_consensus.cc:606] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:54.794988 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 56137.8us
W20260709 13:06:54.796741 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 138640us
I20260709 13:06:54.832659 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:54.832805 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.833906 29220 raft_consensus.cc:2410] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
W20260709 13:06:54.835338  3965 log.cc:927] Time spent T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:06:54.780977  3827 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:54.794715  3917 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:54.756911  3908 consensus_queue.cc:1175] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" responder_term: 16 status { last_received { term: 15 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:06:54.781306  3745 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:54.806339 29871 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:54.804548 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:54.782590  3803 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.807447  3499 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:54.850636  3879 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:54.808398  3875 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:06:54.803381 29071 rpcz_store.cc:276] 0709 13:06:52.769496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:52.769641 (+   145us) service_pool.cc:224] Handling call
0709 13:06:52.797074 (+ 27433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:52.884517 (+ 87443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:52.884551 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:52.885053 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:06:54.497323 (+1612270us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b18004a7640. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:06:54.713206 (+215883us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:54.746609 (+ 33403us) log.cc:844] Serialized 14 byte log entry
0709 13:06:54.746951 (+   342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.747058 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.747095 (+    37us) raft_consensus.cc:1685] finished
0709 13:06:54.757117 (+ 10022us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.757244 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120841,"prepare.queue_time_us":34179,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":93144,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":10237,"spinlock_wait_cycles":3225344,"thread_start_us":215705,"threads_started":1,"wal-append.queue_time_us":1825681}
I20260709 13:06:54.775703  3945 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:06:54.829587  3967 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
W20260709 13:06:54.853837 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:54.802930  3920 consensus_meta.cc:287] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.947s	user 0.014s	sys 0.000s
I20260709 13:06:54.807523  3771 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:06:54.736883  3907 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.844074  3904 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:54.876748  3850 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.834837  3877 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:54.797298 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 64439.4us
I20260709 13:06:54.884752  3926 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:54.866210 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.884414  3957 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.908488 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:54.877200 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 72709.3us
I20260709 13:06:54.869743 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:54.880115 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 73855.8us
I20260709 13:06:54.880632  3933 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:54.880843  3842 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:54.868528  3909 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.951s	user 0.011s	sys 0.000s
I20260709 13:06:54.829638  3876 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:54.781491  3595 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.880774  3776 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:06:54.793488  3772 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.869606  3970 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:06:54.876657  3952 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:54.875981  3917 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:54.844573  3862 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.875695  3745 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:54.831881  3866 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:06:54.881325  3940 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.881181  3780 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:54.883587  3499 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:54.864544 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 88251.7us
I20260709 13:06:54.887645  3945 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.888099  3875 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.915351 29139 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:54.914048 29220 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:54.861960  3494 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.883045  3836 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:06:54.869227  3707 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.928216 29073 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:54.930183 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:54.980803 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.930343 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:54.930929 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:54.865788  3968 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.910585  3926 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.922173  3906 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:06:54.920632  3893 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:54.951094 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:54.921733  3936 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.902076  3920 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.958576 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:54.939335  3810 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.933282  3827 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.910321  3850 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.936688  3904 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.974807 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 1081) took 1675 ms. Trace:
I20260709 13:06:54.944693  3917 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.995362  3917 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:54.987708  3707 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.956944  3745 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:54.945907  3814 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:54.945529  3252 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:54.940362 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:54.923487  3951 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 13:06:54.922047  3957 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.984583 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.020345 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:55.021100 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:54.925607  3842 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:54.974428  3499 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:54.925916  3909 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.966243  3952 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 18 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:54.999840 29139 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:54.994237 29142 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:06:55.022493  3803 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:54.991044  3772 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.006735  3968 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:55.008323  3810 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.009518  3494 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:55.010054 29141 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:55.006737 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:54.927467  3876 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.999454 29068 rpcz_store.cc:276] 0709 13:06:53.324728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.846324 (+521596us) service_pool.cc:224] Handling call
0709 13:06:53.846675 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:54.210579 (+363904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:54.210612 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:54.232603 (+ 21991us) log.cc:844] Serialized 26 byte log entry
0709 13:06:54.520346 (+287743us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:06:54.520383 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:54.963970 (+443587us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b54002d3a54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:54.964024 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:54.969439 (+  5415us) raft_consensus.cc:1685] finished
0709 13:06:54.969462 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:54.974720 (+  5258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":634,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":9681,"spinlock_wait_cycles":822417152,"thread_start_us":216211,"threads_started":1,"wal-append.queue_time_us":282077}
I20260709 13:06:55.014434  3814 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.015559 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 1143) took 1092 ms. Trace:
I20260709 13:06:55.015447  3252 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.015661  3771 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:06:55.017218  3180 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:55.021307 29909 raft_consensus.cc:2410] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:06:55.023212 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:54.991680  3933 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.986985 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.072329 29140 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:55.024566 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.056126  3983 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.023629 29220 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:54.977574  3875 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:06:54.990000  3955 raft_consensus.cc:993] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: : Instructing follower a936878f59c840a6bd176a1d2d7347b1 to start an election
I20260709 13:06:54.982566 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:54.986325  3926 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:54.924196  3877 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:54.977829  3945 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:55.112376  3877 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:55.115772  3909 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:55.116340 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:54.998297  3978 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:06:55.118575  3920 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:55.063145 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:55.063202 29147 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:55.121507  3252 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.121260  3954 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.064330 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.122327 29139 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
W20260709 13:06:55.122522  3980 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:06:55.123245 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.123798 29139 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:55.124681 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.067073  3803 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.065493 29907 rpcz_store.cc:276] 0709 13:06:53.923098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.036354 (+113256us) service_pool.cc:224] Handling call
0709 13:06:54.062859 (+ 26505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.081857 (+ 18998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:54.081892 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:54.081932 (+    40us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:06:54.081967 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.015314 (+933347us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b54007102d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:55.015364 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.015500 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43241344}
I20260709 13:06:55.069975  3771 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:54.998641  3707 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.078373 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:54.992410  3893 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.079838 29871 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:55.024019  3936 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.091882  3814 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.087814  3772 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.144059 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:55.146342 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:55.092288  3810 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.096216 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:55.097919  3813 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.099853  3773 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:55.151384 29902 raft_consensus.cc:2410] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:06:55.101531  3919 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.156150 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:55.102442 29144 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:55.102927  3984 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:55.103968  3884 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:55.103591  3745 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.104651  3856 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:06:55.108192  3842 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:55.144014  3911 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:55.110355 29936 heartbeater.cc:646] Failed to heartbeat to 127.27.71.190:37537 (6 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.27.71.190:37537 timed out after 15.000s (SENT)
I20260709 13:06:55.063732 29214 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:06:54.971827  3940 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.177340 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.023036  3827 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:54.994729  3646 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.000s
W20260709 13:06:55.076716  3596 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.024s	user 0.009s	sys 0.004s
I20260709 13:06:55.192178  3596 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.194518  3596 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.130865 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:55.133457 29139 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:55.196225 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:55.077090  3890 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.497s	user 0.011s	sys 0.004s
I20260709 13:06:55.068264  3957 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:55.023283  3904 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:55.140975 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 61278.2us
I20260709 13:06:55.137311  3803 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:55.073452 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 140021us
I20260709 13:06:55.150714 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:55.101290  3514 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:55.158634 29219 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:55.160833 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3143) took 1453 ms. Trace:
I20260709 13:06:55.163331 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:55.164367 29064 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:06:55.076282  3942 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.084040  3933 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.108580 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:55.126230 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.124217  3952 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.184798 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:55.186506 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.135818  3983 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.176924  3707 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.159333  3984 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:55.165208  3955 raft_consensus.cc:1081] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Signalling peer a936878f59c840a6bd176a1d2d7347b1 to start an election
I20260709 13:06:55.201645 29142 rpcz_store.cc:276] 0709 13:06:53.707164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.077465 (+370301us) service_pool.cc:224] Handling call
0709 13:06:54.077771 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.057263 (+979492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.160599 (+103336us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b5400190d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:55.160775 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2106389376}
I20260709 13:06:55.218046 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.218607 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.207563 29902 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:06:55.165580  3856 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:55.197731 29145 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:06:55.212927 29067 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:06:55.215569 29065 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:06:55.211282  3803 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.161832  3528 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:06:55.218679 29144 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:55.160434  3816 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 13:06:55.220048 29143 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:06:55.165798  3772 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.188616  3850 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.222090 29142 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:55.226996 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.231168 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:55.232942 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.249428  3745 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.250955  3893 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:55.251824  3989 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:55.252682  3989 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:55.254433  3968 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:55.255579  3968 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:55.255962  3968 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:55.256866  3968 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.257893  3968 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:55.258304  3983 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:55.261818 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.262337 29142 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:06:55.262712 29142 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:55.273272 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.221091  3957 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.222846  3596 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:55.263470 29142 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.274052 29067 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:55.211653  3514 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.305047 29142 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:06:55.308063 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.256891  3933 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:55.309475 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:55.260087  3986 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:55.217727  3707 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.247584  3847 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.198966  3771 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.312192 29037 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.206983  3890 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.246193  3926 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.318830 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.322430 29222 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:55.323029  3892 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.323206  3954 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:55.323545  3892 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.326325  3814 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:55.324131  3984 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.327477 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.327683  3772 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.330440  3942 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:55.331022  3707 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.332398  3904 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.333122  3514 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:55.337958  3936 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.338586  3936 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.339041  3936 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.315588  3595 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.330440  3940 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:55.342365  3926 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:55.344856 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 623723us
I20260709 13:06:55.345602 29112 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:55.346575 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.349754 29907 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:55.349802 29112 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:55.351341 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.315840  3986 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.318194  3893 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:55.375362 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.376483 29141 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:55.377404 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:55.378007 29143 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:55.359099 29142 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 13:06:55.359699  3852 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:55.363122 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3072) took 1050 ms. Trace:
I20260709 13:06:55.379895 29145 rpcz_store.cc:276] 0709 13:06:54.312730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.539333 (+226603us) service_pool.cc:224] Handling call
0709 13:06:54.539721 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:55.223033 (+683312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.362803 (+139770us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b54003b3dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:55.363040 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1512550784}
I20260709 13:06:55.381150 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.381781 29145 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:55.382592 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:55.308521  3957 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.379475 29142 pending_rounds.cc:85] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 12
I20260709 13:06:55.309432  3596 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.386646 29112 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:55.384446 29142 pending_rounds.cc:107] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 13:06:55.360220  3920 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:55.390561 29142 raft_consensus.cc:2889] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:55.357988  3595 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:55.357726  3904 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:55.399188 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.399894 29068 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:55.381477  3984 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:55.379956  3707 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:06:55.343142  3845 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:55.409802 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.360260  3986 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:55.408658 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.389508  3803 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:55.408456 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:55.329586  3856 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.410822  3926 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:06:55.404155  3852 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:55.411614  3957 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:55.327008  3945 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.424263 29143 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:55.425279 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.429129 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:55.430495 29903 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:55.430322  3917 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.430915 29908 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:06:55.431255 29908 pending_rounds.cc:85] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 12
I20260709 13:06:55.431457 29908 pending_rounds.cc:107] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:06:55.431697 29908 raft_consensus.cc:2889] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:55.432094 29139 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:06:55.384783  3499 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.437906 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:55.438382 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:55.437980 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:55.439419 29069 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 3064) took 2122 ms (client timeout 2685 ms). Trace:
I20260709 13:06:55.439939 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.438928 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:55.445647 29904 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:55.441974 29906 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
W20260709 13:06:55.444100 29069 rpcz_store.cc:269] 0709 13:06:53.298763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.496759 (+197996us) service_pool.cc:224] Handling call
0709 13:06:55.420549 (+1923790us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b5400186854. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:55.420652 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651217152}
I20260709 13:06:55.433504  3745 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:55.433003  3827 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.451910  3816 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:55.472929 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:55.453897 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.468420 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.468863 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.438091  3845 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.473703 29145 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
W20260709 13:06:55.452965  3931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.016s	sys 0.015s
I20260709 13:06:55.475077 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:55.474288 29143 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:55.470824 29069 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:06:55.476557  3940 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:55.493593 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.500571 29069 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:55.504631 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.510082 29069 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:06:55.513962 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:55.518536 29069 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:55.488153 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:55.531736 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.475857  3856 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:55.475031  3852 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:06:55.466046  3745 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.552285 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:55.548156 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.558620 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:55.473326  3816 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:55.463905  3803 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.108s
I20260709 13:06:55.684453 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.707841 29069 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:06:55.768316 29904 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:06:55.769585 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.703162 29143 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:55.757300 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.779814 29072 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 15.
I20260709 13:06:55.724493  3890 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.487257  3847 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:55.556730  4000 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:06:55.785567 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:55.744946  3884 consensus_meta.cc:287] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.002s
W20260709 13:06:55.760795  3875 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.000s
I20260709 13:06:55.781957  3920 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:55.547773  3952 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:55.551999  3999 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 13:06:55.781661  3931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.217s	user 0.016s	sys 0.015s
I20260709 13:06:55.548028  3996 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:55.678640  3995 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 13:06:55.643239  3917 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.748472  3945 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.748864  3596 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.796419 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.780766  3745 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:55.715873  3844 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:55.834942  3844 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.799067 29066 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
W20260709 13:06:55.806206  4002 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:06:55.799237 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.776420 29064 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:55.840864 29070 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:55.802805  3528 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:55.803619 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:55.581470 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 83682.9us
W20260709 13:06:55.776842 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 218281us
I20260709 13:06:55.745744 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 1262) took 2019 ms. Trace:
I20260709 13:06:55.798372  3920 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.801736  3816 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.845036 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:55.852908 29111 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:55.854257 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.856019 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.850612 29222 rpcz_store.cc:276] 0709 13:06:53.721071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.721353 (+   282us) service_pool.cc:224] Handling call
0709 13:06:55.729085 (+2007732us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54002841d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:55.739966 (+ 10881us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3028629248}
I20260709 13:06:55.865067 29909 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 22: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:06:55.869691  3968 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.001s
I20260709 13:06:55.869962  3929 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.870455  3909 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:55.871006  3909 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:55.871905  3909 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.876471 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:55.877748 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.878883 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.879396 29064 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:55.880034  3816 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:55.881274  3844 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:55.879397 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.880054  3528 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.799966  3875 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.884050  3817 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:55.884712  3999 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:06:55.885170 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.886642 29039 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:55.812453  3499 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:55.888975  3817 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:55.891155  3817 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.891422 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.888173  3528 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:55.893077 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.893666 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.894748 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.883236 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.904605 29072 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:55.898455 29188 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.898872  3817 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:55.864575  3985 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:55.887797  3919 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:55.864735  3920 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:06:55.901203 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:55.902179  3939 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:55.906548 29067 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:06:55.907195  3347 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:55.909139 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.918367 29070 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a"
dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:47974
I20260709 13:06:55.925077 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:55.925870 29067 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:06:55.926126 29072 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:06:55.926460 29070 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:55.927590 29070 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:55.928221  3347 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:55.928653  3347 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.929020  3347 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:55.929425 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:55.932349 29145 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:55.939173 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:55.912532  3861 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:55.911635  3842 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:06:55.908490  3917 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.940109 29141 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:06:55.951666 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.952023 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.952381 29072 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:55.917608  3929 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:55.957501 29113 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.958387 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:55.954792 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3256) took 1086 ms. Trace:
I20260709 13:06:55.928189  3939 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:55.923609  3817 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:55.960491 29218 rpcz_store.cc:276] 0709 13:06:54.868131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.130005 (+261874us) service_pool.cc:224] Handling call
0709 13:06:55.130279 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.318131 (+187852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:55.318176 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:55.318222 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:55.318270 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.954569 (+636299us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400339fd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:55.954618 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:55.954736 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431580928}
I20260709 13:06:55.959417 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:55.960763 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:55.972093 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 258645us
I20260709 13:06:55.977191 29067 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:55.961725 29145 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:55.912926  3745 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.923744  3595 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:06:55.986126  3347 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.988684 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:55.924362  3919 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:55.995141 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.990721 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.997076 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:55.997313 29067 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:55.964635  3904 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:55.962016  3845 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:55.965757  3985 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:56.012184 29905 raft_consensus.cc:2393] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:55.982569  3596 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:56.010102 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.009996 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1257) took 2397 ms. Trace:
I20260709 13:06:56.015069 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:55.985602  3861 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:55.985754  3814 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.013006 29145 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:56.004832 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:55.982796  3874 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.001430  3856 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:56.009788  3862 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:06:56.015964 29072 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:55.985468  3983 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:06:56.015429 29146 rpcz_store.cc:276] 0709 13:06:53.601285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.601451 (+   166us) service_pool.cc:224] Handling call
0709 13:06:53.601799 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:54.755534 (+1153735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.998683 (+1243149us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54001bc6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:55.998854 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2543621888}
I20260709 13:06:55.956656  3917 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.000092  3499 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:55.992720  3844 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:06:56.013616  3347 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.010664  3904 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:56.017757  3596 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.019450  3861 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.027418  3874 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.036176 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.027215  3856 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.028079  3985 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:56.024834  3875 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.038825 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:56.043527 29067 raft_consensus.cc:1275] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:06:56.036993  3499 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.047016  3917 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.037014  3904 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.047878 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.048038  3984 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:56.040359  3861 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:56.022866  3814 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.042790  3856 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:56.052255 29141 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 13:06:56.040777  3933 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:56.037268  3596 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:56.051695  3985 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.036725  3842 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:06:56.058609  3917 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.015103 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:56.026851  3939 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.071506 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:56.072109  3917 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.060824  3983 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:06:56.081192 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 45443.8us
I20260709 13:06:56.068993  3985 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:56.093137 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:56.094482  3347 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:56.095216  3347 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.097702 29072 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.097049  3745 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:56.087544 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 75521.7us
I20260709 13:06:56.078897 29070 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.042313  3817 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.118623 29070 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.080220  3983 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:56.152544  3347 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.106343 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:56.122869 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:56.093104  3940 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:56.067962  3933 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.126657 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:56.108156 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:56.126459 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 212294us
I20260709 13:06:56.159870 29902 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:56.161636 29039 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:56.159986 29903 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.158198  3347 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:06:56.158403 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52141.7us
I20260709 13:06:56.163703  3347 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.158079  3842 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:56.172952  3939 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:56.194168 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.236006  3347 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.241926 29070 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:56.182045  3968 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:56.244041  3347 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:56.184525  3919 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.187732  3957 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.242862  3983 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:56.234859  4010 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:06:56.188199  4017 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:56.245385  3814 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.247195  3347 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.247079  3968 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.239502  3842 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.239634  4014 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:56.251639 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 90118.8us
I20260709 13:06:56.303282 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:56.305188 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.258116 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3249) took 1664 ms. Trace:
I20260709 13:06:56.307142 29219 rpcz_store.cc:276] 0709 13:06:54.593410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.593644 (+   234us) service_pool.cc:224] Handling call
0709 13:06:54.628861 (+ 35217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.201370 (+572509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.257841 (+1056471us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54003a93d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:56.258047 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217888512}
I20260709 13:06:56.269577 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.303648  4014 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.271823  4017 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:56.312562  4015 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:06:56.273180  3936 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.274545  3347 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:56.275391  3968 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:06:56.275609  3992 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.801s	user 0.000s	sys 0.001s
W20260709 13:06:56.277340  4001 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.848s	user 0.002s	sys 0.000s
I20260709 13:06:56.278247  3842 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.323446 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3158) took 1225 ms. Trace:
I20260709 13:06:56.323853 29144 rpcz_store.cc:276] 0709 13:06:55.098113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.225340 (+127227us) service_pool.cc:224] Handling call
0709 13:06:55.298050 (+ 72710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:55.298506 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:55.298545 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:55.299030 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:06:55.417104 (+118074us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:55.417146 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.417219 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:56.276780 (+859561us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:56.323110 (+ 46330us) raft_consensus.cc:1685] finished
0709 13:06:56.323132 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.323377 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":531,"raft.queue_time_us":775,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":5984,"spinlock_wait_cycles":11125120,"thread_start_us":88317,"threads_started":1,"wal-append.queue_time_us":113114}
I20260709 13:06:56.326275 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.284672 29072 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:56.285132 29903 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
W20260709 13:06:56.331710 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.262418  4018 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:06:56.311682  4014 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.327706 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.339071 29144 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:56.253765  4009 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:06:56.252979  3827 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:56.262347  3892 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.263358  3844 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.323868  3814 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:56.353096 29072 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:56.312742  4017 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.260921  3933 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.342144 29902 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:06:56.348152 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:56.261662  3917 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.265292  3983 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:56.264155  3919 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:56.257131  3874 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.253942  3499 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.264438  3957 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.363454 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:56.366888 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:56.367148 29902 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:56.344544  3842 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.358527 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:56.368434 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:56.369012 29144 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:56.248275  3942 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:56.367720 29903 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 24: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:56.345256  3828 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:06:56.371698 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.372727 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:56.356353  3814 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:56.377710 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.377681 29072 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:56.377717 29219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:06:56.379407 29219 raft_consensus.cc:606] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:56.379791 29219 raft_consensus.cc:614] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:56.377067 29039 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:56.386701 29069 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:06:56.387351  3968 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.347416  3940 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:56.350843  4014 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:06:56.398100 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:56.399165 29903 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:06:56.401804 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.356838  3844 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.391656  3814 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:56.394353  3917 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:56.369395  3919 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.419862 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:56.380077  3942 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.432128  3940 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.399065  3904 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.435041  3904 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.375036  3842 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.419545  3968 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:56.422919  3985 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.444419  3920 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.446988  3920 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.420773  3917 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.447997  3917 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:56.452483 29140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 1260) took 2751 ms (client timeout 1703 ms). Trace:
W20260709 13:06:56.453011 29140 rpcz_store.cc:269] 0709 13:06:53.700569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:53.700977 (+   408us) service_pool.cc:224] Handling call
0709 13:06:56.452184 (+2751207us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002e8bd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:56.452362 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2704566656}
I20260709 13:06:56.454427 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.455333 29140 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:56.404016  4024 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:56.456269  4024 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:56.457298  4024 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.461611 29143 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:56.462812 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 1273) took 1049 ms. Trace:
I20260709 13:06:56.467094  3803 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:56.467881  3803 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.469391 29071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 1095) took 1538 ms. Trace:
I20260709 13:06:56.378789  3596 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.463244 29143 rpcz_store.cc:276] 0709 13:06:55.413882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.780959 (+367077us) service_pool.cc:224] Handling call
0709 13:06:56.462541 (+681582us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b54003467d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:56.462715 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1443437312}
I20260709 13:06:56.481483  3911 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:56.481927  3911 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:56.482127 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.482738 29143 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:56.484696 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:56.485554 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 1202) took 1046 ms. Trace:
I20260709 13:06:56.485940 29908 rpcz_store.cc:276] 0709 13:06:55.439663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.439786 (+   123us) service_pool.cc:224] Handling call
0709 13:06:55.453493 (+ 13707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.453702 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 13:06:55.823499 (+369797us) log.cc:844] Serialized 14 byte log entry
0709 13:06:55.991809 (+168310us) log.cc:844] Serialized 14 byte log entry
0709 13:06:56.071644 (+ 79835us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b180037eee0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:06:56.485070 (+413426us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b180037eee0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:06:56.485225 (+   155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.485270 (+    45us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:06:56.485310 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.485375 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.485498 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":473191,"prepare.queue_time_us":446275,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":89,"raft.queue_time_us":494828,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":11612,"spinlock_wait_cycles":29529472,"thread_start_us":536305,"threads_started":2,"wal-append.queue_time_us":245992}
I20260709 13:06:56.488502 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:56.489880  3890 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:56.490296  3890 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:56.492689  3824 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:56.493096  3824 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.493446  3824 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:56.491158  3890 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.498728  3890 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.482749  3911 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.501463  3911 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:56.502202 29145 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:56.503899 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:06:56.507624  3982 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 1.438s	user 0.004s	sys 0.004s
I20260709 13:06:56.510649  3846 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:56.511075  3846 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.511435  3846 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:56.513319  3894 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:56.513535 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.513875  3894 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.496994  3922 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.516177  3922 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.517220 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.518064 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3083) took 1252 ms. Trace:
I20260709 13:06:56.518523 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.519873 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.523562  3863 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:56.528560  3863 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.535607 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.536517 29140 raft_consensus.cc:2393] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 24: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:56.518558 29073 rpcz_store.cc:276] 0709 13:06:55.265397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.899998 (+634601us) service_pool.cc:224] Handling call
0709 13:06:55.905737 (+  5739us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:55.909978 (+  4241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:55.910009 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:55.910544 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:06:56.095539 (+184995us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:56.095727 (+   188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.096211 (+   484us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:56.328337 (+232126us) raft_consensus.cc:1685] finished
0709 13:06:56.328358 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.517730 (+189372us) spinlock_profiling.cc:243] Waited 5.39 ms on lock 0x7b5400321ed0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:56.517984 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12881,"raft.queue_time_us":6854,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":11006,"spinlock_wait_cycles":12393344,"thread_start_us":150478,"threads_started":1,"wal-append.queue_time_us":230672}
I20260709 13:06:56.538877 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:56.483814 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:56.548858 29143 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.525261 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 1100) took 1303 ms. Trace:
W20260709 13:06:56.560458 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 167612us
I20260709 13:06:56.573292 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:56.573621 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 80289.4us
I20260709 13:06:56.574113 29215 raft_consensus.cc:2368] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21.
I20260709 13:06:56.575766 29144 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:06:56.469774 29071 rpcz_store.cc:276] 0709 13:06:54.930521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.016402 (+ 85881us) service_pool.cc:224] Handling call
0709 13:06:55.016809 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:55.017186 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:55.017221 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:55.017761 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 13:06:55.139767 (+122006us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:06:55.150836 (+ 11069us) log.cc:844] Serialized 14 byte log entry
0709 13:06:55.162248 (+ 11412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:55.162337 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:56.467159 (+1304822us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b1800168ee0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:06:56.469085 (+  1926us) raft_consensus.cc:1685] finished
0709 13:06:56.469196 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.469333 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":832,"prepare.queue_time_us":11708,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":22452,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":36575,"spinlock_wait_cycles":30269184,"thread_start_us":116595,"threads_started":1,"wal-append.queue_time_us":167801}
I20260709 13:06:56.582417 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:56.584049 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 221523us
I20260709 13:06:56.584903 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:56.585487 29216 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:56.587415 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.581310 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:56.591027 29220 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:06:56.580610 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:56.561180 29064 rpcz_store.cc:276] 0709 13:06:55.221671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.899423 (+677752us) service_pool.cc:224] Handling call
0709 13:06:55.899861 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:55.900107 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:06:56.135836 (+235729us) log.cc:844] Serialized 14 byte log entry
0709 13:06:56.524572 (+388736us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b1800433180. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a
0709 13:06:56.524841 (+   269us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:56.524905 (+    64us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:06:56.524965 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.525047 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.525191 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20556,"prepare.queue_time_us":287206,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":1331,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":86,"spinlock_wait_cycles":3687808,"thread_start_us":370442,"threads_started":2,"wal-append.queue_time_us":222161}
I20260709 13:06:56.598222 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:56.596149 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:56.603376 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.606411 29214 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:56.600294  3347 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:56.607273 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.607856 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:56.608410 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.607795  3347 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:56.617594 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.607934 29218 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:06:56.621793 29143 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:56.617561  3957 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.601768  3984 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:56.625025  3984 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:56.625398  3894 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:56.625979  3984 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.626546  3803 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.423684  3861 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.628367  3844 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:56.623258 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:56.630050  3861 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:56.630674  3861 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:56.580204  3905 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.631747  3905 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:56.633658 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.633962  3905 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.635054  3842 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.636243 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.636646  3919 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.592070  3869 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 13:06:56.631162  3861 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:56.639207 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.640069  3347 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.527896  3596 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.637972 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:56.591182  3920 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.595289  3904 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.641812 29222 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:06:56.595432  3180 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:56.649606 29141 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:06:56.643139 29186 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:56.618011  3862 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:56.661837  3862 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:56.662060  3876 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.662328  3862 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:56.599290  3916 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:06:56.664465  3916 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:56.664180  3846 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.664813  3916 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:56.665719  3916 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.666846  3916 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.669245  3917 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:56.656054 29139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 3165) took 1460 ms (client timeout 1874 ms). Trace:
W20260709 13:06:56.670603 29139 rpcz_store.cc:269] 0709 13:06:55.188343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.195971 (+  7628us) service_pool.cc:224] Handling call
0709 13:06:56.649001 (+1453030us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54003cc3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:56.649116 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541031296}
W20260709 13:06:56.671362  3856 consensus_meta.cc:287] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.601s	user 0.014s	sys 0.008s
I20260709 13:06:56.671833 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.671949  3856 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.673115 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.673733 29139 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:06:56.674528 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.674868  3894 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.675516  3894 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.678068  3894 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:56.678823  3917 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.680048  3846 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:56.675519 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.681151 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.652984 29216 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:56.650301  3957 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:56.682802 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:56.683504 29067 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.684257 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.685603 29219 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:56.659334  3842 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:56.686364  3942 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:56.660672  3814 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.661378  3985 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:56.669803  3827 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:56.670140 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.681377 29139 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:56.659942  3180 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.655885  3920 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.654511 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.688464 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:56.716084 29142 raft_consensus.cc:2393] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:56.716681 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:56.655843 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:56.648645  3863 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.697359 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.735816 29143 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:56.709800 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:56.736508 29139 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:56.652601  3803 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:06:56.710851 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:56.718030 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 62285.3us
I20260709 13:06:56.734920 29064 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:06:56.744808 29067 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:56.688995  3856 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:56.749423 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.753364 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:56.756752 29144 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 15.
I20260709 13:06:56.758134 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:56.757594 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46869.8us
I20260709 13:06:56.759341 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.760464 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.687768  3904 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.735430  4032 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:06:56.736292  3869 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:06:56.755486 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.772132  4032 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:56.765132 29066 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 1091) took 1835 ms (client timeout 2183 ms). Trace:
W20260709 13:06:56.788913 29066 rpcz_store.cc:269] 0709 13:06:54.929281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:54.930707 (+  1426us) service_pool.cc:224] Handling call
0709 13:06:56.764928 (+1834221us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b5400186d54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:56.765066 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1878824832}
I20260709 13:06:56.740751  3919 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:56.762001  3803 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:56.780781 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.796171 29142 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:06:56.797709 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:56.798537 29217 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.759899 29067 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:06:56.782713 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:56.803290 29220 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:56.764971  3846 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:56.776172 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:56.812964 29147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 3081) took 1805 ms (client timeout 1845 ms). Trace:
W20260709 13:06:56.813542 29147 rpcz_store.cc:269] 0709 13:06:55.008075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.020790 (+ 12715us) service_pool.cc:224] Handling call
0709 13:06:56.812731 (+1791941us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b54001bc1d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:56.812870 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95197440}
I20260709 13:06:56.781243  3869 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:56.791239  4032 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:06:56.793526  3919 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.821959 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.822849 29142 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:06:56.825446 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:56.826150 29147 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:56.826790 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.815104  3846 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.762081  3861 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.836826 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.838061 29142 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:06:56.821305  3869 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:56.842813 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:56.824216  3919 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:56.854141 29220 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:06:56.865715 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3081) took 1650 ms. Trace:
I20260709 13:06:56.866223 29068 rpcz_store.cc:276] 0709 13:06:55.215287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.402266 (+186979us) service_pool.cc:224] Handling call
0709 13:06:55.402542 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:55.403291 (+   749us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:55.403318 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:55.408840 (+  5522us) log.cc:844] Serialized 26 byte log entry
0709 13:06:56.856603 (+1447763us) spinlock_profiling.cc:243] Waited 44.2 ms on lock 0x7b540024a520. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a77090 00007f7836a846b8 00007f7836b0d92b 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c
0709 13:06:56.856692 (+    89us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:56.856755 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.856842 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:56.856893 (+    51us) raft_consensus.cc:1685] finished
0709 13:06:56.856905 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:56.865618 (+  8713us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":445,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":4185,"spinlock_wait_cycles":101642496,"thread_start_us":371182,"threads_started":1,"wal-append.queue_time_us":383977}
I20260709 13:06:56.833803  3846 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:56.821444  3986 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:06:56.874727 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:56.876948 29146 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:56.878149 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.887648 29148 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
W20260709 13:06:56.888176 29907 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 1192) took 1751 ms (client timeout 1658 ms). Trace:
W20260709 13:06:56.888804 29907 rpcz_store.cc:269] 0709 13:06:55.136894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.137061 (+   167us) service_pool.cc:224] Handling call
0709 13:06:56.887931 (+1750870us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54009033d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:56.888095 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338254080}
I20260709 13:06:56.889405 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:56.890209 29148 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:56.892402 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:56.893569 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:56.894240 29222 raft_consensus.cc:2368] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
I20260709 13:06:56.895002 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.888248  3814 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.896932  3817 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:06:56.897344  3983 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.531s	user 0.014s	sys 0.007s
I20260709 13:06:56.904613 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.902235 29222 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:56.904587 29215 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:06:56.906642 29216 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 23. Ops: []
I20260709 13:06:56.906342 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:56.908743 29142 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:56.905752 29146 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:06:56.897433  3817 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:56.911492  3180 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.912099  3817 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:56.913426  3817 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:56.914000  3824 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.921339 29911 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 23: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:56.925549 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.921015  3985 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:56.938898 29139 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
W20260709 13:06:56.931190 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.931667 29147 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:56.932348 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:56.875247 29217 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:06:56.948210 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:56.949951 29185 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:56.955691 29185 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:56.965910 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.967806 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:56.972507 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.975241 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:56.964249 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 127813us
I20260709 13:06:56.986116 29147 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:06:56.987622  3876 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:56.988044 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:56.988440  3876 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:06:56.989034 29213 raft_consensus.cc:2410] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:06:56.980820  3847 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:06:56.989768 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:56.990554 29903 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:56.992329 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:56.994987 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.995676 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:56.996353 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.995785 29068 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:06:57.004455 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:57.026656 29907 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:56.939092  3842 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:57.003432 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:56.968986  3984 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:57.006968 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:57.007089 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:06:57.008143  4049 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:06:57.040634 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:57.042284 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:57.007956 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:57.043290 29065 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:57.044332 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.046775  3845 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:57.047626  3845 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:57.049366 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:56.946126  3850 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:57.051054 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3092) took 1550 ms. Trace:
I20260709 13:06:57.050951  3850 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.051479 29141 rpcz_store.cc:276] 0709 13:06:55.501093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.062606 (+561513us) service_pool.cc:224] Handling call
0709 13:06:56.649218 (+586612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.653605 (+  4387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.050806 (+397201us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b54003cc3d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:57.050986 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1318281472}
I20260709 13:06:57.055423 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 1197) took 1637 ms. Trace:
I20260709 13:06:57.055788 29905 rpcz_store.cc:276] 0709 13:06:55.417505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:55.428874 (+ 11369us) service_pool.cc:224] Handling call
0709 13:06:57.055194 (+1626320us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54004e61d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:57.055349 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337362688}
I20260709 13:06:56.957675  3983 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:57.056905  4034 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:06:56.992105  3957 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:56.987401  3894 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:06:57.064216  3920 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:57.016538 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:57.016830 29140 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:57.019425 29875 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:57.011073  3922 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:56.957922  3985 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:56.966310  3942 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:57.021868  3847 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:57.027547 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:57.031539 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.032600 29066 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 23: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:56.868867  3803 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.040050 29073 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:57.011646  3996 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:06:57.067905 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48560us
I20260709 13:06:57.020344  3745 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:06:57.068670 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 363348us
I20260709 13:06:57.073413 29065 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:57.073361  3849 raft_consensus.cc:2804] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:57.075335  3942 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:57.081696  3985 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:57.025966  3814 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:06:57.087708 29067 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 20: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:57.091789 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:57.139319 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:57.140015 29065 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:57.140528 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:57.123724 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:57.142445 29066 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:57.143035 29905 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 22: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:57.144184 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:57.085271  3847 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:57.094018 29221 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:06:57.157790 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:57.131870  3922 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:57.015620  3869 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.123775 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:57.132125  3942 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:57.129235 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:57.132611 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.133960 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:57.170682 29071 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
W20260709 13:06:57.098093 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 241962us
I20260709 13:06:57.167232 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:57.184128  3985 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:57.184602  4047 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.179s	user 0.001s	sys 0.001s
I20260709 13:06:57.184715  3985 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.187238  3985 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:57.192348  4040 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:57.192675  3845 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:57.193624  3869 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:57.199368 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:57.201599 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:57.202450 29218 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:57.146699  3849 raft_consensus.cc:697] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 18 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:57.129830  4032 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.223896  4032 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:57.224504 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:57.205410 29066 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:06:57.206904 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:57.228408 29904 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:57.229682 29112 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:57.230029 29140 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:57.207793 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.231366 29215 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:06:57.210434 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:57.211521 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:57.212592  3824 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:57.169229  3922 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:06:57.214704  3916 consensus_meta.cc:287] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.000s
I20260709 13:06:57.144234  3862 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.240667 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:57.241510 29213 raft_consensus.cc:2393] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 22: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:06:57.241962 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:57.242735 29147 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:57.244199 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:57.216740 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:57.235021 29906 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:57.213330 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:06:57.236387 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 252018us
I20260709 13:06:57.246840 29911 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:57.237298 29215 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:06:57.249369 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:57.207379  3869 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.227928  3745 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:57.263289 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.266566 29185 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:57.245450  3845 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:57.269759 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.270419 29213 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:57.278512 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.253707  3911 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 13:06:57.287074 29215 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:06:57.287595 29215 pending_rounds.cc:85] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 11
I20260709 13:06:57.288311 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:57.292225 29904 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:06:57.292466 29038 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:57.287860 29215 pending_rounds.cc:107] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:06:57.293907 29215 raft_consensus.cc:2889] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:57.292980 29185 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:57.304809 29073 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:06:57.307544 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:57.309192 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:57.256608  3869 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.314595 29222 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:57.310021 29073 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:06:57.267679  3985 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:06:57.261674  3745 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.296075  3957 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:57.295814  3916 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:57.295678  3983 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:57.247743  3847 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.323220 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:57.252154  3849 consensus_queue.cc:237] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.348044 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:57.270985  3989 consensus_queue.cc:1048] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 13:06:57.349910 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.359597 29222 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:57.355747 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:57.362244 29910 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:06:57.359632 29217 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:57.364049 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.349521  3945 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:57.356644  3940 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.377696 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:57.383862 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.390913 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.362411  4032 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:06:57.393548 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 92622.6us
W20260709 13:06:57.396087 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 142666us
I20260709 13:06:57.396507 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.398075 29871 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:57.376766  3945 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:57.377444  3827 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:57.413210 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:57.426818 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:57.314961  3922 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.380947  3916 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:06:57.397632  4032 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:57.420163  3817 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.506s	user 0.002s	sys 0.010s
I20260709 13:06:57.423063 29214 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:57.434037 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:57.437772 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:57.442617  4057 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:06:57.444597 29065 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:06:57.446270  3916 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:57.446710  3916 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:57.447518  3916 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:57.448244  4048 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:06:57.448836  3916 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:57.448962  4064 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:57.449415  4064 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:57.449781  4064 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:57.435104  4032 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:57.452924  3869 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:57.454196  3745 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:57.395277  3940 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.457417  3940 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:57.457919 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3098) took 1291 ms. Trace:
I20260709 13:06:57.458014  3940 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:57.458614  3940 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:57.459556  3940 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:57.424410  3904 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:57.470835  4017 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:06:57.471300  4017 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:57.471773  4017 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:06:57.470263 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:57.428275  3814 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:57.477672 29217 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:06:57.413877  4024 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:06:57.480870 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.481324 29039 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:57.481757 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:57.488554 29065 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:57.482419 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:57.482550 29068 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:57.458436 29148 rpcz_store.cc:276] 0709 13:06:56.166599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.901904 (+735305us) service_pool.cc:224] Handling call
0709 13:06:56.930235 (+ 28331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:56.957386 (+ 27151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.957419 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:56.983935 (+ 26516us) log.cc:844] Serialized 26 byte log entry
0709 13:06:57.271973 (+288038us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:57.272049 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.272152 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:57.457706 (+185554us) raft_consensus.cc:1685] finished
0709 13:06:57.457727 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.457863 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10397,"raft.queue_time_us":10600,"spinlock_wait_cycles":26343296,"thread_start_us":257554,"threads_started":1,"wal-append.queue_time_us":420882}
I20260709 13:06:57.492882 29218 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
W20260709 13:06:57.486011 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51801.7us
W20260709 13:06:57.395463 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 137044us
I20260709 13:06:57.486503  3817 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:57.497931 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:57.498297  3817 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.498796  3817 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:57.487277 29188 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:57.501266 29067 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:06:57.504992 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:57.505353 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:57.505017 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:57.514226 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:57.516134 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:57.518594 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:57.519276 29141 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:57.515177 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:57.520840 29140 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:57.476657  3922 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:57.524418 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:57.522925 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:57.525419 29139 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:57.499891  4033 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.583s	user 0.001s	sys 0.001s
I20260709 13:06:57.524003 29188 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:57.504619  3846 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.596s	user 0.005s	sys 0.006s
W20260709 13:06:57.477072  3816 consensus_queue.cc:1175] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a936878f59c840a6bd176a1d2d7347b1" responder_term: 17 status { last_received { term: 17 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 17 index: 12 } }
I20260709 13:06:57.526015 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:57.532990 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:57.535409 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:57.536171 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:57.538913 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:57.539196 29143 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:06:57.541347 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:57.542202 29142 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:57.543790 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:57.547216 29139 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:06:57.548398 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:57.549113 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:57.565878 29145 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:06:57.520273  3863 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:57.552644 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 149984us
I20260709 13:06:57.534658  3869 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:57.533736  3904 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:57.511427  3942 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.520432  3814 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:57.568444 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:57.540045  3983 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:57.540212  3985 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:57.520712  3945 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.571854 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:57.593284 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 156861us
I20260709 13:06:57.548697  3845 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:57.565939 29144 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:57.503402  4024 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.571795 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:57.599478 29065 raft_consensus.cc:2468] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:06:57.540709  3846 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.551180  3920 raft_consensus.cc:2764] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:06:57.575672  4054 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:06:57.553344  3596 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:57.595147 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:57.597908 29073 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:57.579492  3919 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.705s	user 0.012s	sys 0.000s
I20260709 13:06:57.574563  3869 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:06:57.621793 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 46524.7us
I20260709 13:06:57.623229 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3101) took 1259 ms. Trace:
I20260709 13:06:57.623782 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:57.623648 29070 rpcz_store.cc:276] 0709 13:06:56.364750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.364904 (+   154us) service_pool.cc:224] Handling call
0709 13:06:56.365231 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:56.376265 (+ 11034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:56.376301 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:56.376796 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:06:56.719937 (+343141us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:56.719994 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:56.720087 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:57.499302 (+779215us) spinlock_profiling.cc:243] Waited 8.17 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:06:57.612581 (+113279us) raft_consensus.cc:1685] finished
0709 13:06:57.612620 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:57.623150 (+ 10530us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10655,"raft.queue_time_us":26376,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4790,"spinlock_wait_cycles":18795904,"thread_start_us":285030,"threads_started":1,"wal-append.queue_time_us":433307}
I20260709 13:06:57.625479 29213 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:57.596462  3845 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:57.582800  3942 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:57.610586 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 92986.4us
I20260709 13:06:57.598702  4073 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:57.576622  3863 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.632198 29874 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:57.621140  4072 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:57.627041  3814 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:57.662478  3814 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.668038  3846 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:57.668475 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.669251 29214 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:57.670109 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.670222  4017 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.671172 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.672353 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:57.672969 29214 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:57.648384 29218 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:06:57.648272 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:57.676836 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:57.629721  3845 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:57.625831  3869 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:57.621780  3919 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.652007  4024 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:57.632148  4073 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.653117 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:57.687668 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 56125.9us
I20260709 13:06:57.655077  4032 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.656270 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:57.657828  3940 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.675206  3957 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:57.760676 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 136957us
I20260709 13:06:57.678385 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:57.762457 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 84138.6us
I20260709 13:06:57.677238 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.680140 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:57.654461 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.688633 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:57.763836 29222 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:57.763968 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:57.764971 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.699373  4077 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:57.701092  3863 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:57.701455  4032 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:57.702497 29038 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:57.841446 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 139025us
I20260709 13:06:57.704097  3849 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:57.660223  3596 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:57.685815  3985 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:57.684337  3983 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:57.708931 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.715684 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:57.847160 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 158626us
I20260709 13:06:57.848140 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:57.744040  4024 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:57.761730 29072 raft_consensus.cc:2368] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:57.758540  4017 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:57.835772 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.708310  4073 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:57.687731  4075 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.149s
I20260709 13:06:57.834810  3861 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:57.841874  4077 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:57.853153 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 173076us
I20260709 13:06:57.866326  3596 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.868425  3596 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.846407  3985 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.849303  3827 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:57.840508  4080 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:57.879691  3940 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:57.867133  3861 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.881757  3861 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:57.882550  3919 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:57.884111  3596 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:57.862694  3780 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:57.885073  3780 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:57.886358  3780 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.897614 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:57.903775 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 52099.6us
I20260709 13:06:57.906059 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:57.861039  3863 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:06:57.896862  4072 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:57.913275  4072 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.916796  4072 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:57.919694 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.924371 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 3262) took 1103 ms. Trace:
I20260709 13:06:57.924955 29221 rpcz_store.cc:276] 0709 13:06:56.820898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.829805 (+  8907us) service_pool.cc:224] Handling call
0709 13:06:57.923970 (+1094165us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54002f62d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:57.924270 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491441536}
I20260709 13:06:57.926586 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.915583  3861 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:57.936056  4071 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 13:06:57.680303 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 84552us
I20260709 13:06:57.938670  3596 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:57.939052 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:57.939954  3596 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:57.940366  3596 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:57.941236  3596 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.942807  3596 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:57.847397  3957 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:06:57.871693  3922 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:57.945128  3922 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:57.946290  3922 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.862910  3983 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:57.949103  3861 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:57.949061  3983 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.861176  4032 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:06:57.863364  4024 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.958568  4024 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:57.963138  3916 consensus_meta.cc:287] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
I20260709 13:06:57.890635  3985 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:57.970279  3827 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:57.890472  4080 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:57.971462  4080 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:57.975704  3869 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:57.976645  3817 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:57.978683  3817 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:57.871411  4077 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:57.987011 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 48058.8us
W20260709 13:06:57.995206 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.995271 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1271) took 1288 ms. Trace:
W20260709 13:06:57.996166 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:57.995818 29216 rpcz_store.cc:276] 0709 13:06:56.706881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:56.707457 (+   576us) service_pool.cc:224] Handling call
0709 13:06:56.708048 (+   591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:56.908399 (+200351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:57.995008 (+1086609us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b5400132ed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:57.995212 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456275712}
I20260709 13:06:57.997165 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.997620  3738 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:06:57.997807 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.998062  3738 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:57.998644 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:57.864915  4076 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:06:57.999150  3738 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.000730  3738 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:06:58.002434  4039 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.881s	user 0.001s	sys 0.009s
W20260709 13:06:58.004611 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.006107 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:58.006285 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:58.008253 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.009861 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:58.009975 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 104007us
I20260709 13:06:58.015460 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:57.978683  3869 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:58.018607  3869 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:06:58.019125  3869 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:58.025924 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:58.028872 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.028779 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 431179us
I20260709 13:06:58.030092 29037 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:58.033521 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:58.013656 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 247985us
I20260709 13:06:58.016150  3817 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:58.021111  4064 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.004s
W20260709 13:06:58.042902 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.043048 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:58.049330 29068 raft_consensus.cc:3060] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.053821 29186 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.059751 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:58.061048 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:58.081413 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:58.082973 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:58.086563 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 238524us
I20260709 13:06:58.088130 29874 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.042724  3827 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:58.056470  4024 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:58.080952  4032 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.110415 29068 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:06:58.084098  4072 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:58.090258  4017 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:58.084668  3919 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:58.116801 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:06:58.112569 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 79236.2us
I20260709 13:06:58.109055 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:58.096258  3814 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:58.092809  3957 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:06:58.124713 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:58.125365 29065 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:58.115296 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:58.072832  4064 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.133867  3845 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.120525  3847 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:58.146623 29071 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:06:58.122195 29113 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:58.136477 29065 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:58.116555  4084 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:58.116955  4017 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:58.149513 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:58.151451  4090 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:58.155227 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 114301us
I20260709 13:06:58.130436  4086 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:06:58.167098 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.142787  3814 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:58.154258  4017 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:58.182806 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:58.178720 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.187089 29071 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:06:58.176579 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 88554.6us
I20260709 13:06:58.184242 29064 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:06:58.189806 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:06:58.198463 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 85346.5us
I20260709 13:06:58.168285  4086 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:58.203016 29874 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:58.203353 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 81252.1us
I20260709 13:06:58.171854  4090 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:58.163661  4085 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:58.207026 29065 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:58.190099 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:58.208742 29185 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:58.184352  3957 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:58.226119 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:58.246793 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60289.2us
I20260709 13:06:58.254482 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:58.261435 29071 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
W20260709 13:06:58.262866 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 60015.4us
I20260709 13:06:58.265367 29068 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 24. Current term is 25. Ops: []
I20260709 13:06:58.206682  3989 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:58.251827  3596 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:06:58.277896  4099 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:58.278681  4099 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:58.279156  4099 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.279608  3869 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.281426  3869 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:06:58.201924  3849 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.062s
W20260709 13:06:58.274454  4089 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:06:58.274850  3919 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:58.277945  3890 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:06:58.280073  4092 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:06:58.313997 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59605.5us
I20260709 13:06:58.212375  4090 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:58.208400  4086 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:58.248260  3846 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:06:58.278172  3596 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:58.257160  3776 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:58.256915  3842 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:58.263083  4094 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:58.285772  3989 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.332748  4094 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.334347  3989 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:58.300908  4064 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.253561  3957 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.339062  3957 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:58.272332  3940 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:06:58.269734  4095 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:58.273962  3814 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.341286  3845 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.304450  4080 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.344229  4080 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.306492  4066 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:58.345062  3850 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:58.345021  4066 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.347348  4066 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:58.310635 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:58.323100 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:58.349820  3940 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.350373  3940 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.348188  4094 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:58.353644 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.354282 29139 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.354598 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:58.321794  3919 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.357699 29142 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:06:58.358014  3940 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:58.358793 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.359380 29143 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:06:58.361688  4099 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.363659 29147 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:06:58.364663  4073 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.365770 29069 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:58.366214  4086 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.321012  4090 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:58.367300 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:58.330513  3776 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:58.381536  3776 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.332049  3842 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:58.384232 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 158239us
I20260709 13:06:58.336246  3596 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:58.366530 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:58.387007 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:58.327481  4092 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:06:58.375383 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.380342 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:58.387905 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.386015 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.390185 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:58.393826 29113 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.387521 29905 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:06:58.395298 29141 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:58.343453  3985 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.400498 29188 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.405320 29144 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:58.405747 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:58.404296 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:58.407297 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.407615 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.407930 29148 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:06:58.405974 29219 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:06:58.408942 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.409017 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.409907 29143 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:06:58.411950 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:58.411129 29139 raft_consensus.cc:2410] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:06:58.358050  3919 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:58.413923 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:58.416136 29141 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 25: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:58.380005  4099 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.422598 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.424496 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.425503 29145 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 17: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:58.386572  4084 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:58.395998  4092 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:58.422966 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:58.339661  4077 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.424885 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:58.397936  3845 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:58.429394 29214 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 24: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:58.459759 29188 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:06:58.461527 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.386511  4091 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:06:58.414513  3890 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:58.420547  3983 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.426479  4088 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:58.428180  3989 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:58.427322  3916 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:06:58.472301  3842 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.472898  3890 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:58.441028  4104 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:58.473767  4084 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:58.464919  3919 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:58.473414  4054 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:58.439407  3846 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 13:06:58.474409  4088 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.490612  4088 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:58.465553  3986 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:58.492463  3845 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.475322  4103 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.493062  3845 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.493271  4103 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.476233  4092 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.495651  3890 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:58.431079  4087 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:58.470707 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.497641 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:58.490144  4077 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.439613  3817 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:06:58.490409 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.488127 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:58.506979 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:58.483218 29148 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
W20260709 13:06:58.503877 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.501819 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:58.464759  3957 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:58.512621 29070 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 24: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:58.477864  4032 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:58.512065 29141 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:58.549386 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 51813.7us
I20260709 13:06:58.548861 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:58.552044 29070 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:58.541951 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.514583 29143 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:58.428180  4079 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:06:58.477361  4099 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 13:06:58.486748  4084 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.485179  4017 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.475422  3596 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.477795  4073 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.560274 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.557834 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:58.556735 29147 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 13:06:58.543470  4077 raft_consensus.cc:2764] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:06:58.557802  4112 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:58.488828  4054 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.584796  3845 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.585479  3845 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:58.479385  3842 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.586097  3845 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.588344  3845 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:58.590451 29070 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:58.591030  3957 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.591503 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:58.592279 29069 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:58.543015  4092 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:58.592677 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:58.594061 29070 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:06:58.594576  4017 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:58.505599  3817 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:58.476663  3985 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:58.504530  4104 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:58.480908  4086 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.541169  4085 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:58.512254  3986 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:58.577643  3846 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:58.599126 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:58.542028  3850 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:58.610088 29070 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:06:58.610224 29139 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:06:58.559567  4103 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:58.585574  4054 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:58.495025  3983 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.624624 29146 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:58.621775 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.620523 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.615126 29145 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:06:58.604080  4108 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.618145 29143 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:06:58.620249 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.621973 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.591605  3957 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.626125 29141 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:58.613355  3817 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:06:58.632349 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 117642us
I20260709 13:06:58.627854 29145 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
W20260709 13:06:58.628218 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.630086 29147 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:58.630879 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:58.634439 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:58.634927 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.635008 29148 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:06:58.629531 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:58.636124 29143 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:06:58.620129  3844 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.211s	user 0.021s	sys 0.005s
I20260709 13:06:58.640609 29113 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:06:58.648976 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.620167  3846 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.624161  3849 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:58.629522  4017 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:58.659108  3844 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.259s	user 0.021s	sys 0.005s
I20260709 13:06:58.615685  4090 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.675549 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:58.662706  3803 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:58.681505 29141 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:06:58.694808 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:06:58.683745 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.695540 29147 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:06:58.662429  3846 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:58.662487  4103 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:06:58.628445  3850 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.701578 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:58.703488 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:58.706527 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:58.707862 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:58.708228 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:58.710182 29140 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:58.709873 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.713588 29070 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:58.716089  3814 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:06:58.720105  3985 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:58.711613  4073 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:06:58.720928  4110 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:06:58.720981  4073 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.721715 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 3119) took 1067 ms. Trace:
I20260709 13:06:58.722129 29219 rpcz_store.cc:276] 0709 13:06:57.653815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.656010 (+  2195us) service_pool.cc:224] Handling call
0709 13:06:58.721536 (+1065526us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b540033a4d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:58.721653 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1560367360}
I20260709 13:06:58.723201 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:58.723392  4073 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.724604 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:58.725257 29071 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:58.725442 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 1170) took 1509 ms. Trace:
I20260709 13:06:58.725791 29910 rpcz_store.cc:276] 0709 13:06:57.216284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.216476 (+   192us) service_pool.cc:224] Handling call
0709 13:06:58.725293 (+1508817us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54004b50d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:58.725388 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267108608}
I20260709 13:06:58.712973 29039 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:58.700304  4115 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.713816 29147 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:58.672197  3849 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.632604  4108 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.727396 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:58.708397  3827 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:58.729839  4088 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.675233  4017 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.744796  4090 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.738647  3850 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.714658  3803 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:06:58.737723  4109 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:06:58.770367 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:58.773895 29140 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:06:58.708947  4079 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:06:58.729838  4032 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:06:58.741446  3945 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:58.705241  4085 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.776405 29188 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.787616 29905 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:06:58.790815 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:58.791541 29073 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
W20260709 13:06:58.782327  4116 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:06:58.774153  3942 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:58.782672 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:58.806702 29219 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:06:58.778471  3894 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:06:58.795782 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:58.742662  3890 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.809553 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:58.812899 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:58.781077 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 53769.1us
I20260709 13:06:58.793875 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:58.816171 29073 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:06:58.818508 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:58.819382  4082 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:58.819895  4082 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:58.820798  4082 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.821372 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:58.821993 29073 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:58.823045 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:58.824118  3816 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:06:58.824126 29113 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.131:45851 timed out after 1.795s (SENT) [suppressed 16 similar messages]
I20260709 13:06:58.824704  3816 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.825090  3816 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:58.751467  4115 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.837113  3894 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:58.837540  3894 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:58.838080  4115 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.838402  3894 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:06:58.784693  4113 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:06:58.714936  3986 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.773684  4054 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:06:58.783232  3983 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:06:58.783973  4111 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:06:58.853837 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:58.785104  3827 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:58.857918 29139 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:58.849462 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:58.855475 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.845172 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:58.803241  3844 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:58.780880  3985 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 13:06:58.866675 29072 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47960 (request call id 3064) took 1839 ms (client timeout 1517 ms). Trace:
I20260709 13:06:58.841172  3803 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:58.866961 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:58.782399  3957 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:58.840889  4088 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:58.813802  3989 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:58.868815 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:58.900158 29220 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:06:58.848587  3842 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:06:58.901172 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:58.795403  4032 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:58.835358  3816 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:58.905036 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:58.885594 29072 rpcz_store.cc:269] 0709 13:06:57.027498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.042006 (+ 14508us) service_pool.cc:224] Handling call
0709 13:06:58.857233 (+1815227us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54001d8b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:58.866592 (+  9359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1652438272}
I20260709 13:06:58.810943  3942 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.841732  3849 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.797603  3776 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 13:06:58.900379  4019 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:06:58.914688 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 69610.1us
I20260709 13:06:58.920019 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 3278) took 1642 ms. Trace:
I20260709 13:06:58.896113 29139 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:58.920413 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.920414 29213 rpcz_store.cc:276] 0709 13:06:57.277968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.278187 (+   219us) service_pool.cc:224] Handling call
0709 13:06:58.919781 (+1641594us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54003cd7d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:58.919950 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":794086784}
I20260709 13:06:58.916817 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:58.902895  3989 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:58.924403 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:58.925194 29213 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:58.911363  3957 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:58.926630 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:58.927484 29220 raft_consensus.cc:2410] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:06:58.898360  3876 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:06:58.911809  3842 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:06:58.910048 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 49399.8us
I20260709 13:06:58.930032 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:58.944351 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:58.903446  4087 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:06:58.925877  3989 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:58.953938 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:58.959172 29142 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24. Ops: []
I20260709 13:06:58.962237 29186 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:58.927762  4019 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:58.929194  3957 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.964980 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:58.968007 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:58.968909 29142 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:06:58.969370 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:58.960022 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:58.972465 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:58.971014 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:06:58.975461 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.977258 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:58.956348  3985 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:06:58.971395 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:58.990854  4063 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.993489  4076 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:58.995330  4105 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:06:58.996217  4105 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:58.996702  4105 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:58.997829  4105 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:58.960518  3876 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:06:58.967110  3957 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:58.981756 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:58.970465 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:06:59.016927 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 46527.9us
I20260709 13:06:59.019384 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:58.973026  4017 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:58.980737  4054 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:59.002753  4085 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:59.027696 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:59.003065  4082 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:59.029556 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:59.006557  3876 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:06:59.033325 29072 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:59.028723 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 108406us
I20260709 13:06:59.034663 29220 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:59.023792  3863 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:59.026752  3850 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:59.040752 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:58.991654  4063 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.014689  4127 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:59.029420  4054 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:59.040481  3863 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.046111 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:58.979166  4032 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.074039  3745 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.049571 29072 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
W20260709 13:06:59.050328 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.050974 29220 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:06:59.032704  4085 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:06:59.006376  3842 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:59.055154 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 348177us
W20260709 13:06:59.056195  4124 log.cc:927] Time spent T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:06:59.060101  4054 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:59.042678  4104 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:59.062165  4122 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:06:59.024412  3817 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.074613  3745 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.051534  3876 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.056183  3846 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.066437  3863 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:59.069172 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:59.118933 29215 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:59.071286 29213 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:06:59.072104  3942 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:59.084506 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 95812.1us
I20260709 13:06:59.086037 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:59.049741  4127 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.123709 29220 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:59.040814  4019 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.117549  3945 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:59.109886 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.113751 29217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 3287) took 1828 ms (client timeout 1795 ms). Trace:
I20260709 13:06:59.076587  3816 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:59.091408  4085 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:59.136343  3989 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.137697 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 3124) took 1482 ms. Trace:
I20260709 13:06:59.138129 29214 rpcz_store.cc:276] 0709 13:06:57.655214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.673411 (+ 18197us) service_pool.cc:224] Handling call
0709 13:06:59.137474 (+1464063us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b54001329d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:59.137627 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1531045120}
I20260709 13:06:59.138628  3989 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:59.119792  4104 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.139459 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:59.120362 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:59.174513 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:59.126957 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.131515 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:06:59.131750 29217 rpcz_store.cc:269] 0709 13:06:57.285127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.364902 (+ 79775us) service_pool.cc:224] Handling call
0709 13:06:59.113491 (+1748589us) spinlock_profiling.cc:243] Waited 84.5 ms on lock 0x7b540007a3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:59.113650 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194297088}
I20260709 13:06:59.100833 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:59.148347 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3123) took 1722 ms. Trace:
I20260709 13:06:59.205804 29213 raft_consensus.cc:2368] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:59.207680 29216 raft_consensus.cc:2393] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:59.183332 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:59.189141 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:59.190330 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:59.208170 29066 rpcz_store.cc:276] 0709 13:06:57.426076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.426201 (+   125us) service_pool.cc:224] Handling call
0709 13:06:57.426517 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:57.613087 (+186570us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:57.629810 (+ 16723us) log.cc:844] Serialized 14 byte log entry
0709 13:06:58.009234 (+379424us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:58.009327 (+    93us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:58.009389 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.148119 (+1138730us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400321ed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:06:59.148173 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.148291 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3527,"prepare.queue_time_us":16874,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":6533,"raft.queue_time_us":4988,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":2591,"spinlock_wait_cycles":428350720,"thread_start_us":375839,"threads_started":1,"wal-append.queue_time_us":250176,"wal-append.run_cpu_time_us":1195,"wal-append.run_wall_time_us":1008292}
I20260709 13:06:59.197317 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:59.208786 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:59.120468  4128 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:06:59.219834 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.178433 29214 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:59.221381 29113 leader_election.cc:400] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
I20260709 13:06:59.221777 29113 leader_election.cc:403] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:59.191285 29215 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:06:59.212144 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 3289) took 1491 ms. Trace:
I20260709 13:06:59.222790 29216 rpcz_store.cc:276] 0709 13:06:57.720659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:57.999992 (+279333us) service_pool.cc:224] Handling call
0709 13:06:59.174307 (+1174315us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8e9c 00007f78339e8a5b 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:06:59.212083 (+ 37776us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13245824}
I20260709 13:06:59.217813 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:59.126967  3876 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:06:59.206346 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:59.125046  4127 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:59.225144 29220 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:59.228173 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1281) took 1546 ms. Trace:
I20260709 13:06:59.229827 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:59.230742 29216 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:59.231601 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3133) took 1551 ms. Trace:
I20260709 13:06:59.232031 29216 rpcz_store.cc:276] 0709 13:06:57.680162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.224541 (+1544379us) service_pool.cc:224] Handling call
0709 13:06:59.230494 (+  5953us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.231305 (+   811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.231533 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.228585 29220 rpcz_store.cc:276] 0709 13:06:57.681787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.224693 (+1542906us) service_pool.cc:224] Handling call
0709 13:06:59.224968 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.225565 (+   597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.228127 (+  2562us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:59.235057 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 134317us
I20260709 13:06:59.236266 29111 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:06:59.168879  3817 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:59.239552 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 231969us
I20260709 13:06:59.238175 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:59.164888  3846 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:59.252808 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:59.141765  4104 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:59.218339  3827 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:59.252373 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:59.252439 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:59.259651 29902 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:06:59.243763 29220 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:59.263068 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3141) took 1217 ms. Trace:
I20260709 13:06:59.247869 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:59.263473 29220 rpcz_store.cc:276] 0709 13:06:58.045662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.243258 (+1197596us) service_pool.cc:224] Handling call
0709 13:06:59.243598 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.262795 (+ 19197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.263002 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.265512 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3143) took 1219 ms. Trace:
I20260709 13:06:59.267853 29220 rpcz_store.cc:276] 0709 13:06:58.046362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.264416 (+1218054us) service_pool.cc:224] Handling call
0709 13:06:59.264749 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.264945 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:59.264974 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:59.265229 (+   255us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:06:59.265274 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.265335 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.265450 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.254984 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:59.213874  3803 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:59.143098  4085 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.256938 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:59.276261  4085 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:59.280305  3817 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.281195  3817 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.283984 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:59.284646 29222 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:59.256217  4134 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.285904  4134 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.252350  4099 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:59.243124  4079 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:59.148236  4063 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:59.242280 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:59.266144 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:59.187253  4054 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.290784 29905 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:06:59.267482 29073 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:59.243489  4117 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:59.254611  4032 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.181710  3862 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:59.265427 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:06:59.256341 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:59.255672  3850 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:59.272799 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:06:59.240063  4074 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:06:59.267176 29903 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 13:06:59.296531 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 54363.4us
I20260709 13:06:59.312060 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:59.313943 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:59.185319  4108 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:59.298015 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:59.282549  3863 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:59.303282 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.233232  3945 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.271377  4104 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:59.330593 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:59.304756  4077 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:06:59.300522 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:59.335527 29073 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 17: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:06:59.322239 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:59.319530  4117 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:59.324884 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:59.295526  4079 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.326534 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:59.327627 29139 raft_consensus.cc:1275] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:06:59.250054 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:59.341714 29220 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:06:59.342376 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3144) took 1295 ms. Trace:
I20260709 13:06:59.342710 29220 rpcz_store.cc:276] 0709 13:06:58.046654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.341142 (+1294488us) service_pool.cc:224] Handling call
0709 13:06:59.341473 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:59.342146 (+   673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.342326 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.328500 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:59.331212 29141 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:59.348842 29906 raft_consensus.cc:1240] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24. Ops: []
I20260709 13:06:59.349325  3771 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:59.329969 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:59.320353 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:59.351274 29908 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:59.343678 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:59.357285 29220 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 24: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:06:59.357429 29141 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:06:59.356158 29222 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:06:59.334535 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3132) took 1653 ms. Trace:
I20260709 13:06:59.358996 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:59.376129 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.392725 29904 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:06:59.395469 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:59.359853 29146 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: []
I20260709 13:06:59.397346 29218 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:59.397872 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1286) took 1132 ms. Trace:
I20260709 13:06:59.316627  3803 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:59.398515 29220 rpcz_store.cc:276] 0709 13:06:58.265837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.397142 (+1131305us) service_pool.cc:224] Handling call
0709 13:06:59.397388 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.397570 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:59.397598 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:59.397638 (+    40us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:06:59.397671 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.397721 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.397823 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.403187 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:59.365334 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:59.399824 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:59.404258 29148 raft_consensus.cc:2410] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:06:59.397953 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1285) took 1132 ms. Trace:
I20260709 13:06:59.404820 29218 rpcz_store.cc:276] 0709 13:06:58.265520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.396908 (+1131388us) service_pool.cc:224] Handling call
0709 13:06:59.397186 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.397763 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.397906 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.394685 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 3138) took 1552 ms. Trace:
I20260709 13:06:59.322124  3850 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:59.332980  4085 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:59.405979 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:59.441330 29145 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:06:59.407236 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:59.406174 29222 rpcz_store.cc:276] 0709 13:06:57.842711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.003580 (+160869us) service_pool.cc:224] Handling call
0709 13:06:59.283813 (+1280233us) spinlock_profiling.cc:243] Waited 6.07 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:06:59.394624 (+110811us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13961984}
I20260709 13:06:59.442520 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:59.443099 29222 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
W20260709 13:06:59.443025 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 113132us
I20260709 13:06:59.407683 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:59.444269 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:59.310112  4032 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.360595  4079 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:59.410557 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:59.412271 29072 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:06:59.414034 29065 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:06:59.415804 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:59.423295 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:59.453886 29070 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:06:59.454528  4099 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17.
W20260709 13:06:59.394284 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55937us
I20260709 13:06:59.330750  4104 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.460224  4104 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:59.435111 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:06:59.437795 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 78871.3us
I20260709 13:06:59.444828 29073 raft_consensus.cc:2368] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24.
I20260709 13:06:59.409868 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:59.444962 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:06:59.446717 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:59.468870 29066 raft_consensus.cc:2368] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:59.468335 29222 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:06:59.399251  3904 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:06:59.401618  4109 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:06:59.454092 29148 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 25: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:06:59.333935  3996 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.331399  3817 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:59.473448 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:59.425556  4066 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:06:59.401870  4077 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:59.449051  4133 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:59.465761 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:06:59.484354 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:59.487879 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.476385 29218 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:59.501044 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1289) took 1004 ms. Trace:
I20260709 13:06:59.501519 29218 rpcz_store.cc:276] 0709 13:06:58.496719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.437616 (+940897us) service_pool.cc:224] Handling call
0709 13:06:59.476125 (+ 38509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.500687 (+ 24562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.500969 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:59.479198  3347 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:59.449620  3803 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:59.480924  4137 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:59.481221  3862 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:59.481011  4091 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:06:59.481123  3919 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:06:59.481869  3904 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.474602  3929 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:06:59.520840  3803 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.521775  4032 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:59.524798  4142 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:06:59.525055  4054 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:59.517822  4133 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:06:59.561031  4133 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.562795 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:59.563728 29142 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:06:59.525743  4141 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:06:59.497366  4066 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.480544  3776 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 13:06:59.546959 29144 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:59.547201 29145 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:06:59.566103 29145 pending_rounds.cc:85] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 10
I20260709 13:06:59.566354 29145 pending_rounds.cc:107] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:06:59.566599 29145 raft_consensus.cc:2889] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:59.566788  4066 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:59.514809 29218 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:59.549525 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:59.569113 29148 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:06:59.550163 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:59.511730  3817 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:59.535775  3957 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.506s	user 0.004s	sys 0.006s
I20260709 13:06:59.528736  4134 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:59.555787 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:06:59.575088 29218 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:06:59.546164  3904 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:59.489591  3996 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.579907 29903 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:06:59.581349 29218 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:06:59.549458  3919 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:59.588672 29218 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:06:59.552742  3844 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.593341 29218 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:06:59.591509 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3121) took 1937 ms. Trace:
I20260709 13:06:59.595929 29217 rpcz_store.cc:276] 0709 13:06:57.654418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.219164 (+1564746us) service_pool.cc:224] Handling call
0709 13:06:59.254728 (+ 35564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.254969 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:06:59.268191 (+ 13222us) log.cc:844] Serialized 14 byte log entry
0709 13:06:59.591035 (+322844us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:59.591111 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:59.591167 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.591256 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.591426 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13065,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":128,"thread_start_us":322322,"threads_started":1}
W20260709 13:06:59.608929  4144 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:06:59.610301  4145 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:06:59.606479  3844 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.551576  3850 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.577616  4146 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:59.617426  3844 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:59.640792  3984 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.632192  3863 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:06:59.611868  4157 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:06:59.658998  3984 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.637072  3904 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.636787  3945 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:59.589386  3957 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.666656  4079 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.619292  3996 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:59.641629  3919 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.637823  4146 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:59.672569  4146 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.673568  4146 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:59.673476  4104 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.677151  4104 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:59.718786 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:06:59.726469 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.728825 29139 raft_consensus.cc:2368] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21.
I20260709 13:06:59.732332 29141 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:59.733011 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:59.734449 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:06:59.738847 29221 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:06:59.741072 29214 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:59.654352  3827 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:06:59.742012 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.667771  3904 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:59.754040 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:06:59.761757  3827 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:59.741590 29910 consensus_meta.cc:287] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.003s
I20260709 13:06:59.763618 29910 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:06:59.766238 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:59.773530  3929 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:06:59.774670  3929 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.776719  4032 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.788775  4032 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.794875  4079 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:59.783718  3929 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:59.796233  4073 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:06:59.810786  4163 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:59.812778  3929 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:06:59.813592  3929 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:06:59.824298  3957 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:59.827351  3890 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:06:59.810561 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44614us
I20260709 13:06:59.829244  3996 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:59.830431  3996 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.807309  4073 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:59.832654  4073 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:06:59.833925  3894 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:59.834290 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:59.815465  3929 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:06:59.846946  3996 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:59.856559 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:06:59.860169  3890 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:06:59.866148  4032 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:59.864291  3890 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:06:59.868752  4127 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.624s	user 0.016s	sys 0.004s
I20260709 13:06:59.869364  4127 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.872072 29139 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:06:59.874665 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:06:59.874850  4104 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:59.875538  3863 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:59.875766 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:06:59.880048  3904 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.880961 29064 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:59.880102  3890 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:06:59.889731  4128 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:59.907992 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:59.904563 29111 leader_election.cc:400] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24.
I20260709 13:06:59.910249 29111 leader_election.cc:403] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:06:59.901225  3863 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:06:59.908713 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 155211us
I20260709 13:06:59.911378  3863 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:06:59.913259  3863 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.904810  4104 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:59.916128  3863 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:59.908820  4128 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:59.902263  3904 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.920692  4104 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24.
I20260709 13:06:59.923755 29111 leader_election.cc:400] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:59.932611 29111 leader_election.cc:403] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:06:59.933604  4104 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:06:59.940994 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:59.945796 29219 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:06:59.949728 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:59.950677 29218 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:59.952284 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:06:59.926400  4158 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.399s	user 0.003s	sys 0.000s
I20260709 13:06:59.899206  4127 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:59.953133  3904 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:06:59.971527 29064 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:06:59.973837 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:06:59.976672  3817 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:59.981438  4077 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:59.982026 29221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 3290) took 2261 ms (client timeout 2684 ms). Trace:
W20260709 13:06:59.982578 29221 rpcz_store.cc:269] 0709 13:06:57.720959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.016899 (+295940us) service_pool.cc:224] Handling call
0709 13:06:59.981727 (+1964828us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54003557d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:59.981920 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3865951104}
I20260709 13:06:59.984105 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:06:59.984943 29221 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:59.984120 29039 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:06:59.985814  4128 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:06:59.987514 29213 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:06:59.988402 29217 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:06:59.988816  4128 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:06:59.989225 29213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3127) took 2333 ms. Trace:
I20260709 13:06:59.982044 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:06:59.989763 29213 rpcz_store.cc:276] 0709 13:06:57.674936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.233588 (+1558652us) service_pool.cc:224] Handling call
0709 13:06:59.986834 (+753246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.987941 (+  1107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.988928 (+   987us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b54003557d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:06:59.989129 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1722274048}
I20260709 13:06:59.988397  3929 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:00.008234 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:00.008790 29071 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:00.009856 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:00.010763 29216 raft_consensus.cc:2376] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21. Re-sending same reply.
I20260709 13:07:00.011123 29113 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:00.000489  3929 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.011977  3929 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:00.015599 29218 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
W20260709 13:07:00.022953  4154 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.498s	user 0.000s	sys 0.002s
I20260709 13:07:00.026551 29221 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:07:00.048849 29216 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:00.036363  4088 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:00.050398  4077 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:00.050972  4088 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:00.052717  4088 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.053231  4073 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.054388  4088 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:00.054576 29067 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:06:59.365260 29215 rpcz_store.cc:276] 0709 13:06:57.679851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.222997 (+1543146us) service_pool.cc:224] Handling call
0709 13:06:59.223321 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.223581 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:06:59.245303 (+ 21722us) log.cc:844] Serialized 14 byte log entry
0709 13:06:59.333293 (+ 87990us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:59.333341 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:59.333388 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:59.333465 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:59.333636 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1076,"prepare.queue_time_us":30675,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":1846,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":55572,"thread_start_us":86263,"threads_started":1}
I20260709 13:07:00.058215  3890 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:00.060098  3890 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:00.060637  3890 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:00.061939  4105 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:00.062655  3890 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.049014 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:00.064944 29218 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:00.067175  4151 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:00.068159  4151 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:00.069912  4151 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.070868 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:00.072039 29140 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:00.075129  4153 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:07:00.077579  4153 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:07:00.079459 29186 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:00.084079  3846 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:00.084614  3846 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:00.085052  3846 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:00.085616 29219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 3145) took 2038 ms (client timeout 2168 ms). Trace:
W20260709 13:07:00.086107 29219 rpcz_store.cc:269] 0709 13:06:58.046939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:58.866663 (+819724us) service_pool.cc:224] Handling call
0709 13:07:00.085305 (+1218642us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003cdcd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:00.085523 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2423270656}
I20260709 13:07:00.088717  3745 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:00.090713 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3129) took 2415 ms. Trace:
I20260709 13:07:00.091607  3814 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:07:00.091187 29214 rpcz_store.cc:276] 0709 13:06:57.675698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.221805 (+1546107us) service_pool.cc:224] Handling call
0709 13:06:59.222305 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.756960 (+534655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.090434 (+333474us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54002f5dd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:00.090655 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1190782720}
I20260709 13:07:00.095208 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:00.096755 29217 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:00.096863 29039 leader_election.cc:400] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21.
I20260709 13:07:00.088441  3894 consensus_queue.cc:1059] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:07:00.099619 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 79154.9us
I20260709 13:07:00.100934  3894 raft_consensus.cc:3055] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:00.101467  3894 raft_consensus.cc:740] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:00.102821  3984 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:00.102970  3847 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.103942  3847 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.105731  4090 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:00.106218  4090 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:00.106613  3847 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:00.096252  4077 consensus_queue.cc:1243] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:07:00.112972 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:00.114722 29218 raft_consensus.cc:2393] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 23: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:00.118758  4063 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:00.119266  4063 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:00.107704  4090 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:00.125313 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.126902  3904 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:00.114162  3929 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.129165 29217 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:07:00.127602 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:00.128221  3827 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:00.129392  4079 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:00.130918 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:00.130357 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:00.127455  3904 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:00.154209  3904 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.156329 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:00.157064 29216 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:00.142028 29146 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:00.142567 29147 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 13:07:00.143627 29142 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:00.135269 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:00.144572  4079 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:00.130373  4063 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.147221  3894 consensus_queue.cc:260] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.152426 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:00.143793  3827 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:00.158869 29147 pending_rounds.cc:85] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 14
I20260709 13:07:00.155731  3904 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:00.188102 29215 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:00.188930 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:00.190717 29144 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:07:00.192199 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:00.192397  3929 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:00.192863 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:00.174628  3827 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:00.161041  4079 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:00.194986 29219 raft_consensus.cc:2410] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 25 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:07:00.194752  3827 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.175015 29147 pending_rounds.cc:107] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:07:00.192471  3894 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:00.197881 29147 raft_consensus.cc:2889] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:00.203672 29139 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:00.204116  3996 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:00.204774  3996 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.207748 29141 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:00.203750 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:00.220072 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:00.217332 29213 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:00.228811 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:00.246443 29141 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:00.248390  3916 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 13:07:00.200873  4104 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 13:07:00.224584  4063 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:00.259222 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:00.230687  3996 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:00.239626  3827 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:00.273697 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:00.195003  4079 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.312386 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:00.311116 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 76717.3us
I20260709 13:07:00.276086  3890 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:00.321880 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:07:00.325729 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.326699 29906 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:07:00.330178 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 196991us
I20260709 13:07:00.330878 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:00.332432 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:00.332893  4167 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:00.333449  4017 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:07:00.342542  3916 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:00.344364 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3305) took 1539 ms. Trace:
I20260709 13:07:00.344934 29222 rpcz_store.cc:276] 0709 13:06:58.750362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.470056 (+719694us) service_pool.cc:224] Handling call
0709 13:06:59.470297 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:59.470472 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:06:59.485907 (+ 15435us) log.cc:844] Serialized 14 byte log entry
0709 13:06:59.935124 (+449217us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b18003b37a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a 00007f7836b17a2a
0709 13:07:00.213394 (+278270us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:00.229948 (+ 16554us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:00.256039 (+ 26091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.270404 (+ 14365us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:00.289541 (+ 19137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15349,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":11489,"raft.queue_time_us":3087,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"spinlock_wait_cycles":8862080,"thread_start_us":199922,"threads_started":1}
I20260709 13:07:00.346593 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:07:00.347298 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.348987  4073 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:00.349692  3844 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.347337 29911 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:00.350359  3844 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.356289  3844 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:00.351676 29910 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:07:00.361501  4104 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.362159  4104 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.363828 29038 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:00.365154  4062 consensus_queue.cc:1048] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:07:00.366302 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.366518  4167 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:00.366779  4082 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:00.367466  4062 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:00.366367  3916 raft_consensus.cc:2764] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:07:00.369855  4115 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:00.370249 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:00.370711  4115 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:00.371567  3745 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:07:00.372576  4082 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.372254  4115 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.373108  4082 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.375176  4082 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:00.367798  3814 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:00.376211  3814 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:00.376737 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:00.377187 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:00.377256  3814 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:00.385241 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.378464 29215 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:00.294173  3863 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:00.401377  3894 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.379172 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:00.394101 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:00.403322 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:00.430338 29903 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:00.432816 29141 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
W20260709 13:07:00.428006 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 50828.2us
I20260709 13:07:00.429819 29071 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
W20260709 13:07:00.428438 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 198148us
I20260709 13:07:00.377600  4017 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:07:00.440812 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.394593  4079 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:00.445745  3894 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.457823  4104 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:00.460147  4062 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:00.461220 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:00.461956  4062 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:00.463565  3904 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:00.464315  4076 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:00.464506  3745 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:00.464512  3904 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.466094  4076 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:07:00.466490  4076 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:00.466979  4068 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:00.467378  4068 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:00.468276  4068 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.469705  4068 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:00.446944 29911 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:00.447198  4095 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:00.396747  3850 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:00.482421 29906 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:00.491575  3940 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.491671 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:00.492403  3940 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.494833  3940 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:00.484886 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:00.481616 29146 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:00.507303 29222 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:00.480302  3894 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:00.481266  4095 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.384649 28962 catalog_manager.cc:5697] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:00.514942 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:00.485476  3850 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:00.480444  3904 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:00.518126 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:00.530385 29067 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:07:00.518648 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:00.538911 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:00.539839  3940 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.540822  3940 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.545943  3940 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:00.467610  4076 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.522837  4095 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:00.546445  3894 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.540786 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:00.549747 29071 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:00.526795  4062 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:07:00.575342  3814 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:00.575002  4032 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:00.576354  4079 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:00.508721  4105 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:07:00.583424  3929 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:00.581653  3844 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:00.590025  3849 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:00.591536  4105 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:00.592082  4105 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:00.593930  4105 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.596825  4105 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:00.595546 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62519.4us
W20260709 13:07:00.601668 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.601598  4133 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:00.607415 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:00.613097 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:00.618482 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:00.623286 29908 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 24: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:00.618059  3929 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.627139  3929 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:00.632701 29220 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:00.635057 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1288) took 2151 ms. Trace:
I20260709 13:07:00.636729 29067 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:00.616389  3745 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:00.635414 29220 rpcz_store.cc:276] 0709 13:06:58.483583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.403305 (+919722us) service_pool.cc:224] Handling call
0709 13:06:59.410265 (+  6960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:00.633319 (+1223054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:00.634725 (+  1406us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003552d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:00.634956 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2797135232}
I20260709 13:07:00.655339 29213 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
W20260709 13:07:00.640192 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.022s	sys 0.025s
W20260709 13:07:00.660264 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.022s	sys 0.025s
W20260709 13:07:00.660773 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 586365us
I20260709 13:07:00.661752 29113 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:00.663060 29219 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:07:00.663257 29068 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:07:00.663906 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.667553 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 152735us
I20260709 13:07:00.625521  4079 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:00.668123 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 157031us
I20260709 13:07:00.666004  3849 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.620820  3996 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
W20260709 13:07:00.737291 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.625847  3894 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.729235 29903 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:00.747694 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:00.740517 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:00.656843  3850 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.769351  3850 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:00.771835 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:00.772006  3863 consensus_queue.cc:1175] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a936878f59c840a6bd176a1d2d7347b1" responder_term: 24 status { last_received { term: 21 index: 16 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: [23.17-23.17]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:00.774427  3894 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:00.775012  3940 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:00.775642  3940 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.759469  3849 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.777851  3940 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:00.736944  3890 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:00.812763 29067 raft_consensus.cc:2410] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:07:00.820235 29071 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:00.817137 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:00.818526  3817 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:00.801069  3890 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:00.831664  3916 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:00.831935  3817 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.834649  3817 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:00.836845 29904 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:07:00.837244 29904 pending_rounds.cc:85] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 12
W20260709 13:07:00.837267 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.832427  3916 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:00.837519 29904 pending_rounds.cc:107] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 18.13
W20260709 13:07:00.835834 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.842417 29904 raft_consensus.cc:2889] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:00.802340  3940 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:00.789028  4151 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:07:00.819677  4032 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:00.827705  4076 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:00.855046 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:00.830955  3846 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.745s	user 0.010s	sys 0.005s
I20260709 13:07:00.819375  4073 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:00.867847 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:00.838428  3890 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:00.820734  3894 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
W20260709 13:07:00.877851 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 216176us
I20260709 13:07:00.884055 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:00.900995 29068 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:00.840579  4079 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.909039 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:00.872315  4104 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:00.878443  4166 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:07:00.917944  4179 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:07:00.886518  3940 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.918293  3904 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:00.919118  4082 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:00.921681  3849 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:00.922955  4068 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:00.923661  3850 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:00.925022 29113 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:00.957612  4115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.002s	sys 0.071s
I20260709 13:07:00.927513 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:00.927999 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:00.870067  4032 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.930902 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:07:00.938371 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.943048 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:00.943574 28962 catalog_manager.cc:5697] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 18, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 18 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:00.948652  4183 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:07:00.949611  4088 consensus_meta.cc:287] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.895s	user 0.009s	sys 0.006s
I20260709 13:07:00.950147 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:00.950642  4090 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:00.900336  4073 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.892972  4167 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:00.982513 29214 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:07:00.959815 29906 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:07:00.898013  4017 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:00.984184 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:00.984541 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:00.984303  4166 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.985412 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:00.988611  3803 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.030s	sys 0.084s
W20260709 13:07:00.989022  3803 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.030s	sys 0.084s
I20260709 13:07:00.990841  4032 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:00.970901 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:00.994890 29068 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:00.914880  3745 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:07:00.976804 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51913.6us
I20260709 13:07:00.971845  3904 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:00.965240  4082 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:01.015846 29072 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:00.971899  4115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.002s	sys 0.072s
I20260709 13:07:00.895629  3846 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:00.981612  3817 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:01.050947 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:00.993462  4017 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:01.052212  4077 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:07:01.053488 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 159334us
I20260709 13:07:01.035701  3745 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:01.061213 29215 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:00.977669  4068 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.052140  3940 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:00.972545  4104 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.078455  3904 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:01.083002  4104 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.086922 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:01.086779  3850 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:01.057382  4077 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:01.016602  4167 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.053081  4082 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:01.098104  3814 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.512s	user 0.007s	sys 0.004s
I20260709 13:07:01.059834  3817 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.098062  4077 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:01.097373  4073 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.158032 29064 raft_consensus.cc:2410] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 13:07:01.158696 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.165468 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 62617.4us
I20260709 13:07:01.165580 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.168246 29068 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
W20260709 13:07:01.160761 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.170120 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.126101  3863 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:01.003355  3916 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.113318  3940 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:01.151196  4151 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:01.157577  4082 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:01.180737 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:01.181448 29213 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:01.184284 29906 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:07:01.160557  3850 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:07:01.187510 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 375465us
I20260709 13:07:01.188170 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.195739 29068 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
W20260709 13:07:01.175750  4019 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.011s	sys 0.034s
I20260709 13:07:01.103377  3904 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:07:01.220635 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.221583 29219 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:01.226248  3844 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:01.226868  3844 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:07:01.216094  4019 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.012s	sys 0.034s
W20260709 13:07:01.229254 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.230909 29215 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:07:01.231995 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.234467 29213 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:07:01.191087  4166 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:01.235559 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.242070  3904 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.242902  3904 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.245687  3904 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.246340  3904 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:01.246982  3904 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.248097  4104 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:01.248854  4104 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.251448  4104 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:01.253453  4187 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
W20260709 13:07:01.256079 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.220511  3850 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:01.268515 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 396726us
I20260709 13:07:01.259541  3850 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:01.272500 29219 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:01.274026 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.274714 29213 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:07:01.285089 29068 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:01.309864 29215 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:01.310835 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:01.312435 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.220180  4151 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:01.313457 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44162.9us
W20260709 13:07:01.285051  4095 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.668s	user 0.013s	sys 0.000s
I20260709 13:07:01.333691 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.325845 28962 catalog_manager.cc:5697] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 11 to 15, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 15 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:01.271127  4104 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:01.334499 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:07:01.345099 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:01.341104 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.346710 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.347187 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.343040 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.345016 29219 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:01.336699 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3150) took 1607 ms. Trace:
I20260709 13:07:01.364140 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.321035  3846 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.236904  3844 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.382107 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:01.350008 29148 rpcz_store.cc:276] 0709 13:06:59.723062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:59.727633 (+  4571us) service_pool.cc:224] Handling call
0709 13:06:59.728743 (+  1110us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:59.729911 (+  1168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:59.729950 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:00.132101 (+402151us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b2c00010da8. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:07:00.132784 (+   683us) log.cc:844] Serialized 52 byte log entry
0709 13:07:00.587802 (+455018us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:01.070174 (+482372us) log.cc:844] Serialized 14 byte log entry
0709 13:07:01.070461 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.070562 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:01.148695 (+ 78133us) raft_consensus.cc:1685] finished
0709 13:07:01.158098 (+  9403us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.158294 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":171375,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":18812,"raft.run_cpu_time_us":560,"raft.run_wall_time_us":4868,"spinlock_wait_cycles":94731008,"thread_start_us":876595,"threads_started":2,"wal-append.queue_time_us":683741}
I20260709 13:07:01.409312 29213 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:01.346546  4197 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.366830 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.410375 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.441020 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:01.442190 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.411509 29113 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:01.433030 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:01.384574 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:01.418478  3890 consensus_meta.cc:287] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.000s
W20260709 13:07:01.420032  4191 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:07:01.420436 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:01.420892  4185 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.179s	user 0.001s	sys 0.001s
I20260709 13:07:01.421203 29069 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:07:01.421270  4105 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.824s	user 0.012s	sys 0.001s
I20260709 13:07:01.421610  4080 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:01.422681  4192 log.cc:927] Time spent T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:07:01.540307  4080 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:01.423664 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 69149.8us
I20260709 13:07:01.542997 29188 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:01.543284  4080 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.544155  4088 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:01.545092  4088 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:01.547561  4088 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:01.424336  3827 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:01.429282 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.344046  4104 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.549209  3827 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:01.549796 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.550722  4104 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.414785 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:01.550539  3827 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:01.551664 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 137181us
I20260709 13:07:01.553673 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:01.554653  4204 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:07:01.555184 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.346423  3863 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:01.556768  4204 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:01.562747  4166 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:01.563257  4166 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:07:01.365413 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 179737us
W20260709 13:07:01.442608 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 50374us
I20260709 13:07:01.566416  4204 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:01.568409 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:01.392095  4153 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:01.570235  4153 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:01.571485  4153 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.565987 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:01.573809  4199 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:01.574543  4199 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:01.575038  4199 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:01.575493  4214 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:01.577603 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:01.563683  4166 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:01.371232  4068 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:01.457056  3817 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.639336 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.457755 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.458254  3890 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.458933 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:01.442572  4063 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:01.460716  4197 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.641045 29139 raft_consensus.cc:2410] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:07:01.640764  4063 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.642282  4197 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:01.464095 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.643204 29213 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:07:01.473219 29219 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
W20260709 13:07:01.482335 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 71218.7us
I20260709 13:07:01.439226  4167 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:01.418485  3844 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:01.484287  3850 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:01.484688 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:01.488893  3904 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.489734  3846 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:01.489975 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70990.5us
I20260709 13:07:01.655993 29186 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:01.656553 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:01.496558  4090 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.003s
I20260709 13:07:01.500859 29148 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:01.502072 29144 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:07:01.479682  4215 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:07:01.511155 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 103346us
I20260709 13:07:01.643149  4214 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:01.522943 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:01.527849  4196 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.528380 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.529242 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.530000 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.535789 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.467554 29072 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:01.341077  4095 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.633822 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:01.642251 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:01.671265 29139 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:01.460768 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:01.672336 29140 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:01.672042  4196 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.673807  4095 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:01.674491 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:01.675151 29144 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:01.676124 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:01.676934 29144 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:01.677662 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:01.678196 29144 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:07:01.678973 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:01.648633  4104 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.680313  4104 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:01.682057  4017 consensus_meta.cc:287] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.000s
I20260709 13:07:01.682559  4017 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.645428 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:01.649504 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:01.684018  4077 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.000s
I20260709 13:07:01.684446  4077 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.651579 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.686062  3890 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.686533  4077 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:01.686866  3890 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:01.687005 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.687152  4077 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:01.658795 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:01.687877 29213 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:01.661304 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:01.688807 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.515758  3940 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:01.689396 29213 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:01.689823  3940 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.690660 29213 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
W20260709 13:07:01.690951  4211 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.172s	user 0.002s	sys 0.000s
I20260709 13:07:01.666379 29217 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:01.644111 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:01.692106  4090 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:01.692848 29143 raft_consensus.cc:2393] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:01.692907 29217 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 13:07:01.693392 29217 pending_rounds.cc:85] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 13
I20260709 13:07:01.693472  3904 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.694213  3904 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.695422  4095 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:01.695852  4095 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.696131 29141 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:01.696290  4095 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:01.654013 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.697340 29221 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:01.666917 29066 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 25: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:01.665546 29073 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:07:01.700647  3940 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.701179 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:01.701964 29911 raft_consensus.cc:2368] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22.
I20260709 13:07:01.650523  3850 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.702859  3850 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:01.668299 29146 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:07:01.703874  3940 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.693704 29217 pending_rounds.cc:107] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 13:07:01.707064  4215 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.707616 29217 raft_consensus.cc:2889] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:01.688050  4201 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:07:01.647866  4080 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.709667  4080 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.666690 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:01.711668  4151 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:01.687389  3890 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:07:01.715216 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 81543.5us
I20260709 13:07:01.692690  4090 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.639178  4133 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:07:01.705848 29146 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:01.703768 29147 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
W20260709 13:07:01.727698 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 61092.1us
I20260709 13:07:01.724781 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:01.730137 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:01.682950  4104 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.718761 29144 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
W20260709 13:07:01.724128 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 53904.1us
I20260709 13:07:01.691707  3929 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:01.656790  4063 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:01.726518 29906 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:07:01.695802 29143 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:07:01.711907  4080 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:01.736765 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:01.735857 29112 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:01.723891  4201 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:01.649995  4197 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:07:01.744966 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:01.748863 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.764273 29068 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:07:01.750775 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:01.751362 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:01.649895  4217 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 13:07:01.755877 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.749823 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:01.723665  3890 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:01.755759 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:01.769866 29073 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:01.767575 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.712891  3904 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:01.765512 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.734077  4151 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.744751  4079 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:01.821040  4079 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:01.821487  4079 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:01.738229  3846 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:01.733213  4019 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:01.780894  4181 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 13:07:01.782645  4063 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:01.738592  4073 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:01.799126  3890 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.759953  3814 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:01.841385  3904 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:01.801800 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 66063.9us
I20260709 13:07:01.729149  3940 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:01.805138  4194 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.417s	user 0.000s	sys 0.001s
I20260709 13:07:01.805694 29073 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
W20260709 13:07:01.806056  3745 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.708s	user 0.010s	sys 0.000s
I20260709 13:07:01.806294  4153 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:01.807714  4215 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:01.815236  4082 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.000s
I20260709 13:07:01.816406  4151 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.818432 29111 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:01.820272  4090 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:01.820658 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.833747 29037 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:01.919239 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 158663us
W20260709 13:07:01.920245 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 86597.8us
I20260709 13:07:01.920989 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:01.845080 29140 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:07:01.855957 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:01.922158 29064 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:07:01.733601  4167 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:07:01.861241 29145 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:01.861764 29906 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
W20260709 13:07:01.915128 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 271077us
I20260709 13:07:01.691188  3863 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.763233  3929 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:01.801715  4104 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:01.883267 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:01.819823  4197 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:01.841807  3846 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:01.888784 29186 leader_election.cc:400] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22.
I20260709 13:07:01.861245  4068 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.890604 29068 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:07:01.934285 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:01.893631 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1303) took 1384 ms. Trace:
I20260709 13:07:01.863813  4214 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:01.907881 29112 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:01.871238  4105 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:01.871641  4073 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:01.881136  3814 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:07:01.926077 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 107725us
I20260709 13:07:01.915755  3827 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:01.891507  4019 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:01.930207 29186 leader_election.cc:403] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:07:01.877616  4063 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.892021  3904 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.925784  4082 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:01.926482 29113 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:01.928436  3817 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:01.934978 29216 rpcz_store.cc:276] 0709 13:07:00.509232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.509428 (+   196us) service_pool.cc:224] Handling call
0709 13:07:00.531719 (+ 22291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.588128 (+ 56409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.588172 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.636564 (+ 48392us) log.cc:844] Serialized 26 byte log entry
0709 13:07:01.149117 (+512553us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:01.149180 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.149281 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:01.804449 (+655168us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:01.893330 (+ 88881us) raft_consensus.cc:1685] finished
0709 13:07:01.893352 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:01.893569 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20185,"raft.queue_time_us":2635,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5347,"spinlock_wait_cycles":89089792,"thread_start_us":406811,"threads_started":1,"wal-append.queue_time_us":633308}
I20260709 13:07:01.461663 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:01.934206  4080 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.930832  3803 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:01.934753  3846 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:07:01.971848 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 45472.2us
I20260709 13:07:01.954347  3814 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:01.978036 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:01.952340  4073 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:01.949265  4231 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.980609 29213 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:07:01.974946  4217 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:01.977356  4215 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:01.952303  4090 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:01.910790  3745 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.972121  4077 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:07:01.988611 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.988763  4214 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:01.991110 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 111710us
W20260709 13:07:01.980505 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 54310us
I20260709 13:07:01.974246 28962 catalog_manager.cc:5697] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 16 to 17, leader changed from <none> to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 17 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:01.995251 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:01.997340 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 89578.9us
I20260709 13:07:02.009255 29112 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:01.948019  4068 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.955756  3850 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.954756  4197 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:01.952880  4105 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.995585  4076 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:01.990217  4151 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.018692 29904 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:01.970786  3827 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:01.999626  4215 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:01.997838  4202 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:01.989619  4019 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.035955 29140 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:07:01.983526  4080 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.022435 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.023435 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.015151  4217 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:01.977537  4166 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.049014  4219 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:02.049804  4236 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:02.050490  4166 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.052896  3940 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:02.021994  4088 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:02.053798  4224 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:07:02.006003  4234 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.054085  4088 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.054651  4234 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.055269 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.037195 29142 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:07:02.056510 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.057571 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.058653 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.998319  3803 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.060294 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.060874 29908 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:07:02.062057 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:02.062137  4217 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.032840  4215 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:02.063015  3894 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:02.063437  3894 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.063784  3894 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:02.022359  4076 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.046844 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:02.048617  4200 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.618s	user 0.003s	sys 0.000s
I20260709 13:07:02.067596 29910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 1188) took 1375 ms. Trace:
I20260709 13:07:02.068095 29910 rpcz_store.cc:276] 0709 13:07:00.692689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.740995 (+ 48306us) service_pool.cc:224] Handling call
0709 13:07:00.749902 (+  8907us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:00.750427 (+   525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:00.750468 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:00.802043 (+ 51575us) log.cc:844] Serialized 26 byte log entry
0709 13:07:01.160387 (+358344us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:01.160453 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:01.160560 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:02.014588 (+854028us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:02.067227 (+ 52639us) raft_consensus.cc:1685] finished
0709 13:07:02.067254 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.067504 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":145,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":25038,"spinlock_wait_cycles":6695808,"thread_start_us":304487,"threads_started":1,"wal-append.queue_time_us":507905}
I20260709 13:07:02.048331  4206 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.078s
W20260709 13:07:02.053037  4222 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:07:02.074594 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:01.998411  4205 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
W20260709 13:07:02.096045  4220 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
W20260709 13:07:02.102403 29145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 3157) took 1932 ms (client timeout 1830 ms). Trace:
W20260709 13:07:02.103034 29145 rpcz_store.cc:269] 0709 13:07:00.170408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:00.170672 (+   264us) service_pool.cc:224] Handling call
0709 13:07:02.102068 (+1931396us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540027d8d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:02.102258 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3385530752}
I20260709 13:07:02.110611 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.111568 29220 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
W20260709 13:07:02.104171  4233 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:07:02.114486 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:07:02.118095 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 71333.7us
W20260709 13:07:02.121878 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 200318us
I20260709 13:07:02.099665 28962 catalog_manager.cc:5697] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 14 to 15, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 15 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:02.059672 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.123905 29871 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:02.001394  4231 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.003455  3929 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.086738  3890 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:02.133912 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 138743us
I20260709 13:07:02.140874 29146 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:02.037698  4196 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.142949 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.138957 29185 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:02.161000 29220 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:07:02.169124  4252 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:02.169855  4166 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:02.174692 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.179276 29214 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 13:07:02.179190 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55401.6us
I20260709 13:07:02.185959 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.188164 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:02.201393 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:02.188093 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:02.208328  4254 consensus_queue.cc:1059] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:02.209589  4254 raft_consensus.cc:3055] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:02.209959  4254 raft_consensus.cc:740] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:02.177205  4076 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:02.211815 29145 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:07:02.212306 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:02.214583 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:02.173941  4252 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:02.176517  3745 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:02.224536 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 234650us
W20260709 13:07:02.230280 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 96165us
I20260709 13:07:02.228601  4206 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:07:02.233644  4206 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:02.217743  3940 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:02.219939  4252 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:02.217165  3803 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:02.250969 29139 raft_consensus.cc:1240] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:07:02.211768  4166 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.244374  3940 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.235870  4234 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:02.242239  3904 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:02.271972 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:02.284438 29139 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:02.211040  4254 consensus_queue.cc:260] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:02.277387 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 54326.1us
W20260709 13:07:02.257210  4199 consensus_meta.cc:287] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.682s	user 0.006s	sys 0.014s
I20260709 13:07:02.253363  4195 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:02.260092  3850 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:02.281970  4188 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.912s	user 0.007s	sys 0.000s
I20260709 13:07:02.258716  4068 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.277662  3940 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:02.297384  3844 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.645s	user 0.005s	sys 0.008s
I20260709 13:07:02.293067  3904 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.298148  4229 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:02.313016 29215 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
W20260709 13:07:02.316274  4247 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:07:02.323029  4231 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:02.324285  4207 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:02.324810  4207 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:02.325256  4207 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:02.330148  4166 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:02.289289  4079 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.337999  4079 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.339066  3904 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.341790  3904 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.365326  4215 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.370132  4215 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.371141  3894 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.372159  4240 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:02.379822  4240 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.291590  4199 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.382478  4199 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.383240  4240 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:02.385790  3894 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:02.368049  4090 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:07:02.304317  3827 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:02.359377  4254 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:02.397475  4095 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.701s	user 0.006s	sys 0.005s
I20260709 13:07:02.398149  4095 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.399832 29911 raft_consensus.cc:2368] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25.
I20260709 13:07:02.412530  4080 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:02.415570 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.420825 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:02.421203 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:02.421795 29146 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
I20260709 13:07:02.422263 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:02.422403 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 466135us
I20260709 13:07:02.423178 29111 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:02.429025 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.436131  4080 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:02.436600  4080 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:02.437028  4080 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:07:02.439756  4073 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.435727 29214 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:07:02.441287  4230 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.492s	user 0.003s	sys 0.000s
I20260709 13:07:02.441847 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.442575 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:02.443092 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.443768  4243 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:02.445676  4073 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.445842  4243 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:02.449259 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:02.450367  3846 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.455542 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:07:02.458532 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:02.466213  4197 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.466794  4201 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:02.467478  4088 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:02.468680 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:02.549039 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 80463.8us
I20260709 13:07:02.552085 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.470701  4229 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:02.467494  4244 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.493386 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3225) took 1018 ms. Trace:
W20260709 13:07:02.473749  4242 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.272s	user 0.002s	sys 0.000s
I20260709 13:07:02.495427  4063 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:02.498560  3814 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.499277 29215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 3169) took 1188 ms. Trace:
I20260709 13:07:02.563519 29215 rpcz_store.cc:276] 0709 13:07:01.311226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.369760 (+ 58534us) service_pool.cc:224] Handling call
0709 13:07:02.499097 (+1129337us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b54002f58d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:02.499217 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1991255808}
I20260709 13:07:02.500633  4095 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.501140  4017 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.478924  4080 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.503000  4236 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22.
I20260709 13:07:02.503711 28962 catalog_manager.cc:5697] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 16 to 22, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 22 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.506328 29220 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:07:02.506789 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:02.510916 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 371999us
I20260709 13:07:02.573853 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:02.574752  3814 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.511899 29221 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:07:02.513384 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.578330 29216 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:02.514371  4240 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.579100  3814 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:02.519752 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:02.579825  3814 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:02.580766 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 61098us
I20260709 13:07:02.582196 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:02.521072 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.523205 29139 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
W20260709 13:07:02.528245 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 79061.8us
I20260709 13:07:02.531258 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3326) took 1100 ms. Trace:
I20260709 13:07:02.494307  4105 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.554939 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.556099 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:02.583266  3814 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.579450  4240 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.558082 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.554124 29142 rpcz_store.cc:276] 0709 13:07:01.474582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.475478 (+   896us) service_pool.cc:224] Handling call
0709 13:07:01.496507 (+ 21029us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:02.056096 (+559589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.493105 (+437009us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b54001bbcd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:02.493324 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1242873216}
I20260709 13:07:02.524101  4205 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:02.589895  4205 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.570268  4080 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.576663  3844 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:02.593984  3844 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.594616  3844 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:02.594985 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.556970  4206 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:02.561654  4262 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:02.596375  4206 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.584234 29217 rpcz_store.cc:276] 0709 13:07:01.430989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.710238 (+279249us) service_pool.cc:224] Handling call
0709 13:07:01.710530 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:01.726999 (+ 16469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:01.727026 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:01.760750 (+ 33724us) log.cc:844] Serialized 26 byte log entry
0709 13:07:01.938446 (+177696us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:01.975784 (+ 37338us) log.cc:844] Serialized 14 byte log entry
0709 13:07:02.443933 (+468149us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b18002b3840. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:07:02.530840 (+ 86907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.530917 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:02.530965 (+    48us) raft_consensus.cc:1685] finished
0709 13:07:02.531063 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.531198 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46003,"prepare.queue_time_us":40480,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":177673,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":35406,"spinlock_wait_cycles":2813312,"thread_start_us":211845,"threads_started":2,"wal-append.queue_time_us":207763}
I20260709 13:07:02.584267  3890 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:02.599663 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.584628 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:02.600814 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.584056  4197 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:02.602676  4197 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:07:02.602988  4207 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.558883  4201 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.599298  3890 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.606071  4201 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.606396  3890 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:02.533627  4243 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:02.607513  4243 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:02.608736 29908 raft_consensus.cc:2410] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:07:02.596822  4262 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:02.611764  4262 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:02.612511  4240 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:02.583283 29113 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:02.613515  3940 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.614809 29218 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:02.533215  4032 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 13:07:02.616329  4104 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:02.617141  4104 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.617604  4104 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:02.620819  3940 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:02.555207  4151 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:02.622191  4151 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.622685  4151 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:02.562222  4244 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.624382 28962 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 12 to 15, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 15 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:02.626123  4244 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.555101  4229 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:02.626667 28962 catalog_manager.cc:5697] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 11 to 15, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 15 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:07:02.629709 28963 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 275) took 56399 ms (client timeout 15000 ms). Trace:
I20260709 13:07:02.631312  4207 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:02.599390  4252 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.575583 29185 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:02.551782 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 263259us
W20260709 13:07:02.638306 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 62375.7us
I20260709 13:07:02.596817  4205 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.641876 29037 leader_election.cc:400] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25.
I20260709 13:07:02.649163 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.639245 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:02.659493 29906 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:07:02.660444 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:02.662213 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.662511 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.663239 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.675415 29142 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:07:02.644232  4088 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:07:02.599390  4206 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:02.663512 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.679838 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:02.679961 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:02.680534 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:02.680778 29139 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:07:02.681147 29910 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:02.663585 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.672907 29904 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:07:02.664371 29065 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:02.678047  3842 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:02.650517 28962 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:02.664604 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:02.677688 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:02.678380 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:02.684782 29146 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:02.682399 29905 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:07:02.680810 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:02.685622 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:02.686208 29067 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:07:02.683291 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:02.688889 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:02.739401 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 56221.1us
I20260709 13:07:02.740332 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.656386  4032 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:07:02.749511 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 67297us
I20260709 13:07:02.754369 29139 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:02.758736 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.760819 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.761363 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.687584 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.763818 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.764182 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.764195 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.662039  4265 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:02.650225  4240 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.091s
I20260709 13:07:02.687146 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:02.770709 29112 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:02.772392 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.766860 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.790486 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:02.791006 29216 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 16: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:02.738860  3929 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:02.764750 28962 catalog_manager.cc:5697] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:02.793918 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.755191  3803 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:02.778483 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:02.783077 29220 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 23: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:02.801666 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.784571 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.775323 29908 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:07:02.782338 29217 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:02.787954 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.806250 29221 raft_consensus.cc:3060] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:02.661571  4262 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.791152 29904 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:02.796428 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:02.796779 29112 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:02.797667 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 110612us
I20260709 13:07:02.838764 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:02.808019 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:02.841606 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44939.2us
I20260709 13:07:02.835429 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:02.843215 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:02.843712 29142 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:02.845291 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48971us
I20260709 13:07:02.788734  4265 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.837821 28962 catalog_manager.cc:5697] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 13 to 15, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 15 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:02.799405  3814 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.793133  3929 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:02.857369 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:02.838076 29874 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:02.802156  3803 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.799664  4082 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:02.807278  4073 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:02.865125 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:02.884311 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:02.885227 29140 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:07:02.884193 29216 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:07:02.839167  4254 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:02.887351 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:02.904197 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:07:02.928853 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 121196us
I20260709 13:07:02.932363 29143 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:02.792589  4032 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.901201  4017 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:02.937371 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:02.935207 29216 raft_consensus.cc:1240] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 13:07:02.867491  3929 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:02.932205 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 85957us
I20260709 13:07:02.937749  4105 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:02.936524  4082 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.940922  4201 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:02.947021  4082 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.950426 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:02.955557 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:02.956765 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:02.957757 29064 raft_consensus.cc:2410] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:07:02.958473 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:02.959474 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:02.862458  4265 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:02.960381 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:02.867790  4244 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:02.961380  4244 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:02.961562 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:02.864566  4166 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:02.960268 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:02.964833 29220 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:07:02.941188 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:02.941352 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:02.854810  4240 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:02.966420 29140 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:02.942554 28962 catalog_manager.cc:5697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 13 to 18, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 18 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:02.941984 29221 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
I20260709 13:07:02.861563  4195 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:02.971856 29064 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:02.962157 29070 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:02.973272 29906 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:07:02.974373 29906 pending_rounds.cc:85] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 12
I20260709 13:07:02.974627 29906 pending_rounds.cc:107] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:07:02.974862 29906 raft_consensus.cc:2889] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:02.972841 29908 raft_consensus.cc:2368] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25.
I20260709 13:07:02.978166 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:02.964814 29143 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:02.982064 29142 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:02.982841 29073 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:07:02.968370 28962 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 17 to 19, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 19 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:02.986182  4082 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:02.986359  4244 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:02.986778  4082 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.988452  3846 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:02.989037  4082 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.955106  3890 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.990545  4063 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:02.990944  4063 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:02.991292  4063 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:02.868009  3814 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.993170  3814 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:02.937462  4017 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:02.994453  4017 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:02.996390  4269 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
W20260709 13:07:02.997473  4270 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 13:07:02.996567 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 59275.9us
I20260709 13:07:02.999018 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3164) took 1689 ms. Trace:
I20260709 13:07:02.998783  3844 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:02.939769  4258 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:07:03.002235 29141 raft_consensus.cc:2368] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:07:02.996024  4240 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:02.995692  4017 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:02.890884  3803 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:02.999455 29222 rpcz_store.cc:276] 0709 13:07:01.309966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:01.310073 (+   107us) service_pool.cc:224] Handling call
0709 13:07:01.310350 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:01.340107 (+ 29757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:01.340137 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:01.354267 (+ 14130us) log.cc:844] Serialized 26 byte log entry
0709 13:07:01.508336 (+154069us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b18003e20c0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20557 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:07:02.427156 (+918820us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b18003e20c0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:07:02.633110 (+205954us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:02.647447 (+ 14337us) log.cc:844] Serialized 14 byte log entry
0709 13:07:02.647705 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:02.647799 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:02.998781 (+350982us) raft_consensus.cc:1685] finished
0709 13:07:02.998802 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:02.998956 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7541,"prepare.queue_time_us":4377,"prepare.run_wall_time_us":3189,"raft.queue_time_us":48726,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":51156,"spinlock_wait_cycles":44915072,"thread_start_us":205770,"threads_started":1,"wal-append.queue_time_us":1373017}
W20260709 13:07:03.021215 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 133956us
I20260709 13:07:03.029631 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:03.021764 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:02.964964  4200 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:07:02.986469 29071 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:02.998545 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:03.024288 28962 catalog_manager.cc:5697] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:07:03.025209 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47160.4us
W20260709 13:07:03.047245 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 48793.4us
I20260709 13:07:02.940121  4073 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:03.030463 29068 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 15.
I20260709 13:07:02.965863  3904 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:03.061081 29874 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:02.980724  4217 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:03.072672 29215 raft_consensus.cc:2368] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24.
I20260709 13:07:03.067899 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:02.999116  4151 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:03.074261 29144 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:03.075767 28962 catalog_manager.cc:5697] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 19, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 19 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.936182  4254 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:03.087436 28962 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 17 to 22. New cstate: current_term: 22 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:03.071853  4203 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:03.065464  4252 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.106040 29147 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
W20260709 13:07:03.101603 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 258434us
I20260709 13:07:02.980290  4166 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.110412 28962 catalog_manager.cc:5697] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 18 to 21, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 21 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.097033 29064 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:03.083880  4095 raft_consensus.cc:2804] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:03.113808 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:03.121398 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:03.107585  4203 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.133548 29910 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:03.136154 28962 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 20, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 20 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:03.137578  4166 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.138257  4203 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:03.138425 28962 catalog_manager.cc:5697] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 16 to 19, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 19 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:03.124830  3904 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:03.143198  4254 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:03.145651  4264 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
I20260709 13:07:03.146165  4235 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:03.147038  4032 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.070923  4229 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:03.155967 29146 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:07:03.124639  4279 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:03.130026  4245 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:07:03.130744  4095 raft_consensus.cc:697] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:03.167539 28962 catalog_manager.cc:5697] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 9 to 14, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 14 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:03.156891  4202 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:03.184231 29066 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
W20260709 13:07:03.195750 29065 consensus_meta.cc:287] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
W20260709 13:07:03.196516 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 75219.7us
I20260709 13:07:03.197130 29065 raft_consensus.cc:2468] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
I20260709 13:07:03.192495 28962 catalog_manager.cc:5697] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 17 to 19, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 19 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:03.201138 29112 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:03.169497  4262 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:03.194128  4274 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:07:03.212898 29185 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:03.208799 28962 catalog_manager.cc:5697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 11 to 17, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 17 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:03.216032 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 279192us
I20260709 13:07:03.220069 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:03.226840 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:03.228232 29064 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:03.234527 29065 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:03.157514  4073 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:03.221421 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 147062us
I20260709 13:07:03.239866 28962 catalog_manager.cc:5697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 6 to 21, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 21 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:03.252835 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.255261 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:03.251130 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:03.204358  4032 consensus_queue.cc:1059] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:07:03.204972  4088 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:07:03.202571  4202 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.204555  3827 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:07:03.264225 29066 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:03.219970  3846 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:03.251719  4279 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:07:03.257954  4273 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:07:03.268585 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:03.215229  4019 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:07:03.255707  3844 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:03.253518  4063 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:03.253713  4234 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 13:07:02.630183 28963 rpcz_store.cc:269] 0709 13:06:06.230547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.230794 (+   247us) service_pool.cc:224] Handling call
0709 13:07:02.629479 (+56398685us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:51.715119 (+     0us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b18003e2ba0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:06:51.715330 (+   211us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:51.754496 (+ 39166us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:51.760271 (+  5775us) write_op.cc:435] Acquired schema lock
0709 13:06:51.761925 (+  1654us) tablet.cc:662] Decoding operations
0709 13:06:52.313525 (+551600us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:52.313667 (+   142us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:52.313696 (+    29us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:06:52.314645 (+   949us) tablet.cc:701] Row locks acquired
0709 13:06:52.314723 (+    78us) write_op.cc:260] PREPARE: finished
0709 13:06:52.314953 (+   230us) write_op.cc:270] Start()
0709 13:06:52.315197 (+   244us) write_op.cc:276] Timestamp: P: 1783602412314861 usec, L: 0
0709 13:06:52.315232 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:06:52.322574 (+  7342us) log.cc:844] Serialized 4010 byte log entry
0709 13:06:53.157551 (+834977us) op_driver.cc:464] REPLICATION: finished
0709 13:06:53.548128 (+390577us) write_op.cc:301] APPLY: starting
0709 13:06:53.548281 (+   153us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:53.555829 (+  7548us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:53.555871 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:54.500575 (+944704us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:54.557582 (+ 57007us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:06:54.557658 (+    76us) write_op.cc:312] APPLY: finished
0709 13:06:54.558065 (+   407us) log.cc:844] Serialized 97 byte log entry
0709 13:06:55.360261 (+802196us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f7833a030ac 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b
0709 13:06:55.836420 (+476159us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:55.836567 (+   147us) write_op.cc:454] Released schema lock
0709 13:06:55.836639 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:55.837081 (+   442us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:55.837109 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":573043,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":374444,"apply.run_cpu_time_us":53709,"apply.run_wall_time_us":2288964,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45927,"num_ops":10,"prepare.queue_time_us":145784,"prepare.run_cpu_time_us":57107,"prepare.run_wall_time_us":838311,"raft.queue_time_us":481179,"raft.run_cpu_time_us":24588,"raft.run_wall_time_us":347008,"replication_time_us":808093,"spinlock_wait_cycles":121567232,"thread_start_us":1974178,"threads_started":5,"wal-append.queue_time_us":1243591,"wal-append.run_cpu_time_us":34114,"wal-append.run_wall_time_us":2608129}]]}
I20260709 13:07:03.355860  4287 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:03.356685  4287 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:03.357807  4287 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:03.284996 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:03.364900 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:03.365267 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:03.341812 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3346) took 1211 ms. Trace:
I20260709 13:07:03.298830 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:03.238962  4280 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:07:03.348107 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 79770.3us
I20260709 13:07:03.306241 28962 catalog_manager.cc:5697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.257359  3745 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:03.366617 29217 rpcz_store.cc:276] 0709 13:07:02.130807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.866280 (+735473us) service_pool.cc:224] Handling call
0709 13:07:02.866570 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:02.866763 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 13:07:03.175664 (+308901us) log.cc:844] Serialized 14 byte log entry
0709 13:07:03.341334 (+165670us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:03.341410 (+    76us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:07:03.341472 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.341564 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.341734 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":386796,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":13541,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":473762,"threads_started":2}
I20260709 13:07:03.373652 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:03.381510 29216 raft_consensus.cc:2368] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20.
W20260709 13:07:03.366032  4243 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.758s	user 0.013s	sys 0.000s
I20260709 13:07:03.275532  4279 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.310936  4232 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.411595 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:03.374279  4283 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:07:03.304514  4240 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.408772 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:03.408942 29065 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:07:03.407286 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:03.409070 28962 catalog_manager.cc:5697] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:03.312407  4254 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:03.413188 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:03.418995 29071 raft_consensus.cc:2410] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:07:03.312901  3846 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:03.413918 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:03.304828  4151 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:03.418519  4254 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:03.448119  4254 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:03.455343 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3342) took 1466 ms. Trace:
I20260709 13:07:03.455847 29214 rpcz_store.cc:276] 0709 13:07:01.988982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.840226 (+851244us) service_pool.cc:224] Handling call
0709 13:07:02.840609 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:02.860412 (+ 19803us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.860443 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:02.953010 (+ 92567us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b2c00127298. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:07:02.953442 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 13:07:03.154564 (+201122us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:07:03.154629 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.154770 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:03.455051 (+300281us) raft_consensus.cc:1685] finished
0709 13:07:03.455073 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.455279 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52530,"raft.queue_time_us":26808,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":17090,"spinlock_wait_cycles":4353280,"thread_start_us":200665,"threads_started":1,"wal-append.queue_time_us":282974}
I20260709 13:07:03.457679  4063 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.416260  4232 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.461133 29221 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:03.462113  4232 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:03.408226  4243 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:03.464725  4265 consensus_meta.cc:287] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.000s
I20260709 13:07:03.465174  4265 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.466114  4243 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.440054  4280 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.468142 29905 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:03.468483  4280 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.401536  4271 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:03.469714  4271 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:03.470789  4280 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.470623  4271 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.471244  3814 raft_consensus.cc:3055] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:07:03.471778  3814 raft_consensus.cc:740] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:03.471854  3846 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.472761  3814 consensus_queue.cc:260] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.474004  3814 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:03.476446  4202 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:03.408468  4279 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:03.445752 28962 catalog_manager.cc:5697] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:03.279980  4281 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.176s
I20260709 13:07:03.466275 29219 raft_consensus.cc:2368] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
W20260709 13:07:03.417174  4104 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.799s	user 0.012s	sys 0.006s
I20260709 13:07:03.262473  4095 consensus_queue.cc:237] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:03.488286 28962 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 19, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 19 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:03.492043  4202 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:03.485520  4271 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:03.504285 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:03.492259  4254 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:03.506428 28962 catalog_manager.cc:5697] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 5 to 15, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 15 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:03.506672 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1316) took 1012 ms. Trace:
I20260709 13:07:03.504930 29142 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:03.507619 29220 rpcz_store.cc:276] 0709 13:07:02.489876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.420760 (+930884us) service_pool.cc:224] Handling call
0709 13:07:03.502078 (+ 81318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:03.502316 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:03.502353 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:03.502410 (+    57us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:07:03.502456 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.502541 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.502678 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:03.511866 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:03.511893 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:03.512667 29148 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:03.513082 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:03.541002 28962 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 14 to 19, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 19 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:03.543555 28962 catalog_manager.cc:5697] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 13 to 17, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 17 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:03.553227  4254 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.554180  3890 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.555891  4229 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:03.558116  3803 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:03.560513  4272 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.604s	user 0.003s	sys 0.000s
I20260709 13:07:03.560920 29145 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:03.564180  3803 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.503831  4203 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:03.504266  4104 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.566236  4203 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.569288 29142 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:07:03.569518 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:03.570293 29066 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:03.570983 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:03.571209 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:03.571794 29071 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:03.572706 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:03.573186 29070 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:03.577837 29185 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:03.578718  3894 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:03.579154  3894 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:03.579603  3894 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:03.581403  4104 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:03.583225  3929 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.000s
I20260709 13:07:03.589267 29070 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
I20260709 13:07:03.590821 29185 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:03.591735  4229 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:03.592375  4229 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:03.593536  4229 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.522377  4265 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:03.599627 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:03.600081 29911 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:07:03.601565 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:03.602645 28962 catalog_manager.cc:5697] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 18, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 18 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:03.603596 29142 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:03.546793  4263 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.612897  4203 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:03.613554  4203 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:03.614468  4203 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:03.617763  4017 consensus_meta.cc:287] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.000s
I20260709 13:07:03.617880 29064 raft_consensus.cc:2468] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:03.619876 29185 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:03.620054  4263 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:03.632879 29142 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
I20260709 13:07:03.632422 28962 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 16 to 17, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 17 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:03.633656 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:03.636806 29148 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 26.
I20260709 13:07:03.640012 29213 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:03.645947 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:03.642787 29875 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:03.647760 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:03.583671  3929 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.650574 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:03.654381 29037 leader_election.cc:400] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25.
I20260709 13:07:03.654762 29037 leader_election.cc:403] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:07:03.664090 29064 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:03.669670 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:03.652127  4206 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:03.606014  4104 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:07:03.649559  4240 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:03.652208  4265 raft_consensus.cc:2804] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:03.654284  4068 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:03.652365  4287 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:03.655504  4032 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:03.650653  3745 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:07:03.674651  3814 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25.
I20260709 13:07:03.672113  3842 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:07:03.705636  3894 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.651090  4166 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 13:07:03.700423  4032 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:03.721973  4032 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:03.709719  4265 raft_consensus.cc:697] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:07:03.726117  4295 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.168s	user 0.002s	sys 0.000s
I20260709 13:07:03.726711  4265 consensus_queue.cc:237] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:03.730383  4296 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 13:07:03.734885  4214 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:03.735529  4214 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:03.744230  4032 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:03.706979  4068 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:03.715308  4287 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:07:03.815769  4301 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:07:03.861928 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:03.863214 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 210012us
I20260709 13:07:03.860261 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3196) took 1185 ms. Trace:
I20260709 13:07:03.865324 29216 rpcz_store.cc:276] 0709 13:07:02.674888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.417269 (+742381us) service_pool.cc:224] Handling call
0709 13:07:03.417570 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:03.417778 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:07:03.436789 (+ 19011us) log.cc:844] Serialized 14 byte log entry
0709 13:07:03.621109 (+184320us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b1800419ec0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a 00007f7836b17a2a
0709 13:07:03.846235 (+225126us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:03.859952 (+ 13717us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:03.860003 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.860071 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:03.860198 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16557,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":241,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"spinlock_wait_cycles":82759040,"thread_start_us":224796,"threads_started":1}
I20260709 13:07:03.874312 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:03.875399 29148 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
W20260709 13:07:03.861023 29188 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.522s (SENT) [suppressed 4 similar messages]
W20260709 13:07:03.943297 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 208002us
W20260709 13:07:03.882207  4303 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.183s	user 0.000s	sys 0.006s
I20260709 13:07:03.880903  4017 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.955088 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:03.956226 29186 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:03.961542 29911 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:07:03.963014 29905 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:03.963147 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:03.878360  4279 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:03.966447 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:03.968997  4302 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:03.955749  3929 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:03.960136  4204 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:03.961695  4263 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:03.985773 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 124773us
I20260709 13:07:03.969166  4202 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:04.001189 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:03.970510  4302 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:03.970109  4214 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.962013  4017 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:03.977499  4229 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:04.011662 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:04.009303 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:03.990720  4202 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.018828 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:04.021575 29071 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:07:04.017444 29871 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:04.023710  4073 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:04.024979  4073 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:04.025424  4073 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:04.026371  4073 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:04.028023  4202 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:04.029603  4240 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:04.031284  4284 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:04.031723  4284 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:04.032639  4308 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:07:04.032737  4284 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:04.034698  4307 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:07:03.989142  4204 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.036038  4206 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:04.034638  4284 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:04.027829  4073 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:04.040589  4204 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:04.039027 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:04.046015  3814 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:04.051815  3814 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:04.052302  3814 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:04.047868 29072 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:04.018322 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:04.056195 29064 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:04.057251 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:04.058941  3850 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:04.059376  3850 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:04.059773  4279 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:04.056496 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:04.003052  3894 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:04.062196 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 44846.3us
I20260709 13:07:04.074909 29072 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:03.976500  4287 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.016562  4309 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.075s
I20260709 13:07:04.020428  3929 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:07:04.020499  4229 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:03.977941  4068 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.072211 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:04.068106  4206 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:07:04.016196  4166 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:07:04.070647  4252 raft_consensus.cc:2764] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:07:04.106844 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:04.079999 29064 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:07:04.073669  4214 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:04.101351 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.101120 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:04.083508  4241 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:04.114485 29871 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:04.121572  4229 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.136060  4263 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:04.140847 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:04.141870 29068 raft_consensus.cc:2393] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:04.141743 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:04.142597 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:04.142884 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:04.143793 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:04.144093 29065 raft_consensus.cc:2368] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
I20260709 13:07:04.143170 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:04.146286 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:04.098739  4279 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:04.154449 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.156000 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.156304 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.156989 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.157214 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:04.098553  4104 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:04.165486 29148 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:04.165879 29140 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:04.104405  4032 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:04.166908 29145 raft_consensus.cc:2410] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:07:04.104288  3863 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.106850  4206 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:04.171816 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:04.172735 29073 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:04.188434 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.189704 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3340) took 2201 ms. Trace:
I20260709 13:07:04.187831 29215 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:07:04.190585 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.191339 29148 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:07:04.170960 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:04.093802  3850 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.183936 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.184674 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:04.191974 29188 leader_election.cc:336] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.830s (SENT)
I20260709 13:07:04.184837 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:04.207139 29142 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:04.207753 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:04.190376 29218 rpcz_store.cc:276] 0709 13:07:01.988406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.839299 (+850893us) service_pool.cc:224] Handling call
0709 13:07:02.839640 (+   341us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:02.883763 (+ 44123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:02.883798 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:03.626038 (+742240us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b18002d8ec0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a
0709 13:07:03.626883 (+   845us) spinlock_profiling.cc:243] Waited 5.71 ms on lock 0x7b2c00127298. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:07:03.627576 (+   693us) log.cc:844] Serialized 52 byte log entry
0709 13:07:03.848655 (+221079us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:03.848782 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.848980 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:04.189261 (+340281us) raft_consensus.cc:1685] finished
0709 13:07:04.189291 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.189598 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46603,"raft.queue_time_us":29358,"raft.run_cpu_time_us":6816,"raft.run_wall_time_us":41719,"spinlock_wait_cycles":21074560,"thread_start_us":220414,"threads_started":1,"wal-append.queue_time_us":476471}
I20260709 13:07:04.203408 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:04.209975 29146 raft_consensus.cc:2368] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 24. Current term is 25.
I20260709 13:07:04.203792 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:04.181114 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.211071 29145 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:04.222313 29068 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 16.
I20260709 13:07:04.162377  3803 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.226513 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:04.173915  4236 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.234174 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:07:04.235667 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 43770us
I20260709 13:07:04.239764 29142 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:07:04.227360 29148 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:04.167727  3940 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:04.232489 29140 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:07:04.173226  3894 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.203358  4206 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:07:04.244966 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 278602us
I20260709 13:07:04.241272 29143 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:04.248364 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:04.105268  4302 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.251065  4302 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:04.253463 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:04.255971 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:04.256668  4316 log.cc:927] Time spent T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:07:04.256785 29215 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:07:04.257329  4017 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:04.212284  3863 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:04.258668 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:04.259424 29220 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:04.260627  3863 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:04.262029  3842 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:04.264122  3894 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:04.214002  4254 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:04.266520 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:04.267714 29147 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:04.275369 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:04.276211 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.277016 29140 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:04.278069 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.174430  4279 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:04.286783 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:04.206539  4263 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.078s
W20260709 13:07:04.288060 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 123581us
I20260709 13:07:04.290452 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:04.297046 29141 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:04.186326  4104 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.294122 29142 raft_consensus.cc:1240] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:04.237689  3846 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:07:04.278064 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 204498us
I20260709 13:07:04.263643  3842 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:07:04.356989 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:04.302156 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:04.242826  3940 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:07:04.351346 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 52487.6us
I20260709 13:07:04.360383 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:04.361662 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.370440 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:04.371726 29140 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:04.353310 29220 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
W20260709 13:07:04.269919  4244 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.005s	sys 0.008s
I20260709 13:07:04.377415 29141 raft_consensus.cc:2393] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:07:04.377703 29213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 1310) took 2406 ms (client timeout 2045 ms). Trace:
I20260709 13:07:04.237874  3803 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:04.378113 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:04.418303 29142 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:04.380360 29221 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:07:04.378090 29213 rpcz_store.cc:269] 0709 13:07:01.958215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.081902 (+123687us) service_pool.cc:224] Handling call
0709 13:07:04.363935 (+2282033us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54002832d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:04.364082 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3446586624}
W20260709 13:07:04.373664 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 219709us
I20260709 13:07:04.292898  4254 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:04.417016 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.245143  4236 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:04.269918  4235 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 13:07:04.422905 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:04.486073 29140 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:07:04.422582 29214 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:04.421804 29148 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
W20260709 13:07:04.467897 29908 consensus_meta.cc:287] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.000s
I20260709 13:07:04.471424 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:04.489701 29908 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:04.491443 29905 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:04.492055 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:04.362051  3929 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.498332 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:04.500773 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:04.506753 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:04.506997 29911 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:04.407401  4244 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.006s	sys 0.009s
I20260709 13:07:04.505645 29143 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:07:04.298535  3814 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.361205  4166 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.145s
I20260709 13:07:04.504837  3745 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:04.539836 29874 leader_election.cc:400] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20.
W20260709 13:07:04.527518  4298 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.970s	user 0.003s	sys 0.000s
I20260709 13:07:04.545795 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.548810 29140 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:04.506079  3846 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:07:04.553994 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 193690us
I20260709 13:07:04.547286 29070 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: []
W20260709 13:07:04.531333 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 170319us
I20260709 13:07:04.555562 29911 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
W20260709 13:07:04.545682 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 298671us
W20260709 13:07:04.581264 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 201428us
I20260709 13:07:04.581877 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:04.555063 29142 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
W20260709 13:07:04.602276 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.536403  4323 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:04.522554  3929 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.522797  4320 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 13:07:04.617797 29221 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
W20260709 13:07:04.654213 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 90203.3us
I20260709 13:07:04.655355 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.656031 29142 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 16.
I20260709 13:07:04.627328  4105 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:04.689129  4105 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.621905  4214 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:07:04.711233 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 162278us
I20260709 13:07:04.712459 29874 leader_election.cc:400] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24.
W20260709 13:07:04.713603 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.692560  4341 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:04.646008  4287 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:04.729375 29219 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:04.744987 29220 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:04.748059 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:04.759354 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.765731 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:04.766546 29213 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:07:04.591693  3846 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:04.769090 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 187304us
I20260709 13:07:04.769868 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:04.691893  4105 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:04.680609  4325 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:04.760268 29147 raft_consensus.cc:2376] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26. Re-sending same reply.
W20260709 13:07:04.784967 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 364458us
I20260709 13:07:04.644994  4323 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.743906  3894 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:04.777359  4293 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 1.325s	user 0.005s	sys 0.007s
I20260709 13:07:04.717837  4341 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:04.646871  3940 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:04.785913  4284 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.747s	user 0.012s	sys 0.001s
I20260709 13:07:04.743438  4214 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.811245 29071 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:07:04.695995  4254 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.791383  4203 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:04.763252  4287 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:04.829375 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.813740  3940 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:04.828953 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 59172us
I20260709 13:07:04.799207  4105 consensus_queue.cc:1059] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 13:07:04.835798 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:04.815227  3803 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:04.811476  4338 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:07:04.831393  4095 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.242s	user 0.007s	sys 0.019s
I20260709 13:07:04.831817  4236 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:04.833137  3814 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:04.845786  4105 consensus_queue.cc:1243] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:07:04.861596 29066 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:04.856941 29039 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:04.847978  3803 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:04.863447 29065 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:04.846562  4104 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:04.866910 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:04.864706 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:04.867470 29065 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:07:04.869626 29066 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:07:04.867034 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:04.871634 29065 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:07:04.871963 29073 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:04.848997  4357 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:04.852258  4338 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:04.856367  4309 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:04.870019 29069 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:04.857615 29213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:07:04.882157 29213 raft_consensus.cc:606] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:04.882602 29213 raft_consensus.cc:614] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:04.886703  4362 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:04.874025 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:07:04.856518  4095 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.266s	user 0.008s	sys 0.019s
W20260709 13:07:04.862584  4324 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
W20260709 13:07:04.875118  4345 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:07:04.915267 29069 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:04.894924  4323 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:04.883567  4309 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:04.901356 29213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "7e78ffbcc98242068a874a5d9e341477"
 from {username='slave'} at 127.0.0.1:44946
W20260709 13:07:04.877992  4322 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
W20260709 13:07:04.875793  4334 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:07:04.873286  3929 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:04.920351 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46414.2us
I20260709 13:07:04.932691 29213 raft_consensus.cc:606] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Received request to transfer leadership to TS 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:04.941917 29213 raft_consensus.cc:614] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:04.887391  4362 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:04.942615 29039 leader_election.cc:400] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:07:04.875560  3803 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:04.950237 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:04.951287 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3193) took 1988 ms. Trace:
I20260709 13:07:04.845310  4341 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.935823  4309 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:04.956796  4321 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:07:04.967502  4302 consensus_meta.cc:287] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.716s	user 0.010s	sys 0.001s
W20260709 13:07:04.969008  4202 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.940s	user 0.008s	sys 0.005s
W20260709 13:07:04.970021  4073 consensus_meta.cc:287] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.928s	user 0.011s	sys 0.004s
I20260709 13:07:04.970075  3814 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:04.970633  4229 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:04.975173 29069 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:04.977612  4338 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:04.978685  4341 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:04.884114  4357 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:04.978986  4236 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:04.980659  4287 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:04.988056  3929 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:04.953019 29067 rpcz_store.cc:276] 0709 13:07:02.956698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:02.962754 (+  6056us) service_pool.cc:224] Handling call
0709 13:07:03.026727 (+ 63973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:03.027024 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:03.027062 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:03.061942 (+ 34880us) log.cc:844] Serialized 26 byte log entry
0709 13:07:03.266840 (+204898us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:03.266892 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:03.267002 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:04.504845 (+1237843us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:04.931831 (+426986us) raft_consensus.cc:1685] finished
0709 13:07:04.931853 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:04.944776 (+ 12923us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80195328,"thread_start_us":158839,"threads_started":1,"wal-append.queue_time_us":321978}
W20260709 13:07:05.081400 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:05.083896 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 130335us
I20260709 13:07:05.084355 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:04.999892  3814 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:05.020123  4243 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:05.022874 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 49395.9us
I20260709 13:07:05.044874  4265 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:05.046463 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:05.050045 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:05.062440  4302 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:05.082980  4206 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.790s	user 0.012s	sys 0.001s
I20260709 13:07:05.062765 29147 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:05.063548 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:05.064177  4095 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:05.105302  4362 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.106137  4095 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.064282 29142 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:07:05.083806  3842 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.108174  4337 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:05.108367  3842 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.108753  4337 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.110607  3842 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:05.110868  4337 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.111974  4353 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:07:05.112603  4333 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.295s	user 0.000s	sys 0.003s
I20260709 13:07:05.112520  4353 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.115058  4353 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:05.121486 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:05.066133  4279 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:04.981310  3894 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:05.074525 29904 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:05.005856  4202 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.076506  3803 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.017966  4235 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:05.094097 29909 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:05.095387 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:05.062659  4352 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:07:05.101388 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.101779 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:05.068962  4214 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.081993  4263 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.105370  4377 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.149437  3745 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.149991  3745 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:05.150563 29905 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 1219) took 1136 ms. Trace:
I20260709 13:07:05.150956 29905 rpcz_store.cc:276] 0709 13:07:04.014191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.015460 (+  1269us) service_pool.cc:224] Handling call
0709 13:07:04.023832 (+  8372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.491845 (+468013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.150307 (+658462us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54007d0354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:05.150502 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074908032}
I20260709 13:07:05.103601  3929 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:05.152318  4202 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.152637  3929 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.153452  4236 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:05.153945  4319 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:05.153703  4352 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.154105  4236 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.122943 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:05.156701 29902 raft_consensus.cc:2410] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
W20260709 13:07:05.157686 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.083469  3863 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:05.130267 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:05.163684 29904 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:05.098430  3814 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:05.139142 29069 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:05.119855  4357 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.115593  4353 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:07:05.176970  4353 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:05.178822 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:05.154491  4319 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:05.070072  4229 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.187860 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.164227 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.148519 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.169953 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.160544  3863 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:05.174142 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 72441.3us
I20260709 13:07:05.175595 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.175954  4354 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:05.178846  4357 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:05.122259  4206 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.139803  4279 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:05.122033  4265 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.180794  4335 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:05.102648  4243 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.133826  4339 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.104138  4302 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.182273 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.162146  3842 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:05.183300 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:05.087867  4032 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:07:05.160917  3803 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:05.189406 29066 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:05.189615 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:05.190613 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:05.194176 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.192385 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:05.195689 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:05.200062 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:05.190280 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:05.201884 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:05.126116  4337 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:07:05.194545 29222 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:05.191016 29112 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:05.206503 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 58072us
W20260709 13:07:05.208709 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 79059.3us
I20260709 13:07:05.160104  4263 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.212245 29185 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:05.203508  4354 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.221180 29073 raft_consensus.cc:2410] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:07:05.257123  4243 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.257898  4166 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:07:05.205888  4326 consensus_queue.cc:1175] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" responder_term: 22 status { last_received { term: 20 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 20 index: 12 } }
I20260709 13:07:05.260280  3863 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.220849  3842 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.238966  4236 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.209054  4357 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.319159  4309 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.320250  4357 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.213807  4214 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:05.257419 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:07:05.321822 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 64490.6us
I20260709 13:07:05.322574 29064 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:05.207523  4095 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:05.149437  4377 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.234359 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.212724  4340 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:05.266119  4263 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.193464  4244 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.026s
I20260709 13:07:05.326056  4235 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:05.326526  4235 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:05.326882  4235 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:05.239818 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 48862.3us
I20260709 13:07:05.329921 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:05.215760  4332 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 13:07:05.265148  4229 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:05.273195  4265 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.211373  3850 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:05.215590  4339 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:05.239886 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44272.6us
W20260709 13:07:05.285917 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 95706us
I20260709 13:07:05.233665 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.209208  4279 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.378371  4279 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.291345 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3175) took 1134 ms. Trace:
W20260709 13:07:05.298195 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 86019.8us
I20260709 13:07:05.379033 29139 rpcz_store.cc:276] 0709 13:07:04.157153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.157632 (+   479us) service_pool.cc:224] Handling call
0709 13:07:04.172295 (+ 14663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:04.172565 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:07:04.219506 (+ 46941us) log.cc:844] Serialized 14 byte log entry
0709 13:07:05.023760 (+804254us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b18001feb80. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad23a 00007f7836b17a2a
0709 13:07:05.235248 (+211488us) log.cc:844] Serialized 14 byte log entry
0709 13:07:05.290917 (+ 55669us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:05.290988 (+    71us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:07:05.291051 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.291136 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.291277 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51018,"prepare.queue_time_us":53362,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":112,"raft.queue_time_us":55564,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":41122,"spinlock_wait_cycles":3139456,"thread_start_us":194932,"threads_started":1,"wal-append.queue_time_us":1146969}
I20260709 13:07:05.381353 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:05.382221 29067 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:05.246832  4032 consensus_queue.cc:1243] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:07:05.383312 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:05.251205  4104 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:07:05.302657 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 84171.1us
I20260709 13:07:05.281838  4329 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:05.290176  4354 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:05.246080  4319 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.311822 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:05.316730  3863 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.290395  4337 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:05.321234  4309 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.322484  4284 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:05.324821  4320 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:07:05.325024 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 90747.1us
I20260709 13:07:05.325449 29217 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:05.256659  4302 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.252063  4373 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:07:05.330953 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:05.331977 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:05.349695 29143 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 13:07:05.352181 29904 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:05.232798  3803 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.358352 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:05.358695 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:05.366250  4339 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.367408 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3176) took 1210 ms. Trace:
I20260709 13:07:05.392836  4329 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.290709  4166 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:05.289523  4206 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.388278 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:05.442332  4377 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.393230 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:05.443432 29066 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:05.395550 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:07:05.395929 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:05.396733 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 163417us
I20260709 13:07:05.445842 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.405781 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:05.386831  4229 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:05.418035 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:05.418388 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 3360) took 1165 ms. Trace:
I20260709 13:07:05.419075 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:05.421682 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.422991 29146 raft_consensus.cc:2393] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:05.415771  3863 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.387661  4104 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:05.419934 29902 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:07:05.423982 29141 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 16.
I20260709 13:07:05.387076  4341 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:05.373275 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:05.399232  4236 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:07:05.469302 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47699.9us
I20260709 13:07:05.398376  4356 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:05.399803  4279 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:05.410499  4284 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:05.426803 29144 rpcz_store.cc:276] 0709 13:07:04.157440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.157712 (+   272us) service_pool.cc:224] Handling call
0709 13:07:04.187452 (+ 29740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:04.448931 (+261479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:04.448978 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:04.504745 (+ 55767us) log.cc:844] Serialized 26 byte log entry
0709 13:07:04.779421 (+274676us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:04.792962 (+ 13541us) log.cc:844] Serialized 14 byte log entry
0709 13:07:04.871216 (+ 78254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:04.871286 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:05.315283 (+443997us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b1800265b40. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:07:05.367036 (+ 51753us) raft_consensus.cc:1685] finished
0709 13:07:05.367166 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:05.367333 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13651,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":645397,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":44714,"spinlock_wait_cycles":16426752,"thread_start_us":436008,"threads_started":3,"wal-append.queue_time_us":185731}
W20260709 13:07:05.465731 29039 leader_election.cc:336] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.086s (SENT)
I20260709 13:07:05.447180 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.455238 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:05.509253 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 43573.7us
I20260709 13:07:05.415458  4244 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:05.467666 29218 rpcz_store.cc:276] 0709 13:07:04.252738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.274079 (+ 21341us) service_pool.cc:224] Handling call
0709 13:07:05.311606 (+1037527us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b58001b3998. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:07:05.418322 (+106716us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5386112}
I20260709 13:07:05.446812 29070 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:05.425591  4373 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:05.470157 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.443681  4320 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:05.467410 29219 raft_consensus.cc:2368] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:07:05.437305  4329 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:05.423344  4353 consensus_queue.cc:1059] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:07:05.514380 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.528748 29218 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 26.
I20260709 13:07:05.529987 29218 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 13:07:05.516356 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46266.6us
W20260709 13:07:05.478802 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 90584us
I20260709 13:07:05.530874 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.470032  4104 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:05.443279  4337 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:05.520485 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 147293us
I20260709 13:07:05.469497  4355 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:05.566848 29218 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:07:05.426568  4323 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:07:05.468195  4326 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:07:05.535027 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:05.522707  4332 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:05.512217  4339 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.538473  4353 raft_consensus.cc:3055] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:05.581158  4353 raft_consensus.cc:740] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:05.582070  4353 consensus_queue.cc:260] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.583410  4353 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:05.584451  4326 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:05.584841  4326 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:05.585065  3863 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.585731  4326 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.585942  4320 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.586953  4326 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:05.513144  4279 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.589046  3814 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.589360  4339 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.590773  4279 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.591214  3814 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.593150  4284 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.477869  4341 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:05.594581 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59657.8us
I20260709 13:07:05.596267  4328 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:05.572722  4265 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:05.596717  4328 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:05.597154  4328 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:05.515290  4382 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:05.550267 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.550853  4354 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.552901  4252 raft_consensus.cc:3055] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:05.599630  4252 raft_consensus.cc:740] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:05.599386  4382 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.600303  4337 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.600955  4354 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:05.557009  3842 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.477583  4346 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:05.566777 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:05.601747  4382 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.602298  4382 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:05.471853  4229 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:05.559267  4311 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 1.489s	user 0.000s	sys 0.013s
I20260709 13:07:05.448119  4302 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.085s
I20260709 13:07:05.572697 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.620362 29218 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:05.519451  4373 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:05.576730 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:05.579380  4365 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:05.579663  4323 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:07:05.580790  4386 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:07:05.572835  4166 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.600546  4252 consensus_queue.cc:260] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.651935  4252 raft_consensus.cc:3060] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:05.512519  4263 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:05.520435  4244 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.526721 29066 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:07:05.604053 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.530460  4356 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:05.606498 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.608014 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:05.608315 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:05.663043 29908 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:05.608875 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:05.609169 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:05.619001 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:05.568751 29113 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:05.591645  4339 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.622628 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:05.672348 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68382.6us
I20260709 13:07:05.674127 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.674894  4354 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:05.675266  4354 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:05.675668  4354 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:05.675981  3842 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:05.676124  4356 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:05.676476  3842 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:05.676914  3842 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:05.677233  4356 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.622866  4383 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.616021  4346 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:05.680325  4383 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:05.626894  4387 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:07:05.629829  4229 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.595731  4341 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:05.602718  4382 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.649904 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:05.680259  3940 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:07:05.647109 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.650383  3745 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.643793  4323 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:05.684026 29907 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:05.690382  4323 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.691813  4323 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:05.698704 29911 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
W20260709 13:07:05.654044 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 198892us
I20260709 13:07:05.671169  4104 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.701292 29902 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:07:05.701750 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:05.704607 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 135912us
I20260709 13:07:05.707042  4365 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:05.695945  4166 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:05.739748  4235 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.703275  4339 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.739871 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51820 (request call id 1326) took 1485 ms. Trace:
I20260709 13:07:05.755209 29221 rpcz_store.cc:276] 0709 13:07:04.254891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.286741 (+ 31850us) service_pool.cc:224] Handling call
0709 13:07:05.739666 (+1452925us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b5400355cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:05.739802 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51764992}
I20260709 13:07:05.756752 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.744992 29214 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:07:05.761128 29142 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:07:05.764667 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 63034.6us
I20260709 13:07:05.763296  4339 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:05.776947 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.762334  3850 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:05.766325  4365 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:05.783730 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.785074 29111 leader_election.cc:400] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
I20260709 13:07:05.785863 29217 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:05.695389  4354 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.766165  4396 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.798237 29111 leader_election.cc:403] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:07:05.805287 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:05.806618 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.789954  4265 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:05.823227  4244 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.805562  4402 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:05.805691  4355 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
I20260709 13:07:05.810745  4279 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:05.811847 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.839251 29111 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:05.803413  3803 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.821300  4235 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.824896  4104 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.825493  4382 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.826975 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:05.828896  4383 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:05.809175 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.837188 29219 raft_consensus.cc:2410] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:07:05.841837 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.820796  4329 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.843015  4365 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:05.799885  3850 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.856530 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.857491 29221 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:05.859083  4354 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:05.861979 29910 raft_consensus.cc:2410] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
W20260709 13:07:05.863283 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.863897 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.865033  4339 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.865638  4279 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:05.865792  4365 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:05.839043  4402 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.870714  3842 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.871467  4402 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.872828 29220 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:05.845479  4326 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:07:05.873907  3850 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:05.874051  4326 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:05.875013  4326 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:05.875859  4329 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.888599 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.891628 29220 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:05.865638 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.901870 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:05.932724 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.933710 29218 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:05.934872 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:05.938642 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:05.939079 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:05.939284 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:05.940155 29148 raft_consensus.cc:2410] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:07:05.948757 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:05.949045 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:05.949716 29143 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:05.949747 29147 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:05.952315 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:05.953413 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:05.954370 29148 raft_consensus.cc:2393] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:05.957000 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:05.959247 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:05.958014 29141 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:05.961411 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:05.962092 29141 raft_consensus.cc:2368] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20.
W20260709 13:07:05.964310  4380 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.696s	user 0.003s	sys 0.003s
I20260709 13:07:05.960019 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:05.965196  3842 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:05.965451 29140 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:05.966235 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:05.955976  4329 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:05.968432 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.973201 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:05.974597 29218 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 17.
I20260709 13:07:05.975898 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.831979  4396 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:05.982105 29220 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 23: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:05.979970 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.983475 29218 raft_consensus.cc:2393] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 26: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:05.985548 29140 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:05.953945  4365 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:05.880925  4352 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:05.989796 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.993916 29144 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 25. Current term is 26. Ops: [24.19-25.20]
I20260709 13:07:05.994602 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:05.995517 29218 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:06.000764  4403 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.008790  4403 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.010737  4396 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:06.011240  4403 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:06.011528 29141 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:07:06.019871  4241 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:06.020407  4241 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:06.021251  4241 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.022286  4241 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:06.023398 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.024108 29069 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:06.025158 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.026809 29065 raft_consensus.cc:1240] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:07:06.027597 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:06.027782  4017 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 13:07:06.028529 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:06.028889  4368 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:06.029383  4368 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.028591  4017 consensus_queue.cc:1243] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 26.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 13:07:06.029700 29037 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:06.030367 29067 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 17: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:06.030619 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:06.036761 29066 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:06.035342 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:06.025544 29217 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:07:06.039315 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.030992 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:06.031504 29143 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:06.040270 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.042696 29070 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:06.031958 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:07:06.035382 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 92120us
I20260709 13:07:05.879873  4373 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.029958  4339 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:06.050899 29066 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:06.053118 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.029750  4368 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:06.057608 29141 pending_rounds.cc:85] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 16
I20260709 13:07:06.062829 29141 pending_rounds.cc:107] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 13:07:06.063791 29141 raft_consensus.cc:2889] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:06.062834 29220 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:06.047473  4017 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:07:06.069674 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 112190us
I20260709 13:07:06.046162  4347 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:06.072536  4347 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.073102  4347 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:06.073400 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.071180 29147 raft_consensus.cc:2410] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:07:06.077661 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.078637 29142 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:06.052744  4339 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.081868 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:06.089715  4319 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:06.090242 29144 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 16.
I20260709 13:07:06.091289 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.092427 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:06.092875 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.093004 29144 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:06.094030 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.094822 29144 raft_consensus.cc:2410] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:07:06.095570 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.096222  4236 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:06.096436  4328 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:06.097700 29909 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 22: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:06.098212  4339 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:06.063249  4350 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:07:06.103899 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:06.104573 29218 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:06.105611 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:06.105775  4373 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:06.106235 29217 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:06.096681 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:06.096791  4236 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:06.122565 29909 raft_consensus.cc:1217] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 LEADER]: Deduplicated request from leader. Original: 24.19->[25.20-25.20]   Dedup: 25.20->[]
I20260709 13:07:06.128444 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.143186 29140 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:06.113900  4350 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:07:06.135733 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.145572 29142 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:07:06.146559 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.147245 29142 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:06.138370 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54878 (request call id 1210) took 1075 ms. Trace:
I20260709 13:07:06.149466 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.150413 29909 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 13:07:06.142501 29144 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:06.113236  4017 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:06.148067 29906 rpcz_store.cc:276] 0709 13:07:05.063113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.080656 (+ 17543us) service_pool.cc:224] Handling call
0709 13:07:05.081023 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.389276 (+308253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.389306 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.389768 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.533727 (+143959us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:05.533918 (+   191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.134749 (+600831us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b5400682754. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:06.134833 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.134900 (+    67us) raft_consensus.cc:1685] finished
0709 13:07:06.134920 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.138289 (+  3369us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22340,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":327808256,"thread_start_us":250106,"threads_started":2,"wal-append.queue_time_us":235897}
I20260709 13:07:06.171175 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:06.174536 29140 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:06.185415 29144 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:06.206851 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.229050 29146 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:06.215942 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:06.220494 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:06.231105 29113 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:06.231581 29146 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:07:06.232538 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:06.161553  4017 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:06.235697 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:06.244660 29142 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:06.206858  4339 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:06.246529 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:06.249644 29113 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:06.234036  4365 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:06.250924 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:06.264472 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:06.235149  4323 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:06.235339  4326 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:06.247961  4340 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:06.257002  4396 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:06.255228  3745 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:06.270195 29113 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:06.256812  4339 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:06.262775  4356 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:06.286769 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:06.254899  4367 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:07:06.281785 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:06.291378 29217 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:06.281975 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:06.292519 29218 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:06.293816 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:06.294566 29905 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:07:06.296145 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.296869 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.283113 29904 raft_consensus.cc:1275] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 13:07:06.297443 29066 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:07:06.297876 29904 pending_rounds.cc:85] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Aborting all ops after (but not including) 15
I20260709 13:07:06.298309 29904 pending_rounds.cc:107] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 13:07:06.298592 29904 raft_consensus.cc:2889] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:06.285286 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:06.299727 29910 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
W20260709 13:07:06.286547 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.275413  3846 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:06.249348  4402 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:07:06.273304  4396 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.305544 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:06.287665 29911 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:06.306154 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:06.303793 29903 raft_consensus.cc:1240] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:06.274467  4340 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.322005 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:06.306703 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:06.307837 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:06.302798 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:06.304764 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:06.309703 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:06.314922 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:06.315470  4349 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.057s	user 0.003s	sys 0.004s
I20260709 13:07:06.316859  4396 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:06.286684  3850 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:06.333209  3850 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.333699  3850 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:06.308583  4235 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.308832  4032 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:07:06.321825 29220 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:06.278039  4329 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:06.345003 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:06.345214  4329 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.345734  4329 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:06.324231 29147 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:06.349009 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.349303 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3179) took 2191 ms. Trace:
I20260709 13:07:06.349623 29147 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:06.329242 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.330152 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.349677 29145 rpcz_store.cc:276] 0709 13:07:04.158169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:04.211976 (+ 53807us) service_pool.cc:224] Handling call
0709 13:07:04.212377 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:04.377854 (+165477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:04.377884 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:04.485400 (+107516us) log.cc:844] Serialized 26 byte log entry
0709 13:07:04.717505 (+232105us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:04.717556 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.042260 (+324704us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54002e8954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:05.042317 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.284359 (+1242042us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:06.349076 (+ 64717us) raft_consensus.cc:1685] finished
0709 13:07:06.349097 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.349224 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439676032,"thread_start_us":180589,"threads_started":1,"wal-append.queue_time_us":736094}
I20260709 13:07:06.331336 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:06.351553 29071 raft_consensus.cc:2368] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23.
I20260709 13:07:06.332310  3842 consensus_queue.cc:1048] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:06.337803 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.340936 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:06.328337  4340 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:06.287108  4356 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.360581 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.353830 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.356839 29905 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:07:06.351455 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.344324 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:06.356638 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.321659  4202 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:06.357218 29140 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:06.336360  4265 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:06.344136  4319 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:06.384532 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:06.388556 29140 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:06.388708 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44493.2us
I20260709 13:07:06.385560 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:06.390564 29147 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:07:06.392355 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.397508 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:06.398885 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:06.347638  4368 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:06.407106 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:06.413340 29142 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:07:06.358479  3745 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.358302  4369 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:06.414605 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.426985 29144 raft_consensus.cc:2376] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21. Re-sending same reply.
I20260709 13:07:06.415462 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:06.367385  4356 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:06.424283 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:06.350572  4166 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.054s
I20260709 13:07:06.383945  4265 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:06.351228  4235 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.402554  4403 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:06.400892  4319 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:06.408985  4105 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:06.447096 29073 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:07:06.385921  4202 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.457844 29038 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:06.429965  4369 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:06.440371  4265 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:06.461612  4347 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:06.477136 29140 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:07:06.447378  4319 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:06.466920  4284 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:06.480978 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:06.482136 29142 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:06.483975 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:07:06.501091 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 178141us
I20260709 13:07:06.437609  3745 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:06.504978 29909 raft_consensus.cc:2368] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23.
I20260709 13:07:06.505667 29038 leader_election.cc:400] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:07:06.506259 29038 leader_election.cc:403] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:07:06.479782  4415 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:07:06.469041  4328 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:06.511453 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:06.524425 29910 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:06.500077  4403 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:06.531049 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:06.514042  4338 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 16. Current term is 17.
I20260709 13:07:06.499330  3929 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:06.523475  4032 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:06.587072  4235 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:06.588637  4424 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:07:06.588528  4202 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:06.589254 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:06.597890 29217 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
W20260709 13:07:06.631515 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 329873us
I20260709 13:07:06.599011 29218 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
W20260709 13:07:06.606096 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 75134.9us
W20260709 13:07:06.609269 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 302958us
I20260709 13:07:06.632681  4368 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:06.646020 29142 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:06.601138  4032 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.656818  4236 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.663834  4368 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:06.669020  3745 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:06.669520  4241 consensus_meta.cc:287] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.647s	user 0.013s	sys 0.000s
W20260709 13:07:06.671121  4401 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.883s	user 0.004s	sys 0.000s
I20260709 13:07:06.672083  4347 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:06.674608 29147 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 13:07:06.592875  4369 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.686319 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:06.653483 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.687373 29911 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:06.687711 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:06.653652  4032 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:06.581692  4403 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:06.688946 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:06.630905  4367 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:06.748832 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.749935 29064 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 23: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:07:06.691359 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:06.698143 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:06.753873 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.755618 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.654114 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:06.750567 29142 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:06.757606 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.758687 29066 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:06.760026 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.675789  4329 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:06.778095 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 124060us
I20260709 13:07:06.780112 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.780480 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:06.778499 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:06.782030 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.782850 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.701187  4105 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:06.750661  4367 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.796061 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.798728 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1357) took 1441 ms. Trace:
I20260709 13:07:06.798987 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.799146 29139 rpcz_store.cc:276] 0709 13:07:05.357001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.381042 (+ 24041us) service_pool.cc:224] Handling call
0709 13:07:05.381396 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.381785 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.381817 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:05.382295 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:07:05.649079 (+266784us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:07:05.662026 (+ 12947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:05.662644 (+   618us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.670492 (+1007848us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:06.790546 (+120054us) raft_consensus.cc:1685] finished
0709 13:07:06.790570 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.798661 (+  8091us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7798,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":12467,"spinlock_wait_cycles":4193024,"thread_start_us":265123,"threads_started":1,"wal-append.queue_time_us":327927}
I20260709 13:07:06.802891 29143 raft_consensus.cc:2368] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24.
I20260709 13:07:06.802999 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:06.789939  4320 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:06.807428 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:06.808001 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.811626 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.812626 29141 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:07:06.800069  4367 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:06.821646 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:06.823596 29905 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:06.811587  4068 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:06.822759 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:06.828238 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:06.789587  4206 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.20, Next index: 20, Last known committed idx: 19, Time since last communication: 0.022s
I20260709 13:07:06.830061 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3223) took 1500 ms. Trace:
I20260709 13:07:06.808787  4368 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:06.832736 29065 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:07:06.808485  4338 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:06.833556 29216 rpcz_store.cc:276] 0709 13:07:05.329998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.520352 (+190354us) service_pool.cc:224] Handling call
0709 13:07:05.522633 (+  2281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.630516 (+107883us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.630562 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:05.630622 (+    60us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:07:05.630668 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.829785 (+1199117us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540016c954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:06.829842 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.829989 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247524096}
I20260709 13:07:06.831120 29141 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:06.812099  4320 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:06.835567 29112 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:06.836108 29065 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:06.836107 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55691.1us
I20260709 13:07:06.845685 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:06.844247 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 53656.2us
I20260709 13:07:06.848709 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:06.849185 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:06.816824  4356 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.824671  4068 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:06.853935 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:06.853431 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:06.843393  4206 consensus_queue.cc:1243] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 7e78ffbcc98242068a874a5d9e341477 log is divergent from this leader: its last log entry 26.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 13:07:06.827499  4265 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:06.844791  4320 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:06.898238 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44403.9us
I20260709 13:07:06.839532  4326 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:07:06.850103  4396 consensus_meta.cc:287] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.000s
I20260709 13:07:06.906081 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:06.847240  4402 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:06.839288  4340 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:06.884388  3850 consensus_meta.cc:287] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.002s
I20260709 13:07:06.901257  4430 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:06.851256  4073 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:06.928117 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:06.901567  4350 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:06.929169 29141 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:06.886030  4377 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:06.937134 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:06.913477  4326 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:06.906725  4428 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:06.912786  4241 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:06.941776 29145 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:06.922032  4281 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:06.944542 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:06.945241 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 91879.4us
I20260709 13:07:06.925695  4236 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:06.949095 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:06.936203  4350 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:06.955163 29039 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:06.961716 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3268) took 1010 ms. Trace:
I20260709 13:07:06.962329 29148 rpcz_store.cc:276] 0709 13:07:05.950804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.978306 (+ 27502us) service_pool.cc:224] Handling call
0709 13:07:05.978709 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:05.992578 (+ 13869us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:05.992611 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:06.001023 (+  8412us) log.cc:844] Serialized 26 byte log entry
0709 13:07:06.165007 (+163984us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:06.165069 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:06.947502 (+782433us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b540027ced4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:06.947549 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:06.947599 (+    50us) raft_consensus.cc:1685] finished
0709 13:07:06.947613 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:06.961621 (+ 14008us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3682,"raft.queue_time_us":12393,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":22473088,"thread_start_us":156160,"threads_started":1,"wal-append.queue_time_us":377789}
I20260709 13:07:06.915720  4396 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.998052  4396 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:06.998648  4396 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:06.999322  4436 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:06.999974  4436 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:07.000891  4436 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.945925  4428 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:07.004292  4428 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:07.004704  3929 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:07.008050  4368 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:07.008301  4320 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.009833  4032 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.011149 29067 raft_consensus.cc:2368] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 26. Current term is 27.
I20260709 13:07:06.966547 29039 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:07.011953  4032 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:07.012568 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 46158.5us
I20260709 13:07:07.009859 29146 raft_consensus.cc:2393] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 24: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:07:06.968801  4432 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:07:06.971066  4356 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:06.948648  4402 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:06.943486  4435 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:06.930382  3850 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.935240  4430 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:06.977809  4367 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:06.982712  4425 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:07:06.944083  4326 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:06.985709  4403 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:06.993052  4340 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:06.993934  4319 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.995591  4068 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:06.951063  4338 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:06.954142  4241 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:06.965205  4350 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:07.056681  4329 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.028390  4402 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.061208  3850 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.012583  4032 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:07.021471  4435 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:07.062310  4402 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.062544  4435 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:07.065904 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3371) took 1216 ms. Trace:
I20260709 13:07:07.062311  4032 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:07.069211  4032 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:07.071256 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 681152us
I20260709 13:07:07.071758  4426 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.072369 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.072573  4427 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:07.076448  4354 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:07.076973  4354 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:07.077212  4265 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.077456  4354 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:07.077587  4241 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.078325 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:07.042062  4368 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.080444  4368 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.044764 29066 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 25: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:07:07.039825 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 90951.4us
I20260709 13:07:07.055709  4367 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.019400  4377 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:07.040280  4338 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:07.071808  4202 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.053689  4430 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.082772 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:07.072412 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 427770us
I20260709 13:07:07.090768 29218 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:07.092978 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:07.108614 29215 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:07.093918 29213 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:07.109386 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:07.111280 29218 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:07.095649 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:07.112253 29222 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:07:07.093856 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:07.066493 29221 rpcz_store.cc:276] 0709 13:07:05.849396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:05.860504 (+ 11108us) service_pool.cc:224] Handling call
0709 13:07:05.860778 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:05.860970 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:06.041550 (+180580us) log.cc:844] Serialized 14 byte log entry
0709 13:07:06.381648 (+340098us) log.cc:844] Serialized 14 byte log entry
0709 13:07:07.065178 (+683530us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b18004a0f20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:07:07.065394 (+   216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:07.065466 (+    72us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:07.065532 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.065639 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:07.065807 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4853,"prepare.queue_time_us":234979,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":81,"raft.queue_time_us":4943,"raft.run_wall_time_us":15832,"spinlock_wait_cycles":38557952,"thread_start_us":519751,"threads_started":2,"wal-append.queue_time_us":487060}
I20260709 13:07:07.115564 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:07.117735  4376 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:07.117985  4435 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.118460  4376 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:07.119699  4376 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.120555  4435 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.126073  4435 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:07.126775  4435 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.129765  4435 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:07.138195 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 55449us
I20260709 13:07:07.140829 29219 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:07:07.143615 29216 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:07.139776 29111 leader_election.cc:400] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 26. Current term is 27.
I20260709 13:07:07.146216 29111 leader_election.cc:403] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:07.148135  3803 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:07.149745  3842 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:07.150992  3842 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:07.151420  3842 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:07.157835 29217 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:07.164588 29146 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:07.169800 29140 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [17.11-19.12]
I20260709 13:07:07.104164  4338 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:07.165688 29065 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.172530 29185 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:07.174897 29065 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:07.115598  4073 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:07.187541 29215 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 27.
I20260709 13:07:07.188808 29071 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:07:07.189361 29071 pending_rounds.cc:85] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 10
I20260709 13:07:07.189641 29071 pending_rounds.cc:107] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:07:07.189939 29071 raft_consensus.cc:2889] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:07.211344 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:07.213637 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:07.171252  3803 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:07.240502 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:07.242146 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:07.242324  4319 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:07.243462 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:07.244655  4265 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:07.245020  3842 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.246976 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:07.072517  4426 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.187286  4320 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.137424  4202 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.231192  4073 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:07.291013 29143 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:07.208415  4284 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.098s
W20260709 13:07:07.289786  4440 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:07:07.294744  4017 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.317476 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:07.317929 29222 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:07.317543 29215 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:07.306816  4265 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:07:07.319026 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 246731us
W20260709 13:07:07.319628 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 817577us
I20260709 13:07:07.315403 29065 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 13:07:07.321218 29216 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
W20260709 13:07:07.319289 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 143741us
I20260709 13:07:07.263854  4326 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.330443 29188 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.328706 29065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 1188) took 1302 ms. Trace:
I20260709 13:07:07.251603  4428 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.333523 29065 rpcz_store.cc:276] 0709 13:07:06.025940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.027538 (+  1598us) service_pool.cc:224] Handling call
0709 13:07:06.027757 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.832638 (+804881us) spinlock_profiling.cc:243] Waited 9.58 ms on lock 0x7b540005dd20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a84dc0 00007f7836b0f1c0 00007f7836b10231 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c
0709 13:07:07.165252 (+332614us) spinlock_profiling.cc:243] Waited 53 ms on lock 0x7b540005dd20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a8b06f 00007f7836b03f60 00007f7836b057de 00007f7836b0fb49 00007f7836b10416 00007f7836b0cd8f 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62
0709 13:07:07.328119 (+162867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.328650 (+   531us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143921792}
I20260709 13:07:07.331784 29215 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:07.330899 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.333269 29143 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:07.339336 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:07.339962 29215 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:07:07.340404 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:07.340786 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:07.341058 29143 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:07.342519 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:07.343101 29143 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 26.
I20260709 13:07:07.343214 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:07.321862  4265 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:07.344309 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:07.344444 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:07.344901 29221 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:07.352581 29215 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:07:07.327545  3842 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:07.354521 29143 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 25: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:07.355221 29185 leader_election.cc:400] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23.
I20260709 13:07:07.355706 29185 leader_election.cc:403] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
W20260709 13:07:07.333513  4357 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 26 current term: 26 is higher.
I20260709 13:07:07.367513 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:07.368515 29146 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:07.369390 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:07.369721 29146 raft_consensus.cc:1217] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Deduplicated request from leader. Original: 23.18->[24.19-25.20]   Dedup: 25.20->[]
I20260709 13:07:07.363320 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.370348 29146 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 25. Current term is 26. Ops: [24.19-25.20]
I20260709 13:07:07.366225 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:07.371306 29219 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:07.375257 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:07.321382  4017 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.377980 29216 raft_consensus.cc:2393] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 25: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:07.389644 29144 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:07:07.391512 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 62403.7us
I20260709 13:07:07.366416  4402 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.360507  4368 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.362037  4435 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 13:07:07.367202  4456 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:07.361794  4284 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:07:07.353967  4376 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:07:07.396839  4435 consensus_queue.cc:1243] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:07:07.402240 29874 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.368431  3850 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:07:07.378751  4356 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:07:07.404107 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:07.424036  4451 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:07.424578  4451 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.424947 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:07.399413  4458 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23.
I20260709 13:07:07.387359  4445 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:07:07.406311  4428 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.426316  4458 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:07.400853 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:07.407608 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:07.427023  4458 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:07.427474  4202 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:07.427696 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:07.408679 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:07.428079  4202 consensus_queue.cc:1059] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:07.428663  4356 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:07:07.366919  4073 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.429067  4356 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:07.429502 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:07.429906  4356 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.430403 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:07.395447  4350 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.431072  4356 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:07:07.409984 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 48677.1us
I20260709 13:07:07.411908  4068 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:07:07.410728  4456 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:07.412524 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:07.412763  4284 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.413551 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:07:07.416487  4452 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:07:07.417111 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:07.441172 29213 raft_consensus.cc:2376] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25. Re-sending same reply.
I20260709 13:07:07.417248  4017 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.418246  4376 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:07.418641  4032 consensus_queue.cc:1059] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:07.409487  4206 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.426774  4451 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.420073  4236 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.420933  4354 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.425668 29219 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 24: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:07.404721  4426 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.437268  4263 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:07.375111  4446 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 13:07:07.434480 29222 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 23: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:07.437668 29221 raft_consensus.cc:3060] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:07.445582 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 60204.7us
W20260709 13:07:07.448105 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 71406.1us
I20260709 13:07:07.445106  4376 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:07.437223  4445 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:07.432119 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:07.466672 29038 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:07.439814  4329 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:07.397984  4402 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.407930  4368 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:07:07.462239 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:07.429595  4326 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.465231 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3236) took 1170 ms. Trace:
I20260709 13:07:07.465405 29188 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:07.439581  4435 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:07:07.468531 29215 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:07.473270 29073 rpcz_store.cc:276] 0709 13:07:06.295028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.295200 (+   172us) service_pool.cc:224] Handling call
0709 13:07:06.302314 (+  7114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:06.447633 (+145319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.464976 (+1017343us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400187c54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:07.465174 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324341120}
I20260709 13:07:07.458618  4456 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.445477  4284 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.462448  4426 raft_consensus.cc:3055] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:07.481488 29221 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:07.455462  4428 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:07:07.456228  4206 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.467566  4445 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:07.492976 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.457041  4236 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.499003 29067 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:07:07.501220 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.504340 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.486044  4426 raft_consensus.cc:740] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:07.508750 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.514148  4464 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:07.516589 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.495352  4017 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 13:07:07.525493 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 89287.8us
I20260709 13:07:07.501471  4445 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:07.500372  4451 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:07:07.545578 29188 leader_election.cc:400] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24.
I20260709 13:07:07.546073 29188 leader_election.cc:403] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:07.529220 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 62594.4us
I20260709 13:07:07.510555  4462 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:07:07.534178  4326 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:07.488654  4456 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.504693  4466 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:07.547199 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:07.548427 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.505170  3842 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:07:07.490743  4338 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.492751  4469 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
W20260709 13:07:07.556759 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 164193us
W20260709 13:07:07.583287 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.606880 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.607685  4329 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:07.609090  4329 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.614637 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.612841  4466 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:07.625320  4466 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.555145  4350 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:07.618791  4428 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:07.629153  4463 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.505617  4376 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.629156  4446 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.663518 29214 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:07.554692  4326 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:07.673465 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 126368us
I20260709 13:07:07.655485  4471 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:07.687417 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 72897.4us
I20260709 13:07:07.612985  4462 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:07.633306  4263 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.659833  3850 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:07.660244  3842 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24.
I20260709 13:07:07.683243 29038 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:07.660796  4428 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:07.720212 29221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:07:07.720515  4428 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:07.720861 29221 raft_consensus.cc:606] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.721205 29221 raft_consensus.cc:614] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:07.700354 29908 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:07.725283  4206 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.701474 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:07.726027  4206 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:07.663749  4350 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:07.726490  4206 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:07.663380  4465 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:07.715582  4467 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:07:07.686199  4017 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 27 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:07.728266  4476 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:07:07.732551  4478 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.733174  4017 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.720001  3842 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:07.748347  3842 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.750818  3842 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.733325  4478 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.719051  3803 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 26. Current term is 27.
I20260709 13:07:07.736301 29221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:07:07.754665  4428 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:07.762287 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 300123us
I20260709 13:07:07.763754 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:07.681926  4473 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:07.690626  4464 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.699251  4376 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:07.684082  4451 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:07.771117 29139 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 27.
I20260709 13:07:07.654758  4426 consensus_queue.cc:260] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:07.760998 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59365.2us
I20260709 13:07:07.681298  4461 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 13:07:07.879289  4347 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 13:07:07.711859  4354 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:07.766197  4402 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.872073  4236 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.873309  4456 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:07.873119  4484 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:07.874284  4166 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:07.743687  4263 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:07.878376  4265 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.002s
I20260709 13:07:07.885294  4482 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:07.686201  4446 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.890599  4428 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.666906  4463 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.727532  4206 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.893422  4478 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:07.893503  3803 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.896561  4326 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.890658  4451 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.769305  4323 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:07.813968  4473 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.686264  4474 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:07.735337  4469 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:07.854681  4284 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:07.921741 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 376323us
W20260709 13:07:07.924676 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 376307us
W20260709 13:07:07.928756 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 170742us
I20260709 13:07:07.691987  4471 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:07.948719 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 185055us
I20260709 13:07:07.923026  4402 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:07.933890  4482 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:07:07.969646 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:07.950629  4323 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:07.973320 29874 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:07.980019  4402 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.981022  4350 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:07.981580  4350 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:07.981695 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:07.982132  4402 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.982923 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:07.936029  4265 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.984898 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.974215 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:07.976939 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:07.980671  4323 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.963842  4206 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:07.983254  4347 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:07.983554  4482 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:07.985317  4486 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:07.968165  4338 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:07.950421  3803 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:07.992084 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:07.986560  4489 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:07.973344  4284 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:07.995805 29907 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:07.871294  4464 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:07.998654 29221 raft_consensus.cc:606] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:08.006523 29221 raft_consensus.cc:614] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:07.998865 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1351) took 1921 ms. Trace:
I20260709 13:07:08.007094 29220 rpcz_store.cc:276] 0709 13:07:06.078073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.078384 (+   311us) service_pool.cc:224] Handling call
0709 13:07:06.078783 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.343734 (+264951us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:07.998590 (+1654856us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54000cc6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:07.998795 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558122240}
I20260709 13:07:07.982052  4350 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:07.973140  4469 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:08.000447 29905 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 1250) took 1600 ms (client timeout 1869 ms). Trace:
W20260709 13:07:08.011448 29905 rpcz_store.cc:269] 0709 13:07:06.395662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.396329 (+   667us) service_pool.cc:224] Handling call
0709 13:07:07.995416 (+1599087us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b540063d654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:07.995545 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":841709440}
I20260709 13:07:07.995035  4357 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:08.013128 29144 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:07:07.990427  4252 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.021s
I20260709 13:07:08.014103  4357 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.040536  4357 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:08.011073  4486 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:08.044682  4469 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.044948  4486 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.040542  4490 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:08.047665  4490 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:08.047756  4263 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.048759  4329 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.049330  4329 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:08.013643  4356 consensus_meta.cc:287] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.582s	user 0.016s	sys 0.002s
I20260709 13:07:08.057698 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.060011 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.060052 29220 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:08.061383 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.060735 29214 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:08.062162 29222 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:08.064646 29218 raft_consensus.cc:2393] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:08.068909 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:08.069693  4446 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.071403  4486 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:08.072548  4493 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:07:08.073076  3850 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:08.073542  3850 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:08.074172  4473 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:08.074363  4357 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.075842  4464 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:08.077808 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.009104  4284 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.092270 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.012714  4495 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:07:08.093247 29218 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
W20260709 13:07:08.081065  4472 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:07:08.100809 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:08.007893  4489 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.080415  4462 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:08.105038 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:08.109624 29216 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:07:08.110016 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 117984us
I20260709 13:07:08.105867  4326 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.010s
I20260709 13:07:08.114389  4326 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:08.114826  4326 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.115170  4326 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:08.108292  4356 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:08.110939 29874 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:08.124689 29066 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: []
W20260709 13:07:08.126034 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 132864us
I20260709 13:07:08.130338  4471 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:08.132522 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:08.133818  4105 raft_consensus.cc:2804] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:08.134426  4105 raft_consensus.cc:697] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:08.135377  4105 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.120916  4356 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.122373  4329 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.146699 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:08.148874  3803 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.119150  4463 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:08.153720 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:08.154605 29066 raft_consensus.cc:2410] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:07:08.159243  4436 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:08.159780  4436 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:08.160764  4436 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:08.161530  4445 consensus_meta.cc:287] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.606s	user 0.006s	sys 0.004s
I20260709 13:07:08.161893  4436 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:08.162010  4445 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.164858  4445 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:08.150619  4356 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:08.169111  4252 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:08.169840  4482 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.146519  4265 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:08.174811 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:07:08.180173 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 75215.7us
W20260709 13:07:08.205575 29068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40862 (request call id 1183) took 2158 ms (client timeout 2091 ms). Trace:
W20260709 13:07:08.206049 29068 rpcz_store.cc:269] 0709 13:07:06.024224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.030304 (+  6080us) service_pool.cc:224] Handling call
0709 13:07:08.182646 (+2152342us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54002ab054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:08.182853 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2630591872}
I20260709 13:07:08.159529  3850 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.209939 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:07:08.218043 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.218590 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:08.226176 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:08.231118 29141 raft_consensus.cc:2393] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:08.167327  3803 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:08.237453  3803 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.181043  4350 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.201292  4357 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.240708  4350 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.241461 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:08.242733  3803 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:08.245553  4420 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 1.762s	user 0.008s	sys 0.003s
I20260709 13:07:08.245175  4326 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.254827 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.255014 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:08.255752 29141 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:08.255776 29222 raft_consensus.cc:2410] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:07:08.246660  4489 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.263499  3850 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:08.260531  4445 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:07:08.270238 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:08.271982  4337 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:08.272444  4337 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:08.273411  4337 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.274555  4337 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:08.257824  4377 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:08.276549  4377 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.277889  4396 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:08.278298  4396 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:08.278786  4377 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:08.279175  4396 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.282220 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1365) took 1477 ms. Trace:
I20260709 13:07:08.283087  4323 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:08.259173  4284 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:08.282637 29142 rpcz_store.cc:276] 0709 13:07:06.804396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.986796 (+182400us) service_pool.cc:224] Handling call
0709 13:07:06.987177 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.063032 (+ 75855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:07.063062 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:07.063104 (+    42us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:07.063135 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.281962 (+1218827us) spinlock_profiling.cc:243] Waited 75.6 ms on lock 0x7b5400342454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:08.282014 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.282151 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173972736}
I20260709 13:07:08.257513 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:08.292984 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:08.296519 29141 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:08.306912 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:08.305673 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 3158) took 1528 ms. Trace:
I20260709 13:07:08.323014 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.311564 29186 leader_election.cc:400] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23.
I20260709 13:07:08.325531 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:08.322329 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:08.322728 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.331537 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.332441 29222 raft_consensus.cc:2410] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:07:08.304652  4489 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.304457  4329 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:08.304889  4430 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:08.333771 29069 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 26. Current term is 27. Ops: []
I20260709 13:07:08.302596  4445 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:08.333725  4501 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:08.347172  4445 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.347796  4501 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.351068  4501 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:08.351411 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.333884 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:08.354856 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:08.355465 29143 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:08.335705 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:08.357225 29213 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:08.331621 29141 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:08.337122 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:08.359014 29220 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:08.335039 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.338516 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.361241 29216 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:08.343825 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1367) took 1534 ms. Trace:
I20260709 13:07:08.362346 29066 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:08.344223 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.344748 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:08.366542 29148 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:08.367504 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:08.345422 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.304396  4396 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:08.336879 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.384526 29066 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:08.339197 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:08.385378 29144 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:08.385703 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.386291 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:08.386997 29144 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:08.340749 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:08.362739 29147 rpcz_store.cc:276] 0709 13:07:06.809367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.022067 (+212700us) service_pool.cc:224] Handling call
0709 13:07:07.022455 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.039819 (+ 17364us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:07:07.363072 (+323253us) log.cc:844] Serialized 14 byte log entry
0709 13:07:07.657317 (+294245us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:07.657390 (+    73us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:07:07.657449 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.343580 (+686131us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b54001bd0d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:08.343631 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.343764 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41328,"prepare.queue_time_us":148167,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":18435,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":4559,"spinlock_wait_cycles":38813568,"thread_start_us":551081,"threads_started":3,"wal-append.queue_time_us":161386}
I20260709 13:07:08.392968 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:08.393989 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:08.395982 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:08.397122 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:08.397670 29141 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:08.329402 29064 rpcz_store.cc:276] 0709 13:07:06.777651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.895665 (+118014us) service_pool.cc:224] Handling call
0709 13:07:06.896154 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:06.950133 (+ 53979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:06.950164 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:06.950207 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:06.950240 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.305341 (+1355101us) spinlock_profiling.cc:243] Waited 53.7 ms on lock 0x7b5400068bd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:08.305408 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:08.305595 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123520896}
I20260709 13:07:08.463840 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:08.399796 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:08.404347 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:08.319734  4471 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:08.409171 29147 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:08.309060  4284 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.413795 29217 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:07:08.337760  4329 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.416337 29213 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:08.399116  4362 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:08.409507  3803 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.448961 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:08.422221 29065 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:08.424307 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:08.575855 29069 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:08.424980 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:08.428370 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.421523 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.442777 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.428630 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:08.424670 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.424307 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:08.464649  4476 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:08.367583  4373 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:08.470919 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47046.5us
I20260709 13:07:08.583225 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.583663 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:08.497723 29143 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:08.344127  4347 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:08.340065  4482 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.398532 29220 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:08.499586 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:08.518296 29217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 3390) took 1178 ms (client timeout 1556 ms). Trace:
I20260709 13:07:08.499318 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:08.499498 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 386331us
W20260709 13:07:08.599470 29217 rpcz_store.cc:269] 0709 13:07:07.339844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.340496 (+1000652us) service_pool.cc:224] Handling call
0709 13:07:08.518227 (+177731us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:08.518800 29144 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:08.520299 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:08.526546 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 407237us
I20260709 13:07:08.650224 29147 raft_consensus.cc:2393] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:08.548079 29146 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
W20260709 13:07:08.523027 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.652451 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:08.343215  4456 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:07:08.570199 29145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 3211) took 2209 ms (client timeout 1551 ms). Trace:
I20260709 13:07:08.431150 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:08.688347 29145 rpcz_store.cc:269] 0709 13:07:06.338279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.351217 (+ 12938us) service_pool.cc:224] Handling call
0709 13:07:08.546763 (+2195546us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b540029d454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:08.547548 (+   785us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1332944768}
I20260709 13:07:08.689657 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.690099 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:08.691437  4456 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:08.578759 29909 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:08.337373  4489 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.699416 29142 raft_consensus.cc:2393] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:08.700228 29908 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:08.700573  4489 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:08.695983 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 47910.9us
I20260709 13:07:08.424625 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:08.580693 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:08.581136 29904 raft_consensus.cc:2376] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22. Re-sending same reply.
I20260709 13:07:08.496176  4017 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:08.585199 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:08.706233 29070 raft_consensus.cc:2368] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22.
I20260709 13:07:08.706360 29143 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:08.338043  4430 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.586930 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.712502 29911 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:08.420631  4365 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:08.590600 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.713598 29220 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:08.591220 29216 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:08.592350 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:08.571301  4323 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:07:08.621542 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.632417 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:08.545601  4471 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.545915  4284 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.591188  4373 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.580201  4362 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.730248  4362 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:08.731256 29909 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:07:08.732264 29906 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:07:08.732431 29148 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:08.723701  4323 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.736153  4323 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:08.738281  4284 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.647153 29141 raft_consensus.cc:2393] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:07:08.647676 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64115.8us
W20260709 13:07:08.500900 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:08.650895 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:08.568269 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:08.748034 29148 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:08.653810 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.655810  3850 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:08.657104  4340 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:07:08.657200  4515 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:07:08.658267 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:08.660710  4513 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:07:08.582351  4476 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.582710  3803 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.568940  4356 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.668907  4326 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.671334 29907 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54902 (request call id 1234) took 1987 ms. Trace:
I20260709 13:07:08.676362  4329 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.772763 29907 rpcz_store.cc:276] 0709 13:07:06.684252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.684371 (+   119us) service_pool.cc:224] Handling call
0709 13:07:07.995637 (+1311266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.006234 (+ 10597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.671129 (+664895us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540063d650. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:08.671270 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3014909312}
I20260709 13:07:08.409523  4377 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:08.691469 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.695029 29144 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 27.
I20260709 13:07:08.698364 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:08.724009 29143 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:08.721210 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:08.728330  4373 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:08.782335 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:08.765820 29147 raft_consensus.cc:2410] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:07:08.761384 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:08.771654 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:08.768033 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:08.797822 29144 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:08.798549 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:08.799794 29188 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:08.801447 29141 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:08.803790 29141 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:08.759310 29217 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:07:08.741626  4473 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:08.760509  3850 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:08.790771 29143 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:07:08.792413 29222 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:07:08.810019 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:08.780541  4377 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.778090  4458 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:08.723791  4365 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.723510  4347 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 13:07:08.719595  4017 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.806900 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:08.805902 29144 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:08.823529 29143 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 13:07:08.770012  4340 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.813946  3846 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:08.814777  3850 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:08.843357 29145 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:07:08.819161  4329 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.813726  4353 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:08.813510  4265 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:07:08.832410  4337 consensus_meta.cc:287] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.557s	user 0.015s	sys 0.000s
I20260709 13:07:08.710227  4456 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.834025  4347 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:08.835466  3803 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.836426  4017 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.838936 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3301) took 1664 ms. Trace:
I20260709 13:07:08.817502  4430 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:08.844538 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:08.820120  4105 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:08.822965  4377 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:08.829066 29143 pending_rounds.cc:85] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 14
I20260709 13:07:08.848800  4491 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:08.849916  4073 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:08.850283 29222 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:08.891254 29143 pending_rounds.cc:107] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:07:08.891621 29143 raft_consensus.cc:2889] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:08.876374 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47960 (request call id 3159) took 1717 ms. Trace:
I20260709 13:07:08.892131 29067 rpcz_store.cc:276] 0709 13:07:07.159319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.159470 (+   151us) service_pool.cc:224] Handling call
0709 13:07:07.176461 (+ 16991us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:07.503268 (+326807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.876008 (+1372740us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54000681d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:08.876270 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741141376}
I20260709 13:07:08.879144 29146 rpcz_store.cc:276] 0709 13:07:07.174920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.371316 (+196396us) service_pool.cc:224] Handling call
0709 13:07:08.547693 (+1176377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:08.650875 (+103182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.838700 (+187825us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540029d450. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:08.838842 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2704716672}
W20260709 13:07:08.894203 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 49769.7us
I20260709 13:07:08.841089  4471 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:08.894347 29146 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:08.899384 29146 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:08.872720  4265 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.854516  4353 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:08.889521 29039 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:08.829205  4458 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:08.910565 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:08.911322 29112 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.132:36041 timed out after 1.725s (SENT) [suppressed 22 similar messages]
I20260709 13:07:08.910394 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:08.913532 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:08.895236 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:08.874208  4347 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:08.895121  4073 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:08.893087  4456 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:08.905373  4265 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:08.893613  4017 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:08.855628  4329 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.925364 29039 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:08.901851  4339 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:08.927672 29222 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:08.931113  4073 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:08.932613  4073 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:08.932828  4458 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:08.909543  4353 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:08.920251  4373 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:08.921183  4337 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:08.928785  4362 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:07:08.923467  4284 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:07:08.938735  4510 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:07:08.936815  4481 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:08.901093  4491 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.931137  4465 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:08.906970 29146 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:07:08.944916  4487 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:08.945917  4373 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:08.950088  4284 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:08.950605  4340 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:08.937575  4017 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:08.952970  4337 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:08.952734  4476 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:09.007464 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 242283us
I20260709 13:07:08.963207  4362 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 27 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:09.013371 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.005599  4284 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:08.966132  4481 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:09.020121 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:09.020962 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:09.021524 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:09.021888 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:09.022799 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.024346 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.035815 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.036725 29071 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:09.039347 29071 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:09.043227 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:09.045972 29071 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:07:09.051126 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.051676 29222 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:09.052362 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:09.054977 29064 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:08.971127  4465 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.059142 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:09.007189  3929 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.043s
I20260709 13:07:09.006959  4487 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.030458  4365 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:09.013913  4482 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
W20260709 13:07:09.062779 29113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.010s	sys 0.022s
W20260709 13:07:09.082927 29113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.010s	sys 0.022s
W20260709 13:07:09.083334 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 619539us
I20260709 13:07:09.069244 29069 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:07:09.070112 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 348951us
I20260709 13:07:09.014145  4429 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:07:09.078259 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.084396 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:09.092355 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:09.002892  4347 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:09.063526  4519 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:07:09.089792 29871 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:09.090358  3846 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:09.097148 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:07:09.105545 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 194233us
I20260709 13:07:09.108471 29113 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:09.108946 29064 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:07:09.088261  4252 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:09.094936  4458 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:07:09.109601  4518 log.cc:927] Time spent T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
W20260709 13:07:09.128351 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.120589  4471 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:09.107659  4523 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:07:09.151027 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 91990.2us
I20260709 13:07:09.127830  4458 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:09.154170 29186 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:09.156188 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3295) took 1987 ms. Trace:
I20260709 13:07:09.117213  4524 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:09.156834 29140 rpcz_store.cc:276] 0709 13:07:07.169270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.155035 (+1985765us) service_pool.cc:224] Handling call
0709 13:07:09.155484 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:09.155731 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:09.155772 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:09.155831 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:09.155876 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.155968 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.156112 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:09.130654  4105 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:09.130649  4365 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:09.130743  4530 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:07:09.191095 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.163249 29140 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:09.096678  4373 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.173043  4252 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:09.175231  4521 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:07:09.106954  4337 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:09.178704 29139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 3302) took 2003 ms (client timeout 2132 ms). Trace:
W20260709 13:07:09.210448 29139 rpcz_store.cc:269] 0709 13:07:07.175223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.213305 (+ 38082us) service_pool.cc:224] Handling call
0709 13:07:09.178476 (+1965171us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b54003cc3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:09.178625 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1237574016}
I20260709 13:07:09.213210 29140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c"
dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:39218
I20260709 13:07:09.213915 29140 raft_consensus.cc:606] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:09.214311 29140 raft_consensus.cc:614] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:09.166565  4471 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:07:09.197890 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 108166us
I20260709 13:07:09.108418  4362 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.161242  4430 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:09.162434  4458 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:09.159245  4329 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:09.192759  4319 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.813s	user 0.012s	sys 0.008s
I20260709 13:07:09.192482 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:09.217123 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:09.219434  4487 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:09.167723  4523 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:09.186213  4524 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:07:09.226187 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.227933  4426 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:09.229678  4369 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:09.235049 29139 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [17.11-19.12]
W20260709 13:07:09.245589  4525 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:07:09.251509 29907 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:07:09.251777 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.252156 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.253412 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.313086 29911 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:07:09.254009  4535 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:09.255100 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:09.255020  4529 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:09.124116  4481 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.261449 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.214421  4530 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:09.215341  4402 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:09.136720  4543 consensus_queue.cc:1059] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:07:09.187290  4365 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:09.207388  4482 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.056s
W20260709 13:07:09.281047 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 88649.8us
W20260709 13:07:09.282471 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 130264us
W20260709 13:07:09.280450 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 126373us
W20260709 13:07:09.334983 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.262068  4017 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:09.213754  4353 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.281386 29185 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:09.183287  3846 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.292960 29904 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:09.356729 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.358541 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.301635 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.304507 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 1191) took 2893 ms. Trace:
W20260709 13:07:09.305092 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 88056us
I20260709 13:07:09.264539  4465 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:09.360760 29112 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:09.283573  4367 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:09.360255 29072 rpcz_store.cc:276] 0709 13:07:06.390732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:06.398311 (+  7579us) service_pool.cc:224] Handling call
0709 13:07:06.398731 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:06.399162 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:06.399204 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:07.809586 (+1410382us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b180041f980. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a
0709 13:07:08.450192 (+640606us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b2c000014c8. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7835406a11 00007f7835416372 00007f7836b48a01 00007f7836b0d43a 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2
0709 13:07:08.450776 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 13:07:08.785866 (+335090us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:08.785931 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:08.787542 (+  1611us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.284187 (+496645us) raft_consensus.cc:1685] finished
0709 13:07:09.284212 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.284364 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":469,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":9478,"spinlock_wait_cycles":14175488,"thread_start_us":334648,"threads_started":1,"wal-append.queue_time_us":515482}
I20260709 13:07:09.361815  4507 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:09.362205 29112 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:09.337698 29037 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:09.311486  4329 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:09.279096  4319 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.903s	user 0.012s	sys 0.008s
I20260709 13:07:09.362850  4507 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:07:09.343518  4542 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:07:09.372089 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:09.373531 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 95191.5us
I20260709 13:07:09.373652 29139 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:07:09.378940 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:09.380244 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.380584 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.380772 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.389075 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.391369 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.393304 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.346800  4487 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.325119  4535 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.402235 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.319664  4538 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:07:09.330593  4529 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:09.408321 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.332408  4402 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.373782  4476 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:09.404364 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.409996 29064 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 23: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:09.282254  4430 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.375892  4429 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:09.411675 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:09.410920  4553 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.377455  3803 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.381932  4465 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:09.345813  4502 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:07:09.320941  4471 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.346927  4265 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:09.398054  4456 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.442354 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.374864  4522 consensus_queue.cc:1059] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:07:09.440873 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.376621  4548 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:07:09.445412 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3314) took 1308 ms. Trace:
I20260709 13:07:09.421773  4476 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:09.453926 29143 rpcz_store.cc:276] 0709 13:07:08.137161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.828336 (+691175us) service_pool.cc:224] Handling call
0709 13:07:08.828779 (+   443us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:08.903847 (+ 75068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:08.903895 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:08.981730 (+ 77835us) log.cc:844] Serialized 52 byte log entry
0709 13:07:09.101844 (+120114us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:09.101971 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.102107 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.445063 (+342956us) raft_consensus.cc:1685] finished
0709 13:07:09.445097 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.445331 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10279,"raft.queue_time_us":13993,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":6840,"thread_start_us":98919,"threads_started":1,"wal-append.queue_time_us":201046}
I20260709 13:07:09.458585 29143 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:07:09.460937 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 310850us
I20260709 13:07:09.409320  4487 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.449893 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:09.428516  4435 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.933s	user 0.033s	sys 0.027s
I20260709 13:07:09.451823 29069 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 21: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:09.441263  4555 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:09.454321 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:09.423950 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.464323 29067 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 13:07:09.461225  4476 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:09.362779  4523 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:09.467366 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:09.469064  4353 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:09.473623 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 138039us
W20260709 13:07:09.461715  4536 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:07:09.481184 29186 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:09.481827 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.482229 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:09.469663  4435 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.972s	user 0.033s	sys 0.028s
I20260709 13:07:09.411700  4524 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:09.489384 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.462734  4538 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:09.451202  4456 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.469476  4529 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:09.504364 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.516136 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3230) took 1546 ms. Trace:
I20260709 13:07:09.443603  3803 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.518000 29147 rpcz_store.cc:276] 0709 13:07:07.970159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.797491 (+827332us) service_pool.cc:224] Handling call
0709 13:07:08.819940 (+ 22449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.852859 (+ 32919us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:07:08.877156 (+ 24297us) log.cc:844] Serialized 14 byte log entry
0709 13:07:09.096787 (+219631us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:09.096848 (+    61us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:07:09.096910 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.515822 (+418912us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b54002e8954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:09.515876 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.516060 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16487,"prepare.queue_time_us":25733,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":16623,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":2642,"spinlock_wait_cycles":75041024,"thread_start_us":202688,"threads_started":1,"wal-append.queue_time_us":370636}
I20260709 13:07:09.502228 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:09.441280  4553 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.460965  4465 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:07:09.462483  4284 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:09.498416  4545 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:07:09.497172  4546 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:07:09.523766 29215 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 3253) took 2123 ms (client timeout 2447 ms). Trace:
I20260709 13:07:09.523947  4365 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:09.418809  4530 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:09.452577  4507 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:09.536936 29215 rpcz_store.cc:269] 0709 13:07:07.400468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:07.400599 (+   131us) service_pool.cc:224] Handling call
0709 13:07:09.523613 (+2123014us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b540026bbd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:09.523712 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95177472}
I20260709 13:07:09.471972  4549 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:07:09.546902 29073 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:07:09.567991 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 43623.7us
I20260709 13:07:09.536861  4546 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:09.569084  4546 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:09.508021  4554 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:07:09.542330  4538 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:07:09.582741  4557 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.081s	user 0.001s	sys 0.001s
I20260709 13:07:09.592679 29070 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:09.602741  4543 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:09.603358  4543 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:09.604092  4565 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:09.604451  4543 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:09.605978  4556 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:07:09.607790 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3326) took 1238 ms. Trace:
W20260709 13:07:09.608309  4559 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:07:09.608279 29140 rpcz_store.cc:276] 0709 13:07:08.369057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.218392 (+849335us) service_pool.cc:224] Handling call
0709 13:07:09.218828 (+   436us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:09.219253 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:09.219292 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:09.219988 (+   696us) log.cc:844] Serialized 52 byte log entry
0709 13:07:09.415807 (+195819us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:07:09.415864 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:09.415979 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:09.607465 (+191486us) raft_consensus.cc:1685] finished
0709 13:07:09.607494 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:09.607716 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2765,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":195360,"threads_started":1,"wal-append.queue_time_us":237851}
I20260709 13:07:09.610090  4570 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:09.610795  4570 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:09.611311  4570 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:09.612653  4487 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:09.613574  4564 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:07:09.616456  4265 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:09.585359  4402 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:09.616855  4538 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.615579  4535 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:09.655555 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 62821.5us
W20260709 13:07:09.653590 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 171399us
I20260709 13:07:09.657552  4487 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:09.664230 29185 leader_election.cc:400] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22.
I20260709 13:07:09.669720  4456 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:09.672389  4073 consensus_meta.cc:287] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.739s	user 0.010s	sys 0.005s
I20260709 13:07:09.675184 29065 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
W20260709 13:07:09.662634  4557 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:07:09.675709 29065 pending_rounds.cc:85] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 12
I20260709 13:07:09.674471  4356 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:09.680713  4377 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.793s	user 0.004s	sys 0.009s
I20260709 13:07:09.650763  4430 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:09.681766 29065 pending_rounds.cc:107] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:07:09.669304 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:09.691968 29065 raft_consensus.cc:2889] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:09.698067  4252 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:09.677014  4284 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:09.725476  4548 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:07:09.699118  4377 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.720350  3846 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:07:09.725800  4570 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.733650 29908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54878 (request call id 1234) took 1408 ms. Trace:
I20260709 13:07:09.734053 29908 rpcz_store.cc:276] 0709 13:07:08.325082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.351185 (+ 26103us) service_pool.cc:224] Handling call
0709 13:07:09.733488 (+1382303us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b5400682754. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:09.733594 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":800457600}
I20260709 13:07:09.725968  4548 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:09.735318 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.735824  4548 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.738134  4548 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:09.727263  3846 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:09.739187  3846 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:09.739264 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.730906  4202 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.740157  3846 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.740496  4202 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:09.741180 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.741611  3846 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:09.728025  4377 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:09.742470 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:09.743028 29909 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:07:09.744057 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.744663  3803 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:09.727551 29071 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 23: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:09.704661  4553 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:09.668757  4487 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:09.745676 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:09.746235 29902 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:09.747119  4487 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:09.667261  4535 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:07:09.748576  4535 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:09.749064 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:09.749629 29068 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:07:09.751610 29186 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:09.757061 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.716748  4456 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:09.923694  4574 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.924182  4202 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:09.924384  4574 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:09.768246  4570 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:09.928174  4574 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:09.751113  4206 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.929030  4206 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:09.916656 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 195136us
I20260709 13:07:09.960263 29185 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:09.919212 29902 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:09.916891 29909 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:09.749075  4535 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:09.898118  4263 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.897850  4553 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:09.898582  3803 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:09.971114 29185 connection.cc:256] RPC call timeout handler was delayed by -1.0546330039996974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
I20260709 13:07:09.916540  4426 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:09.971308 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.919605  4377 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.969225  4579 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:09.974710  4565 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:07:09.968398 29909 raft_consensus.cc:1240] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:09.967674  4450 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:07:10.020680 29215 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9574de46fcd490e84208e41907200b0"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:55420
I20260709 13:07:10.022478 29215 raft_consensus.cc:606] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Received request to transfer leadership
I20260709 13:07:10.025686 29215 raft_consensus.cc:614] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:07:10.026527 29145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 1373) took 1713 ms (client timeout 2208 ms). Trace:
W20260709 13:07:10.027086 29145 rpcz_store.cc:269] 0709 13:07:08.313366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.689764 (+376398us) service_pool.cc:224] Handling call
0709 13:07:10.026262 (+1336498us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400144e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:10.026432 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350205184}
I20260709 13:07:10.011102  4579 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:10.034053 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:09.987736  4263 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:10.025051  3850 consensus_meta.cc:287] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.169s	user 0.017s	sys 0.006s
W20260709 13:07:10.022692  4458 consensus_meta.cc:287] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.746s	user 0.010s	sys 0.005s
I20260709 13:07:09.996484  4426 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.010265  4369 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:09.995297  3803 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:09.999312  4377 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.024685  4565 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:10.050961 29906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 1262) took 1628 ms. Trace:
I20260709 13:07:10.051622  3803 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.053526 29906 rpcz_store.cc:276] 0709 13:07:08.422799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.588470 (+165671us) service_pool.cc:224] Handling call
0709 13:07:08.731831 (+143361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.732746 (+   915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.050723 (+1317977us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54007c01d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:10.050889 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328946432}
I20260709 13:07:10.022512  4546 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.036803  4535 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.046800  4458 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.066304  4491 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.065762  4565 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.072376  4458 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:10.072796  4547 raft_consensus.cc:3055] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:10.082858  4547 raft_consensus.cc:740] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:10.083870  4546 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:10.084317  4547 consensus_queue.cc:260] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.085778  4579 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:10.079463  4476 consensus_meta.cc:287] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.575s	user 0.013s	sys 0.002s
I20260709 13:07:10.086478  4547 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:10.086661  4476 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:10.078891  4582 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:07:10.081341  4475 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:10.090256  4476 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.090471  4475 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:10.092069  4475 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.094048  4475 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:10.099049  4535 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:10.054193  4369 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.046010  3850 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.058524  4263 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:10.152035  4369 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:10.152619  3803 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.153854 29142 raft_consensus.cc:2393] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 26: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:10.153863  3803 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.158766  4426 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:10.158949  4377 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.151790  3850 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:10.216173 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:10.229803 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 82375.7us
I20260709 13:07:10.236899  3803 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:10.253783 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.253944  4206 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:10.255266 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:10.256222 29068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 3247) took 2001 ms (client timeout 1721 ms). Trace:
W20260709 13:07:10.294976 29068 rpcz_store.cc:269] 0709 13:07:08.254600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.254737 (+   137us) service_pool.cc:224] Handling call
0709 13:07:10.255223 (+2000486us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b5400185e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:10.255713 (+   490us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3435741952}
W20260709 13:07:10.296999 29073 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 3263) took 1249 ms (client timeout 1659 ms). Trace:
I20260709 13:07:10.285480 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 3244) took 2206 ms. Trace:
W20260709 13:07:10.297545 29073 rpcz_store.cc:269] 0709 13:07:09.047196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.049518 (+  2322us) service_pool.cc:224] Handling call
0709 13:07:10.296487 (+1246969us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b540012a054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:10.296786 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132291840}
I20260709 13:07:10.300035 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51820 (request call id 1360) took 1874 ms. Trace:
I20260709 13:07:10.300604 29216 rpcz_store.cc:276] 0709 13:07:08.425590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.428029 (+  2439us) service_pool.cc:224] Handling call
0709 13:07:08.591043 (+163014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.718307 (+127264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.299336 (+1581029us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54002832d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:10.299638 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373978880}
I20260709 13:07:10.304394  4445 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:10.290946 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:10.291463 29146 raft_consensus.cc:1240] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:10.312940 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1370) took 2000 ms. Trace:
I20260709 13:07:10.313378 29146 rpcz_store.cc:276] 0709 13:07:08.312488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.004526 (+692038us) service_pool.cc:224] Handling call
0709 13:07:10.290917 (+1286391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.311799 (+ 20882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.312681 (+   882us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540027d8d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:10.312859 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2954110208}
W20260709 13:07:10.314388 29148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 3312) took 2185 ms (client timeout 1878 ms). Trace:
I20260709 13:07:10.297727 29142 rpcz_store.cc:276] 0709 13:07:08.079059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.709429 (+630370us) service_pool.cc:224] Handling call
0709 13:07:10.284931 (+1575502us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540027d8d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:10.285322 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3118038272}
I20260709 13:07:10.277778  4347 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:10.314922 29148 rpcz_store.cc:269] 0709 13:07:08.129215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.399516 (+270301us) service_pool.cc:224] Handling call
0709 13:07:10.313996 (+1914480us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54003a57d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:10.314151 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492664064}
I20260709 13:07:10.314742 29141 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21. Ops: []
W20260709 13:07:10.326264 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:10.333845 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.328632  4017 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:07:10.334663 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.334810  4017 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:10.272946 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3243) took 2214 ms. Trace:
W20260709 13:07:10.336689 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.336508 29144 rpcz_store.cc:276] 0709 13:07:08.058730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.844474 (+785744us) service_pool.cc:224] Handling call
0709 13:07:10.028749 (+1184275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.028948 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:10.028983 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:10.029043 (+    60us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:07:10.029072 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.029135 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:10.272329 (+243194us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400144e50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:10.272627 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2722758400}
I20260709 13:07:10.340590 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3240) took 2336 ms. Trace:
I20260709 13:07:10.341188  4353 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.344427  4546 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:10.334993 29066 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3252) took 1914 ms. Trace:
I20260709 13:07:10.348670  4546 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:10.349414  4546 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:10.347800 29066 rpcz_store.cc:276] 0709 13:07:08.420840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.430332 (+  9492us) service_pool.cc:224] Handling call
0709 13:07:08.430570 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:08.731097 (+300527us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:07:08.731423 (+   326us) log.cc:844] Serialized 14 byte log entry
0709 13:07:09.065191 (+333768us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:09.065258 (+    67us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:07:09.065312 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.062290 (+996978us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b180049edc0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a20522 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:07:10.334393 (+272103us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54002ace54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:10.334477 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:10.334718 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51197,"prepare.queue_time_us":669,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":81914,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":8616,"spinlock_wait_cycles":701187712,"thread_start_us":280646,"threads_started":1,"wal-append.queue_time_us":374405,"wal-append.run_cpu_time_us":1704,"wal-append.run_wall_time_us":1059437}
I20260709 13:07:10.346583 29141 rpcz_store.cc:276] 0709 13:07:08.003887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:08.805392 (+801505us) service_pool.cc:224] Handling call
0709 13:07:10.314223 (+1508831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:10.339152 (+ 24929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:10.340270 (+  1118us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54003a57d0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:10.340492 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3469112704}
I20260709 13:07:10.328083  4547 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:10.352972 29188 leader_election.cc:336] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.551s (SENT)
I20260709 13:07:10.369414 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:10.370975  4535 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:10.370783  4329 raft_consensus.cc:3055] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:10.372455  4329 raft_consensus.cc:740] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:10.346973  4353 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.374477  4329 consensus_queue.cc:260] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.376283  3803 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.376937  4353 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.377321  3803 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.351332  4546 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.380038 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:10.380335 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.381461  4017 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.382061  4546 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:10.384033  3803 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.366891  4543 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:10.381515 29145 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:10.380247  4329 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:10.389673 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.390702 29148 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:07:10.392242 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.395167 29142 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:10.393808 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.400815 29146 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:10.402163 29144 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:07:10.403683  4206 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:10.404044 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.404567  4241 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.404920 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:10.405026  4377 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.405218  4017 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:10.405921  4339 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:07:10.407307 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:10.335215 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.417587 29148 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
W20260709 13:07:10.319977  3846 consensus_meta.cc:287] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.578s	user 0.019s	sys 0.009s
I20260709 13:07:10.425228 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:10.439327 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.435294 29142 raft_consensus.cc:3060] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:10.427875  4445 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.449931 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:10.426091  4377 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.456585 29148 raft_consensus.cc:1240] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:10.427007  4241 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.456292  4206 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.445619  4547 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.456000  4475 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:10.479599  4576 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:10.481619 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 74648.1us
I20260709 13:07:10.485515 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:10.491931 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:10.496685 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.483755 29037 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:10.488703 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.476850 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:10.483744 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:10.484122 29145 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:10.489413 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.461992  4445 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.485777 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:10.485119 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.484257 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.513311 29070 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:10.492219 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.486179  4206 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.491372  4367 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:10.489678  3803 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:10.522230 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.484529 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.521821 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.492619  4481 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.528210 29071 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:07:10.528594 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 45042.1us
I20260709 13:07:10.531041 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.536044 29148 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:10.540268  4353 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.541285  4353 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.544221  4565 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:10.480629  4576 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.545161  4565 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.546607 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.547174  4576 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:10.547463 29215 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:10.547887  4553 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.548583  4553 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.549053 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.550011 29219 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:10.551443  4553 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:10.540302 29911 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:10.553303  4579 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:10.553573 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.551332 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.554363 29908 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:10.548606  4538 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:10.555261  4353 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.556164  4535 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:10.556123  4538 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.556936  4535 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.559500  4535 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.560786  4475 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.563087  3803 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:10.544348 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 128255us
I20260709 13:07:10.527678  3846 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.568800 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.570711  4476 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.571178 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.580013 29073 raft_consensus.cc:2410] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 23 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
I20260709 13:07:10.571717 29067 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:10.572225 29911 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:10.573582  4475 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.576056  4481 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.576507 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.536871  4369 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:10.559746  4252 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:07:10.565945  4579 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:10.564216 29904 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:07:10.583197 29069 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:10.584852 29910 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:10.586120 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:10.586584  4538 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.587239 29145 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:10.592271 29070 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:10.594686 29071 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:10.595034 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.597795 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.600684 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:10.581924  4565 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:10.602427 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.605440 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:10.593577  4579 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:10.610052 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:10.614897 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:10.615720 29220 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:10.620268 29146 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:10.621213 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.621901 29146 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:10.622462  4578 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.622992  4578 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.604625  4367 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:10.624231  4367 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.624051  4263 consensus_queue.cc:1059] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.027s
I20260709 13:07:10.624329 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.624676  4367 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:10.625008  4263 raft_consensus.cc:3055] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:07:10.625377 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.625991 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.626324 29215 raft_consensus.cc:2410] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:07:10.627025 29214 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 13:07:10.627300 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.628211 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.628649 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.628821 29148 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:07:10.628942  4481 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:10.629771 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.630074 29148 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:10.631018 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.625016 29219 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:10.631299 29148 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:10.632205 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.632437 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.633198 29911 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:10.633347 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.634241 29219 raft_consensus.cc:2410] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:07:10.636076 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:10.637300 29186 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:10.638679 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:10.638787  4560 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:10.639122  4547 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:10.639983 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.640784 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.599462  4476 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.643553  4476 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:10.636255 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.644055 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.644215 29222 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:10.639523  4560 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.644721 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.608732  3846 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.645205 29902 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:10.645431 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.646443 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.631711 29213 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:07:10.648291 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:10.648232 29221 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:10.649542 29141 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:10.649323 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.612703 29140 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 13:07:10.654009 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 49560.6us
I20260709 13:07:10.637756 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.654678 29215 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:07:10.674855 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:10.675763 29139 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:10.677615 29146 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:10.674840 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:10.675998 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.683208 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.683776 29148 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
W20260709 13:07:10.684206 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 70191.3us
I20260709 13:07:10.637640  3803 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:10.685582 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.625409  4263 raft_consensus.cc:740] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:10.687795 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.687004 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.654366  4565 raft_consensus.cc:2804] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:10.690828 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.692342 29073 raft_consensus.cc:2410] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:07:10.694919 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.697939 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.704114 29217 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:07:10.703619 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54892 (request call id 1270) took 1400 ms. Trace:
I20260709 13:07:10.699124 29216 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:10.709514 29904 rpcz_store.cc:276] 0709 13:07:09.303146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:09.359695 (+ 56549us) service_pool.cc:224] Handling call
0709 13:07:10.703340 (+1343645us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54005a4e54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:10.703507 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2720851840}
I20260709 13:07:10.721585 29220 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:10.718523 29186 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:10.727545 29070 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:10.730257 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.731043 29217 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:10.736491 29066 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 21: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:10.736893  4560 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.737920 29067 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:07:10.739230  4206 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:10.740909  4206 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:10.741317  4206 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:10.742328  4206 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.743741  4206 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:10.699160  4475 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:10.746249  4475 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.746718  4475 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:10.752228  4445 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:10.753270 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.754037 29904 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:10.754477  3846 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.756930 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:10.758047  3803 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:10.760479 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:10.761818 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.762903 29217 raft_consensus.cc:2410] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 28 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 13:07:10.764314 29145 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:07:10.764701 29145 pending_rounds.cc:85] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 8
I20260709 13:07:10.764935 29145 pending_rounds.cc:107] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:07:10.765019 29222 raft_consensus.cc:1240] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:10.765213 29145 raft_consensus.cc:2889] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:10.766217 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.766772 29222 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:10.767609 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.763768 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.769188 29214 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 25: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:10.727341 29911 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
W20260709 13:07:10.773743 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.739144 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.724241 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.777685 29908 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:10.776512 29142 raft_consensus.cc:2468] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:10.785195 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.785234 29217 raft_consensus.cc:2393] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 21: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:10.786079 29904 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:10.719815  4263 consensus_queue.cc:260] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.790733 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.789204 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.790136 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.787683 29221 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:07:10.800869 29148 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:07:10.728863  4565 raft_consensus.cc:697] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:10.802901 29911 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:07:10.812877  4565 consensus_queue.cc:237] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.769807  4017 consensus_queue.cc:1059] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:07:10.816749  4017 raft_consensus.cc:3055] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:10.817189  4017 raft_consensus.cc:740] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:10.820079  4241 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:10.824172 29142 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:10.799114 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:10.824801 29213 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:07:10.825228 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.825821 29142 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:10.826444  4501 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:10.826639 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.827010  4501 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:10.827567  4501 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:10.803256  3846 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:07:10.827719 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:10.804850 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 107252us
I20260709 13:07:10.808077 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.795321  4481 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:10.788821  4476 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:10.827878 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.829841 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.829209 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.831246  3846 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:10.829530 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:07:10.841761 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 310964us
I20260709 13:07:10.785349  3803 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 13:07:10.847090 29140 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:10.846665 29139 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
I20260709 13:07:10.847527 29221 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:10.854839 29141 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
W20260709 13:07:10.856879 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.858311 29216 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:10.795591  4546 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.054s
I20260709 13:07:10.865787 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:10.868541 29142 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:10.856284 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:10.866277 29139 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:07:10.856873 29902 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:10.829777  4377 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:10.867278 29147 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:10.875763 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.876757 29148 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:07:10.870734 29220 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:07:10.840319  4476 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:10.848435 29146 raft_consensus.cc:2376] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 27. Re-sending same reply.
I20260709 13:07:10.839165  4578 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:10.883610 29143 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:10.890847 29112 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:10.894389 29148 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:10.894721 29216 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:07:10.897089  4377 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:10.897760 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:10.899139  4329 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:10.899682  4329 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:10.900173  3846 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.900207  4329 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:10.902437  4367 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.903671 29072 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 23: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:10.905876  4367 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.899698 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.838951  4481 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.911190  4329 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.915463  4329 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:10.885094  4546 consensus_queue.cc:1243] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 23.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 13:07:10.925177 29073 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:07:10.925683  4576 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:07:10.926097  4595 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:07:10.925705 29073 pending_rounds.cc:85] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 13
I20260709 13:07:10.928462 29073 pending_rounds.cc:107] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:07:10.932964 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:10.933506 29073 raft_consensus.cc:2889] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:10.926326  4576 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:10.934834 29146 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:07:10.937283 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:10.935424 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.885227  4476 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:10.941781 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.938987 29222 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:10.944247 29140 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:07:10.948782 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:10.958256 29146 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:07:10.884827  4206 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 13:07:10.897802  4377 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:10.968649 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:10.977108 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 81028.5us
I20260709 13:07:10.978215 29067 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:07:10.907789  4579 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.988742 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:10.990813 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:10.987907 29186 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:10.969805  4350 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:07:10.989771 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:11.001351 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.989530 29140 raft_consensus.cc:2376] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21. Re-sending same reply.
I20260709 13:07:11.005910  4068 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:11.006534  4068 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:11.006997 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:11.007614  4068 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.009522 29144 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:07:11.010715  4017 consensus_queue.cc:260] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.012023 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:11.013016  4576 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.014719 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:11.015537 29143 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:11.016050 29217 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
W20260709 13:07:11.016914 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48093.1us
I20260709 13:07:11.017468  4547 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:11.017889  4547 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:11.018425  4547 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:11.021554  4486 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:11.015476 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:11.028872 29147 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:11.029139  4535 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:11.030220  4350 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:11.030854  4350 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.031332  4350 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:11.015486  4017 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:11.018086  4241 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:11.037922  4241 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:11.028914 29186 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:11.040238 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:10.982738  4543 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:11.041997  4543 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.042363  4543 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:11.039467  4241 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:10.985167  4481 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:11.044554 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.045009  4538 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:11.045641 29222 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:11.073921 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:11.074887 29141 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:11.044550  4241 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:11.081722 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:11.085359  4538 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:11.087749  4206 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:11.088449  4206 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:11.091248 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 413797us
I20260709 13:07:11.098448 29142 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:11.112535 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:11.131984 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 126712us
I20260709 13:07:11.138314 29140 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:11.148523 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:07:11.151404 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.153923  4579 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:11.162600  4547 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.166198  4350 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.167649 29910 raft_consensus.cc:2368] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:07:11.179190 29222 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:11.183897 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:11.187672 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:07:11.190008 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 76261.3us
I20260709 13:07:11.192335  4576 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:11.191996  4252 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:07:11.199352 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.200752  4206 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:11.201875  4377 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:11.212239 29185 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:11.214461  3846 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:11.214946  3846 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.215315  3846 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:11.215011  4475 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.217464 29064 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 23: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:11.146636  4367 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.227294 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:11.228137 29146 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:11.243489 29903 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:11.245725  4565 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:11.248294  4367 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.253259 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.254798 29213 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:07:11.255367 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.255789 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.256536 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.257189 29213 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:11.253509 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.257767 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:11.258232 29217 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:11.258931  4565 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:11.259419  4565 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:11.259824  4565 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:11.257577 29213 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:07:11.261266 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.261878 29213 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.263538 29213 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:11.263648  4606 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.139s	user 0.000s	sys 0.002s
I20260709 13:07:11.265607 29220 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:07:11.264730 29222 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:11.267045 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.267906 29216 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 17.
I20260709 13:07:11.272225  4206 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:11.273572  4481 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:11.274060  4481 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:11.275295  4481 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.277451  4481 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:11.287840 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.290839  4565 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:11.303625 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.305977 29213 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:07:11.315045  4547 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:11.219578  4501 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.317186 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:11.340698 29905 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:11.340901 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.227571  4350 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:11.321727  4543 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:11.329749 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 70151.3us
I20260709 13:07:11.357779 29146 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:11.331166  4475 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:11.251571  4367 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:11.335638 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.337638 29213 pending_rounds.cc:85] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 13
I20260709 13:07:11.371119 29213 pending_rounds.cc:107] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:07:11.371503 29213 raft_consensus.cc:2889] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:11.341977 29216 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
W20260709 13:07:11.320509 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:11.343398  4605 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.217s	user 0.000s	sys 0.005s
I20260709 13:07:11.345137  4565 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:11.345927 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:11.346124 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.346746 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:11.346961 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.377015 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:11.379688 29909 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:11.379979 29911 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:11.380776 29220 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 23: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:11.381738 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:11.381925 29904 raft_consensus.cc:1240] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:11.379065 29222 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:11.383363 29217 raft_consensus.cc:2410] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:07:11.385401 29068 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 20: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:11.386238 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:11.388442 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:11.391172 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:11.394035 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:11.395171 29065 raft_consensus.cc:2410] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:07:11.352109  4339 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:07:11.396615  4339 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:11.397117  4339 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:11.400022 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 54231.5us
I20260709 13:07:11.354470  3846 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.404737 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:11.398505  4339 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.413321 29905 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:11.414587 29186 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:11.412758 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:11.422291 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:11.427162 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.418871 29065 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:11.434037 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:11.432593 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:11.438776 29214 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 26: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:11.446761  4068 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:11.447295  4068 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:11.449538  4543 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:11.452068 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.388296  4476 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.420467  4535 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:11.418005  4565 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:11.447696  4068 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:11.463332 29185 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:07:11.460807 29070 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.003s	sys 0.015s
I20260709 13:07:11.441803  4547 raft_consensus.cc:2804] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:07:11.468169 29070 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.004s	sys 0.015s
W20260709 13:07:11.474184 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.484766 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:11.462849  4565 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.488498 29905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:07:11.490945 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 71237.3us
W20260709 13:07:11.494166 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 72926us
I20260709 13:07:11.473320  4445 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:11.503228 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.503439 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.505028 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.507463 29213 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:11.505208 29216 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:11.509879 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:11.495610  4565 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:11.514652 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:11.520298  4560 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.521009  4476 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:11.520870  4560 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.522754 29113 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:11.523157  4560 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:11.524295 29065 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 13:07:11.524669 29065 pending_rounds.cc:85] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 12
I20260709 13:07:11.524928 29065 pending_rounds.cc:107] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 13:07:11.525002 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:11.525626 29064 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:11.501618  4263 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:11.527729 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:11.528815  4603 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:11.530547  4603 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:07:11.532130  4476 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:11.532482 29019 auto_rebalancer.cc:893] Removing replace marker failed: Replica 6ad9faa1c14444b29c9db8e71f46e2d6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:11.532693  4476 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:07:11.533010 29019 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:07:11.533361 29019 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:07:11.523774 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:11.535142  4445 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:11.535691  4445 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:11.536289  4445 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:11.533317  4476 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:11.537783  4608 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:07:11.525162 29065 raft_consensus.cc:2889] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:11.535490  4535 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:11.539654  4501 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:11.555284  4535 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:11.503494  4339 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:11.572891 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:11.573757  4578 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:11.574357  4578 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:11.549937 29910 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:07:11.539592  4576 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.554044  4538 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.007s
W20260709 13:07:11.570775 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.592971 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:11.593773 29909 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:07:11.598099 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 68984.8us
W20260709 13:07:11.607649 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 93144.7us
W20260709 13:07:11.612501 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 60971.8us
I20260709 13:07:11.613401 29112 leader_election.cc:400] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
W20260709 13:07:11.614470  4618 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.003s
I20260709 13:07:11.615180 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:11.617197  4367 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:11.617350 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.618249 29214 raft_consensus.cc:2368] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21.
I20260709 13:07:11.618161  4367 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:07:11.620328  4616 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
I20260709 13:07:11.620615  4241 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:11.622222  4476 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.622926 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:11.624562 29039 leader_election.cc:336] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.132s (SENT)
W20260709 13:07:11.626041  4607 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.496s	user 0.002s	sys 0.000s
I20260709 13:07:11.626200 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:11.628244  4206 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:11.628749  4538 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.628962  4560 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:11.629352  4560 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:11.629305  4538 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:11.630002  4609 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:07:11.396631 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:11.630550  4560 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:11.624516 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 100834us
I20260709 13:07:11.613715 29112 leader_election.cc:403] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:07:11.634478 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.623878 29213 raft_consensus.cc:2410] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 13:07:11.505795  4622 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:11.543996  4350 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.636348 29222 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:11.637593 29871 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:11.553104  3846 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:11.645677 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:11.646901 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.650739 29909 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 26.
I20260709 13:07:11.652256 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.656908 29911 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:11.658511 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:11.658299  4350 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.659497 29218 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:07:11.660842  4350 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:11.661088 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:11.662843 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:11.634995  4206 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:11.663014  4445 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.598387  4576 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.664546 29065 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:11.658807  4347 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:11.663640  4206 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:11.673202 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.665782 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:11.680795 29037 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:11.634665  4538 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:07:11.689605 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:11.690493 29064 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:07:11.696261 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:07:11.700816  4617 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.175s	user 0.000s	sys 0.002s
I20260709 13:07:11.666402  4252 raft_consensus.cc:2764] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
W20260709 13:07:11.705248 28962 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 243) took 30295 ms (client timeout 15000 ms). Trace:
I20260709 13:07:11.637352  4367 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:07:11.706745  4367 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:11.707301  4367 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:11.709715 29067 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 20: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:11.711256 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:11.683148  4560 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:11.575524  4578 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.676232  4622 raft_consensus.cc:697] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:11.715449 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:11.715816 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:11.718874 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 55831.9us
I20260709 13:07:11.697410  4369 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:11.708664  4576 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:11.716344  4476 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:11.696171  4624 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.054s
I20260709 13:07:11.739753 29113 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:11.740135 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 121052us
I20260709 13:07:11.745299  4547 raft_consensus.cc:697] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:07:11.749905 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 75353.1us
I20260709 13:07:11.764523 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:11.765851  4635 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:11.739259  4369 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:11.785904 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 172673us
W20260709 13:07:11.786999 29111 connection.cc:256] RPC call timeout handler was delayed by -2.1441004030002659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
W20260709 13:07:11.794836 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 118148us
I20260709 13:07:11.800550 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:11.801246 29141 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:07:11.705729 28962 rpcz_store.cc:269] 0709 13:06:41.409387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:50.730870 (+9321483us) service_pool.cc:224] Handling call
0709 13:07:11.705005 (+20974135us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:04.882512 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:04.894724 (+ 12212us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:04.894795 (+    71us) write_op.cc:435] Acquired schema lock
0709 13:07:04.894813 (+    18us) tablet.cc:662] Decoding operations
0709 13:07:04.897253 (+  2440us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:04.897356 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:04.897382 (+    26us) tablet.cc:685] Acquiring locks for 32 operations
0709 13:07:05.116793 (+219411us) tablet.cc:701] Row locks acquired
0709 13:07:05.116860 (+    67us) write_op.cc:260] PREPARE: finished
0709 13:07:05.117055 (+   195us) write_op.cc:270] Start()
0709 13:07:05.117245 (+   190us) write_op.cc:276] Timestamp: P: 1783602425116982 usec, L: 0
0709 13:07:05.117284 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:07:05.118133 (+   849us) log.cc:844] Serialized 12612 byte log entry
0709 13:07:05.835744 (+717611us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:06.422119 (+586375us) op_driver.cc:464] REPLICATION: finished
0709 13:07:06.894061 (+471942us) write_op.cc:301] APPLY: starting
0709 13:07:06.915657 (+ 21596us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:07.533292 (+617635us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:07.533336 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:08.172290 (+638954us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:08.172829 (+   539us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=64,key_file_lookups=64,delta_file_lookups=64,mrs_lookups=0
0709 13:07:08.172899 (+    70us) write_op.cc:312] APPLY: finished
0709 13:07:08.173595 (+   696us) log.cc:844] Serialized 276 byte log entry
0709 13:07:08.399256 (+225661us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:08.466674 (+ 67418us) write_op.cc:454] Released schema lock
0709 13:07:08.566652 (+ 99978us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:08.657875 (+ 91223us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:08.657921 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1798688,"reported_tablets":49,"spinlock_wait_cycles":8064,"child_traces":[["op",{"apply.queue_time_us":415752,"apply.run_cpu_time_us":60135,"apply.run_wall_time_us":1806426,"cfile_cache_hit":130,"cfile_cache_hit_bytes":137887,"num_ops":32,"prepare.queue_time_us":300239,"prepare.run_cpu_time_us":25547,"prepare.run_wall_time_us":353511,"raft.queue_time_us":441355,"raft.run_cpu_time_us":18467,"raft.run_wall_time_us":374732,"replication_time_us":1304594,"spinlock_wait_cycles":49678976,"thread_start_us":966204,"threads_started":5,"wal-append.queue_time_us":422959,"wal-append.run_cpu_time_us":40363,"wal-append.run_wall_time_us":3414173}]]}
I20260709 13:07:11.803812 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:11.804186 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 64520.6us
I20260709 13:07:11.806607 29146 raft_consensus.cc:1240] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:11.809355 29140 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:07:11.793074  4475 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:11.748886  4367 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.815274 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:11.816541 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.816749 29908 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 13:07:11.820549 29139 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:07:11.794317  4565 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.810225  4017 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:11.822659 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 44688.5us
I20260709 13:07:11.824079 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:11.827090 29112 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:11.812861  4475 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.828169  4475 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:11.831315 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:11.831393 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:11.831862 29908 raft_consensus.cc:1240] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:11.833015  4546 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:11.833335 29113 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:11.833446  4546 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.834028  4546 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:11.819602  4445 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:11.840476  4627 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:07:11.840737  4445 consensus_queue.cc:1059] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:07:11.841583  4626 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:07:11.841967  4445 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:11.818598  4622 consensus_queue.cc:237] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.842536  4445 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:11.843078  4628 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.840551  4631 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:07:11.845090  4631 consensus_queue.cc:1243] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:07:11.826694  4263 consensus_queue.cc:1059] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 13:07:11.846401  4263 raft_consensus.cc:3055] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:11.846736  4263 raft_consensus.cc:740] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:11.847115  4347 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:11.847872 29911 raft_consensus.cc:1240] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:11.848541 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:11.840642  4632 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 20. Current term is 21.
I20260709 13:07:11.849948  4632 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:11.850217  4241 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:11.850478  4632 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:11.850667  4241 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.851094  4241 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:11.852109  4481 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:11.853653  4481 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:11.854404  4565 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:11.855422  4639 raft_consensus.cc:3055] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:07:11.857234 28962 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 277) took 21007 ms (client timeout 15000 ms). Trace:
W20260709 13:07:11.857642 28962 rpcz_store.cc:269] 0709 13:06:50.850008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.857044 (+21007036us) service_pool.cc:211] Skipping call since client already timed out
0709 13:07:11.857158 (+   114us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:07:11.861049 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.849962 29038 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:11.847636  4263 consensus_queue.cc:260] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.860709  4629 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:07:11.843672  4628 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.873915  4263 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:11.855860  4639 raft_consensus.cc:740] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:11.876596  4367 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:11.876652  4635 consensus_queue.cc:1059] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:07:11.877444  4628 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:11.877945  4635 raft_consensus.cc:3055] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:11.878276  4639 consensus_queue.cc:260] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.879545  4639 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:11.880162  4636 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 13:07:11.881212  4605 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:07:11.886669 29038 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:11.889644 29141 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
I20260709 13:07:11.895223 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:07:11.896783 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 63571.1us
I20260709 13:07:11.898509 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:11.880190 29145 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
W20260709 13:07:11.903017 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 100458us
I20260709 13:07:11.904376  4546 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.906941  4546 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:11.910671 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:11.911293 29141 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:11.826443  4640 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.869583  4339 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:11.843515  4445 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.928665  4377 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:11.933704  4475 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.937222  4637 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:07:11.940034 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.912719 29875 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:11.942425 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:11.951120 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:11.878314  4635 raft_consensus.cc:740] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:07:11.956646 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 58230.1us
I20260709 13:07:11.954190 29906 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:07:11.954365 29875 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:11.851542  4632 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.956101  4481 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:07:11.966285 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 92976.5us
I20260709 13:07:11.967406 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:11.967818 29141 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:07:11.950891  4377 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.900820  4017 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:07:11.901072  4565 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:07:11.868964  4547 consensus_queue.cc:237] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.953249  4637 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:11.885048  4538 consensus_queue.cc:1059] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.074s
I20260709 13:07:11.960713  4501 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:07:11.976822 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 81691.4us
I20260709 13:07:11.961719  4481 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.982396 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:11.977718 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:11.983073 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:11.978790 29875 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:11.951130  4339 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.883338  4369 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:11.973271  4350 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:11.994357  4350 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:11.994841  4350 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:11.974759  4377 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:11.998705 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:11.945948  4640 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:11.987460  4481 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:11.988263  4629 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:11.984441  4641 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:12.041739 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 63024us
I20260709 13:07:12.045292 28965 catalog_manager.cc:5697] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 8 to 10, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 10 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:12.038152  4627 raft_consensus.cc:2804] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:07:12.052423 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 53805.3us
I20260709 13:07:11.998356  4631 consensus_queue.cc:1059] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:07:12.053941  4646 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:12.054157  4017 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:12.038591  4206 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.038388  4636 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:12.038383  4629 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.053278  4622 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:12.082273  4630 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:12.099793  4630 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:12.099627  4622 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.100811  4630 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.101823 29215 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
I20260709 13:07:12.102010  4622 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:12.083900  4643 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:12.084426  4651 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:07:12.085300  4565 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:12.086270  4648 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:07:12.086638  4339 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:12.059684  4646 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.087625  4475 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.065979  4629 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:07:12.088603  4535 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.003s
I20260709 13:07:12.093664  4640 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:12.036293  4635 consensus_queue.cc:260] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.096557 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:12.096881 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 118389us
I20260709 13:07:12.095214  4350 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.098820  3846 raft_consensus.cc:3055] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:07:12.062099  4636 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.116004 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:12.122179 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 181800us
I20260709 13:07:12.064254  4627 raft_consensus.cc:697] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:12.135735  4339 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:12.134048  4646 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:12.136195  4643 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:12.134061  4624 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:07:12.137796  4654 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.055s	user 0.002s	sys 0.000s
W20260709 13:07:12.117286  4068 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.003s
W20260709 13:07:12.140244 29111 connection.cc:256] RPC call timeout handler was delayed by -1.8731682449997606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
I20260709 13:07:12.144407  4636 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:12.147677 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:07:12.154311 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 176629us
W20260709 13:07:12.154500 29111 connection.cc:256] RPC call timeout handler was delayed by -1.7259732410002471s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 13:07:12.156141 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 118906us
W20260709 13:07:12.163419 29111 connection.cc:256] RPC call timeout handler was delayed by -1.7169677120000415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
W20260709 13:07:12.163826 29111 connection.cc:256] RPC call timeout handler was delayed by -1.6998732410002049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
W20260709 13:07:12.164177 29111 connection.cc:256] RPC call timeout handler was delayed by -1.1970687129996804s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260709 13:07:12.164438 29111 connection.cc:256] RPC call timeout handler was delayed by -1.1719005040004049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
W20260709 13:07:12.164700 29111 connection.cc:256] RPC call timeout handler was delayed by -1.0122989999999845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
I20260709 13:07:12.165951  4624 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.166424 29111 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.168401  4624 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:12.141199  3846 raft_consensus.cc:740] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:12.169543  3846 consensus_queue.cc:260] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.156950  4627 consensus_queue.cc:237] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.058936  4637 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.170822  3846 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:12.143337  4535 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.181619  4639 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:12.182237  4535 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:12.171041  4068 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.184573 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:12.157953  4350 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.146224  4206 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:12.159047  4643 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.186424  4068 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.187417 29904 raft_consensus.cc:2410] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:07:12.160435 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:12.173128 29038 leader_election.cc:400] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21.
I20260709 13:07:12.173662 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:12.147258  4501 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:12.175326 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:12.110909  4241 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.193521  4501 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.195142  4241 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.179469 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.196004 29217 raft_consensus.cc:3060] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:07:12.196882  4206 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:12.197682  4206 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:12.162562  4339 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.141789  4565 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.201520  4339 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:12.201931  4565 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:12.203213 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.189145  4637 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:12.204075 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.204214  4628 raft_consensus.cc:3055] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:12.197822 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:12.192742 29038 leader_election.cc:403] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:12.218693 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 52415us
W20260709 13:07:12.222194 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 49307.6us
I20260709 13:07:12.217099  4628 raft_consensus.cc:740] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:12.225414  4628 consensus_queue.cc:260] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.226653  4628 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:12.232856 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:12.237962 29219 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:12.241248  4661 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:12.241406 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:12.242303  4661 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.252799 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:12.240648 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:12.243393 29903 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:12.248088  4241 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:12.249895  4481 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.295070  4629 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.246188  4661 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:12.296051 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 98321.6us
I20260709 13:07:12.296412  4660 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:12.316431 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:12.317274 29068 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:12.290023 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 49498.2us
I20260709 13:07:12.300894 29217 raft_consensus.cc:2468] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:07:12.244127  4339 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:07:12.313390  4655 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:12.315284 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:12.321753 29874 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:12.263888  4377 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.315470  4206 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:07:12.334692 29073 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:07:12.315323  4017 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:07:12.325250  4350 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.337958  4350 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.328284  4263 raft_consensus.cc:2804] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:12.339709  4263 raft_consensus.cc:697] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:07:12.340346 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 357997us
I20260709 13:07:12.338316  4377 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.327337  4660 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:12.374243 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.377839 29070 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:07:12.347555  4661 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:12.384130 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68936.4us
I20260709 13:07:12.340291  4350 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:12.384371 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.316460  4636 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.385351 29113 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:12.386376 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.430523 29141 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:12.378412  4643 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 13:07:12.405339  4664 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:07:12.434409  4645 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:12.428507 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:12.393769 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:12.450405 29217 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
I20260709 13:07:12.431063 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:12.387748 29186 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:12.451784 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 65974.5us
W20260709 13:07:12.433701 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 56680.8us
I20260709 13:07:12.434499 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:12.389237  4263 consensus_queue.cc:237] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.437443  4635 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:12.440702  4629 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.428210 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:12.452910 29073 raft_consensus.cc:3060] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:12.453754  4660 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.455562 29065 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 21: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:12.508230  4629 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:12.454835  4017 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.421976  4646 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.462505 29145 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 26.
I20260709 13:07:12.510077 29111 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.514292  4636 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:12.539520 29068 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:12.540542 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:12.462083  4547 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:12.542639 29073 raft_consensus.cc:2468] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
W20260709 13:07:12.547027 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.547982 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:12.549427 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.549535 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.565624 29221 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:12.565836 29220 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:07:12.550127 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.542485 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:12.567814 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 139390us
I20260709 13:07:12.569726 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.571225 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:12.529618  4476 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:12.578899 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.572238 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:12.599175 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:12.599618 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.599967  4501 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:12.603688 29071 raft_consensus.cc:2368] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:12.599284 29068 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 17.
I20260709 13:07:12.549031  4630 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:12.609285 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:12.552906  4629 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:12.611624 29903 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:07:12.611901 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:12.581226  4637 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:12.600732 29073 raft_consensus.cc:2368] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24.
I20260709 13:07:12.604617 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:12.605326 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.598687 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.605180 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.605844 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.617242 29066 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:12.606097 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:12.546455  4017 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.626260  4017 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.627194 29065 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 25 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
W20260709 13:07:12.601773 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 216459us
I20260709 13:07:12.618148 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.628441 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:12.564996  4547 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.621419  4629 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.605135  4367 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:12.624117  4339 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21.
I20260709 13:07:12.629405 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:12.602859  4661 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:12.635193 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:12.603410  4481 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.630707  4501 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.608350  4377 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:12.642760 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.642897 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.661927 29214 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:12.611840  4206 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:07:12.637948  4476 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:12.665125  4476 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:12.665535  4476 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:12.639848  4367 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.668232  4367 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:12.672032  4638 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:12.672791  4638 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:12.677827 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 48479us
W20260709 13:07:12.678881 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 168922us
W20260709 13:07:12.643174  4667 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 13:07:12.643996 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:12.680410  4657 raft_consensus.cc:2764] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:07:12.632843  4651 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:12.637252  4547 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:12.636337  4655 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.656594 29068 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:12.678906 29113 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:12.688714 29220 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:12.615721  4630 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:12.689935 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:12.691027 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.699343 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:12.700067 29068 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:12.683352  4501 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:12.702824 29139 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:12.684157  4651 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.705323 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.714715 29214 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:12.689636  4671 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:12.722404 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.743337 29071 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:12.698038  4628 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:12.723579 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.745580 29071 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:12.748337 29068 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:12.717720  4475 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.718091  4339 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.716305  4651 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:12.754176 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:12.698454  4535 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:07:12.756353 29221 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:12.689548  4655 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.751452  4628 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:12.773136 29071 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:07:12.754318  4671 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:12.784443 29185 leader_election.cc:400] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:12.785768 29185 leader_election.cc:403] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:12.786799 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:12.789275 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:12.776165 29071 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:12.792253  3846 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:07:12.793056 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:12.794421 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.772001  4657 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:12.798079 29185 leader_election.cc:400] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24.
I20260709 13:07:12.802167 29185 leader_election.cc:403] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 13:07:12.796588 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 211771us
I20260709 13:07:12.783286  4628 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:12.805094 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.787271  4538 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.767210  4475 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.769574  4339 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.793272  4646 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.845038  4645 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:12.816684 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 93232.4us
I20260709 13:07:12.817039  4671 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.818907  4547 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.818945  4475 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.802024  4543 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:12.801899  4641 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:12.795987  4661 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:12.810082  4640 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:12.787495  4579 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:07:12.861116 29068 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:07:12.832049  4538 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.862483  4339 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.864158  4538 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:12.868486 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.832952  4630 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:12.796216  4481 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:12.809135  4650 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:12.838917  4476 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:12.846472 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 90844.8us
I20260709 13:07:12.813210  4501 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.851280  3846 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:12.812029 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.895674 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:12.896471 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 84560.7us
I20260709 13:07:12.896890 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.897235 29185 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.807077  4017 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.878829 29908 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:07:12.834149  4565 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
W20260709 13:07:12.894095 29188 leader_election.cc:336] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.602s (SENT)
I20260709 13:07:12.811622  4578 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.072s
I20260709 13:07:12.853245  4543 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:12.902084 29188 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.902581 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.848265  4367 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.884021  4646 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:12.914925 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:12.907873 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:12.916404 29038 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:12.924270 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:12.924945 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:12.925626 29217 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:12.906035  4017 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.926414  4206 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:12.928020  4017 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.912982  4629 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 13:07:12.904476  4538 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:12.930142  4638 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:12.931262  4547 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:12.910530  4543 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:12.939764 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.940294  4241 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:12.940384 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:12.941498 29217 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:12.948805 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.888823  4686 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 13:07:12.954118 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:12.954782 29216 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:12.910354  4661 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:12.956909 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.957683 29219 raft_consensus.cc:2393] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:12.958463 29141 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:07:12.959302  4641 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:12.959728  4641 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:12.960747  4641 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:12.962122  4641 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:12.964792 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.970952  4501 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.919025  4655 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:12.973531  4367 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:12.974841 29069 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
W20260709 13:07:12.976589 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 108210us
I20260709 13:07:12.978139 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.958395 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:12.979141 29219 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
W20260709 13:07:12.980912 29039 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.805s (SENT)
I20260709 13:07:12.981304 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:12.981528 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:12.949591  4675 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.297s	user 0.003s	sys 0.000s
I20260709 13:07:12.955713 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:12.988549 29216 raft_consensus.cc:2410] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 13:07:12.917722 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:07:12.979460 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 61099.2us
I20260709 13:07:12.985404 29219 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:07:12.987005 29113 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:12.989948 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:12.999394 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 81769.4us
I20260709 13:07:13.017697 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:13.018682 29141 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:07:13.045326 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 64458.4us
I20260709 13:07:13.047976 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:13.048695 29911 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:07:13.048861 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 59013.2us
I20260709 13:07:13.049403  3846 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:13.049877  3846 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:12.985342  4560 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:13.049957 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:07:13.063892 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:13.053849 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 159801us
I20260709 13:07:13.054805 29217 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:07:12.999318  4639 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:13.057025 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:13.065338 29216 raft_consensus.cc:3060] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:07:13.068114 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:12.999259  4646 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:07:13.069379  3846 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:13.082815  3846 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:13.076058 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:13.016119  4475 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:13.078691 29218 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 19. (term mismatch)
W20260709 13:07:13.076121 29188 leader_election.cc:336] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.052s (SENT)
I20260709 13:07:13.003074  4645 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:13.021728  4579 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.088730 29217 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:07:13.081838 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:13.086200 29218 pending_rounds.cc:85] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 18
I20260709 13:07:13.092599 29218 pending_rounds.cc:107] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 25.19
I20260709 13:07:13.092854 29218 raft_consensus.cc:2889] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:13.056686  4565 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:13.115356 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:13.055307  4241 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:13.116755 29141 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 27.
I20260709 13:07:13.056478  4650 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:13.093156 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:13.000638  4630 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 13:07:13.004061  4501 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 13:07:13.129135 29911 raft_consensus.cc:2468] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:13.070048  4206 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:13.133941 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 65928.2us
I20260709 13:07:13.086525  4635 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:07:13.141011 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 47966.7us
I20260709 13:07:13.083592  4655 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:13.056006  4638 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.068s
W20260709 13:07:13.142692 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 77639.2us
I20260709 13:07:13.138100 29186 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:13.147362 29188 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.639s (SENT)
I20260709 13:07:13.118216  4661 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:07:13.155447 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.152156 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:13.115437  4565 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:13.119236  4475 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:13.168358 29216 raft_consensus.cc:2468] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 27.
I20260709 13:07:13.172040 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:13.178778 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 94909.4us
W20260709 13:07:13.183455 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 45478.2us
I20260709 13:07:13.152696  4481 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:13.130874  4579 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:13.165485  4661 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.153116  4645 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:13.170310  4565 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:13.200121 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:13.203146  4661 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:07:13.209609  4694 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:07:13.210047  4691 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:07:13.200753  4547 raft_consensus.cc:2804] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:13.215350  4547 raft_consensus.cc:697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:07:13.216934  4689 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:07:13.190689  4481 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.217643  4481 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:13.218016 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:13.218674  4696 log.cc:927] Time spent T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:07:13.219488 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:13.182619  4377 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:13.228195  4377 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.228732  4377 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:13.228348  4475 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.220244  4645 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:13.220460  4476 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:13.220850  4657 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:13.234678 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:13.235322  4657 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:13.216495  4547 consensus_queue.cc:237] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:13.245576 29903 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
W20260709 13:07:13.247155 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 95235.4us
I20260709 13:07:13.174275  4538 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:07:13.192299  4704 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
W20260709 13:07:13.278367 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 43785.8us
I20260709 13:07:13.279487 29111 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:13.280550  4632 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:13.264734  4565 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:13.282886 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:13.283318  4565 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:13.284026  4632 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:13.238565  4628 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.298065  3846 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:13.298980  3846 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:07:13.302733 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.303113  4579 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:13.240921  4651 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.314303 29073 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 26: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:13.315456  4651 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:13.255232  4650 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:07:13.313205  4628 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:13.317448  4543 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.317934  4705 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:13.240216  4241 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.310961 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:13.292522  4632 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:13.253176  4206 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.306587  4640 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24.
I20260709 13:07:13.346444 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:13.347869 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:13.352654 29216 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:07:13.349500 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.350010 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:13.353988 29188 leader_election.cc:336] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.671s (SENT)
I20260709 13:07:13.350986 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:13.355003 29217 raft_consensus.cc:2368] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24.
I20260709 13:07:13.357879 29218 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:07:13.359828 29220 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
W20260709 13:07:13.366672 29188 leader_election.cc:336] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.875s (SENT)
I20260709 13:07:13.366927 29113 leader_election.cc:400] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24.
I20260709 13:07:13.367311 29113 leader_election.cc:403] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:07:13.331349  4632 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:13.360690  3846 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:13.367249 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:13.371414 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:13.371822 29111 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:13.375204 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:13.360056  4565 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.375124 29216 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:13.359220  4703 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:13.344928  4645 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.042s
I20260709 13:07:13.327886  4663 consensus_queue.cc:1059] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 13:07:13.381224 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:13.361838  4646 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:07:13.396354 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:13.375075  4657 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:13.372920  4637 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:13.416633 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:13.417445 29071 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:13.380043  4703 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.418589 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:13.418234 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:13.389209  4650 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:13.390491  4639 raft_consensus.cc:2764] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:07:13.389430  4635 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:13.374302  4628 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:07:13.400328  4636 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:13.437148 29064 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
W20260709 13:07:13.449934 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 68887.2us
I20260709 13:07:13.452415 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.454366 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.455542 29141 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 28 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 13:07:13.457315 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.458030 29146 raft_consensus.cc:2368] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:13.388386  4565 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.460176 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:13.462634 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:13.463120 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.466323 29146 raft_consensus.cc:2368] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 26. Current term is 27.
I20260709 13:07:13.468888 29214 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 26: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:13.435006  4703 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:13.402678  4377 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.474869 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:13.476011 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.443651  4637 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.418685  4481 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.495363 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.496918 29146 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:13.497893 29140 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:13.499817 29141 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:07:13.503305 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:13.506013 29140 raft_consensus.cc:2376] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20. Re-sending same reply.
I20260709 13:07:13.499311  4663 consensus_queue.cc:1243] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:07:13.518421 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.519493 29141 raft_consensus.cc:2393] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:13.500456  4639 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:13.523367 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.524289 29140 raft_consensus.cc:2393] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:13.527701 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:13.528419 29140 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:07:13.529103 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 109677us
I20260709 13:07:13.509048  4658 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.511649  4718 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.498580  4637 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.546458 29140 raft_consensus.cc:1240] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:13.551050 29071 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:13.555361 29072 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:13.525475  4543 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:13.556864 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.562665 29071 raft_consensus.cc:1275] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:13.563107 29140 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 27.
I20260709 13:07:13.572096 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:13.573454 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:13.574326 29220 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:07:13.589166  4637 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:13.592350 29220 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:07:13.596809  4565 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:13.599519  4716 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:07:13.601400  4377 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:13.601835  4719 log.cc:927] Time spent T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:07:13.530781  4723 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:07:13.610440 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:13.611501 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:13.611397 29141 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:13.667084 29220 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 20. Current term is 21. Ops: []
I20260709 13:07:13.707743 29220 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:13.709137 29220 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:07:13.545315  4658 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:13.714001 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:07:13.715590 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 172115us
I20260709 13:07:13.728386  4632 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.731564 29874 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:13.548542  4635 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.547821  4718 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.736037 29141 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:13.720818  4637 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.735656 29140 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:13.745934 29222 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:13.750489 29145 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:07:13.587774  4680 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.146s
I20260709 13:07:13.733846  4627 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.711503  4661 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.764217  4627 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:13.765702  4661 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:13.766418  4627 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:13.766752  4481 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:13.767002  4627 raft_consensus.cc:2804] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:07:13.767535  4627 raft_consensus.cc:697] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 27 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:13.769172  4658 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:13.752270  4637 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:13.747411  4546 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:13.773644 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:13.774243 29145 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:13.778861  3846 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:13.775185  4627 consensus_queue.cc:237] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:13.786489  4641 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.824s	user 0.014s	sys 0.001s
I20260709 13:07:13.788355 29141 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:13.789685 29140 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
I20260709 13:07:13.789837  4546 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:13.778348  4718 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:13.794073  4546 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:13.797055  4637 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:13.807781 29145 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:13.810480 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:13.811168 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.812678 29148 raft_consensus.cc:1275] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:13.813875 29148 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:13.816874 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.811754 29143 raft_consensus.cc:1275] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:07:13.819515 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.820441 29141 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
W20260709 13:07:13.822247 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 90763us
I20260709 13:07:13.823186 29874 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:13.840037 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.852473 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.847286 29148 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:13.855386 29147 raft_consensus.cc:2368] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:13.856985 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.857779 29148 raft_consensus.cc:2393] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 21: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:13.858523 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.858836 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.859509 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:13.860004  4646 raft_consensus.cc:3055] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:13.860369  4646 raft_consensus.cc:740] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:13.860605 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:13.861411 29145 raft_consensus.cc:2393] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 18: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:13.861202  4646 consensus_queue.cc:260] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:13.859748 29147 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 28 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 13:07:13.863354  4704 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:13.863896  4704 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:13.863981 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:13.864774  4704 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.872627 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:13.852777  4723 consensus_queue.cc:1059] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:07:13.873847  4632 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:13.877563 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 54492.9us
I20260709 13:07:13.894347  4627 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:13.876523  4704 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:13.897591  4703 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:13.873968 29906 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
W20260709 13:07:13.898775  4746 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.057s	user 0.001s	sys 0.002s
I20260709 13:07:13.898916  4657 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24.
I20260709 13:07:13.904462 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:13.863456 29139 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:13.931321 29218 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:13.935313 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:13.937263 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:13.939558 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:13.913160  4350 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:07:13.940207 29219 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:07:13.941015  4350 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:13.941490  4350 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:13.942138 29874 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:13.943007  4546 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:13.942569  4350 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:13.943413  4546 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.944005  4546 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:13.935700 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:13.944268  4652 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 13:07:13.945168  4723 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:13.945595  4723 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.947088 29218 raft_consensus.cc:2368] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
I20260709 13:07:13.923080  4627 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:13.950554 29215 raft_consensus.cc:1240] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:07:14.036067 29146 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:07:13.946036  4723 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:07:14.051308 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.051832 29113 leader_election.cc:400] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
I20260709 13:07:14.052522 29113 leader_election.cc:403] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:07:14.044332  4627 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:14.061148 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.061934 29147 raft_consensus.cc:2368] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21.
I20260709 13:07:14.062772 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.063421 29148 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:07:14.064314 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.052505  4350 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:14.068159 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.064878 29148 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:14.070003 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.072069 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.072700 29148 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:14.057209  4632 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
I20260709 13:07:14.073604 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.074188 29145 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:07:14.080950 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 642100us
W20260709 13:07:14.082594 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.083608  4650 raft_consensus.cc:493] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.058228  4743 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.154s
I20260709 13:07:14.060222  4705 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.084080 29216 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:14.085155  4535 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:14.085474 29221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3454) took 1211 ms. Trace:
W20260709 13:07:14.086931 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.079465  4725 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:14.089998 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:14.091890 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:14.092514 29188 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.738s (SENT) [suppressed 38 similar messages]
I20260709 13:07:14.088855  4703 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:14.104722 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:14.101541 29221 rpcz_store.cc:276] 0709 13:07:12.873761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.084983 (+211222us) service_pool.cc:224] Handling call
0709 13:07:13.118693 (+ 33710us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:13.141561 (+ 22868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.141592 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:13.163120 (+ 21528us) log.cc:844] Serialized 52 byte log entry
0709 13:07:13.383358 (+220238us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:13.506662 (+123304us) log.cc:844] Serialized 14 byte log entry
0709 13:07:14.084869 (+578207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.085012 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.085080 (+    68us) raft_consensus.cc:1685] finished
0709 13:07:14.085094 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.085401 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":587752,"prepare.queue_time_us":63820,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2727,"raft.queue_time_us":581693,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":250,"spinlock_wait_cycles":116608,"thread_start_us":251639,"threads_started":2,"wal-append.queue_time_us":368884}
I20260709 13:07:14.110198 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.151778 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.155787 29145 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:14.163290 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.167845 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.168759 29145 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:14.166179 29147 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:14.170555 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.171058 29147 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:14.174057 29073 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47974 (request call id 3294) took 1560 ms. Trace:
I20260709 13:07:14.174621 29073 rpcz_store.cc:276] 0709 13:07:12.614147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.632378 (+ 18231us) service_pool.cc:224] Handling call
0709 13:07:14.173768 (+1541390us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54003219d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:14.173978 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457693568}
I20260709 13:07:14.106380  4632 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 13:07:14.101848  4705 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:14.099766  4650 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:13.373741 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:14.212103 29145 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:14.215128 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.217149 29145 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:07:14.218971 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.227026 29145 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:14.229179 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 3323) took 1354 ms. Trace:
I20260709 13:07:14.232571 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.234102 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.230190 29222 rpcz_store.cc:276] 0709 13:07:12.868912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:12.885741 (+ 16829us) service_pool.cc:224] Handling call
0709 13:07:14.222773 (+1337032us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b540023c654. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:14.222886 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043431680}
I20260709 13:07:14.234982 29145 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 27.
W20260709 13:07:14.234838 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 131795us
I20260709 13:07:14.236423 29186 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:14.237954 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.241003 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.211023  4743 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:14.219375  4632 consensus_queue.cc:1243] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 24.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:07:14.249852 29145 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:14.223587  4538 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.261989 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:14.222995  4543 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:14.278996 29903 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:14.280206 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.280917 29145 raft_consensus.cc:2376] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24. Re-sending same reply.
I20260709 13:07:14.252630  4565 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:14.287999 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.288688 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.114041  4725 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.296643 29143 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:07:14.305620  4637 raft_consensus.cc:3055] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:14.306217  4637 raft_consensus.cc:740] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:14.307745 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.309399 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.310798 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.311735 29143 raft_consensus.cc:2410] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:07:14.324997  4017 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:07:14.325920 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.325858  4017 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.329180  4017 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:14.329238  4714 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.330652 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.331821 29146 raft_consensus.cc:2410] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:07:14.337241 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:14.339792 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.293322  4565 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:14.342104 29038 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:14.282182  4632 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.053s
I20260709 13:07:14.278764  4538 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.352536 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.348181 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.329881  4714 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.284741  4543 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:14.349638 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 542600us
I20260709 13:07:14.354115 29146 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:14.277325  4743 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:14.344887  4748 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.248s	user 0.003s	sys 0.001s
W20260709 13:07:14.361232  4751 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:07:14.363494  4714 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:14.364583  4650 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:14.372241 29214 raft_consensus.cc:2410] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:07:14.379679 29148 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
W20260709 13:07:14.357318  4739 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.267s	user 0.002s	sys 0.000s
I20260709 13:07:14.348996  4565 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:14.385080 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:14.385736 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:14.388628 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 51477.2us
I20260709 13:07:14.389015 29148 raft_consensus.cc:2410] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 26 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 13:07:14.390164 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.390884 29148 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:07:14.392971 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:14.393913 29073 raft_consensus.cc:2368] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 24. Current term is 25.
I20260709 13:07:14.391731 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:14.397642 29038 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:14.399339 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:14.406482 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.382786  4629 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:14.382757  4699 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:14.411249 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:14.412485 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:14.414471 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:14.441764 29143 raft_consensus.cc:2368] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21.
I20260709 13:07:14.407964  4730 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:14.445848 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:14.446208 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 181847us
I20260709 13:07:14.410547  4714 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:07:14.448855 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:14.454034  4657 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:14.454824  4657 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.461673  4705 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:14.462354  4705 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:14.462833  4705 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.466123 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3273) took 1005 ms. Trace:
I20260709 13:07:14.466549 29139 rpcz_store.cc:276] 0709 13:07:13.461091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.461245 (+   154us) service_pool.cc:224] Handling call
0709 13:07:13.461650 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:13.931883 (+470233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.465870 (+533987us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54001bbcd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:14.466066 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919072000}
I20260709 13:07:14.407816  4377 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:07:14.476080 29139 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:14.476848 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:14.477495 29139 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:07:14.478791 29139 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:07:14.469529 29146 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
W20260709 13:07:14.471410 29113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.446410 29148 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:14.472491  4657 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:14.474403  4538 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:14.475525 29140 raft_consensus.cc:1240] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:14.475698  4241 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:14.486714 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 394199us
I20260709 13:07:14.445016  4730 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.498312 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3311) took 1081 ms. Trace:
I20260709 13:07:14.392107  4546 raft_consensus.cc:515] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.498724 29068 rpcz_store.cc:276] 0709 13:07:13.416549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.416720 (+   171us) service_pool.cc:224] Handling call
0709 13:07:13.417157 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:13.442278 (+ 25121us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.442304 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:13.471558 (+ 29254us) log.cc:844] Serialized 26 byte log entry
0709 13:07:13.847502 (+375944us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:07:13.847546 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:13.847615 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.498098 (+650483us) raft_consensus.cc:1685] finished
0709 13:07:14.498120 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.498250 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33182,"raft.queue_time_us":25244,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":22102,"thread_start_us":337881,"threads_started":1,"wal-append.queue_time_us":551615}
I20260709 13:07:14.504328 29143 raft_consensus.cc:1240] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:14.505797 29143 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:07:14.506999 29143 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:07:14.510150  4546 leader_election.cc:290] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:14.510767 29904 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 25: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:07:14.494766 29188 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 2.225s (SENT)
I20260709 13:07:14.515462 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:14.515825  4637 consensus_queue.cc:260] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:14.521337 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.524446 29069 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 1245) took 1099 ms. Trace:
I20260709 13:07:14.481386  4699 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:14.543771 29188 leader_election.cc:400] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 26. Current term is 27.
I20260709 13:07:14.544520 29188 leader_election.cc:403] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:14.517882  4730 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:14.553948 29219 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:14.554028 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:14.556070 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:14.524293  4650 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:14.557430 29219 raft_consensus.cc:2410] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 13:07:14.524295  4637 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:14.560214  4752 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:07:14.537240  4743 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:14.560938  4753 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 26. Current term is 27.
W20260709 13:07:14.578264 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.580046 29073 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:07:14.530632 29069 rpcz_store.cc:276] 0709 13:07:13.424723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.462123 (+ 37400us) service_pool.cc:224] Handling call
0709 13:07:13.475210 (+ 13087us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:13.571343 (+ 96133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:13.571400 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:13.574895 (+  3495us) log.cc:844] Serialized 26 byte log entry
0709 13:07:14.034666 (+459771us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:07:14.073209 (+ 38543us) log.cc:844] Serialized 14 byte log entry
0709 13:07:14.315478 (+242269us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:14.328412 (+ 12934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.328502 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:14.517566 (+189064us) raft_consensus.cc:1685] finished
0709 13:07:14.524209 (+  6643us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:14.524379 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":276291,"prepare.queue_time_us":33252,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":36,"raft.queue_time_us":283505,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3032,"spinlock_wait_cycles":4241152,"thread_start_us":417293,"threads_started":2,"wal-append.queue_time_us":371091}
I20260709 13:07:14.570080 29147 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:14.571321 29220 raft_consensus.cc:1240] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:14.574582 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.589215 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.590227 29147 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:07:14.591518 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.594918 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:14.596908 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.598608 29147 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:07:14.586207 29143 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
W20260709 13:07:14.643680  4758 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:07:14.572959  4560 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:14.655100 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 250194us
I20260709 13:07:14.648383 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:07:14.674328 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 147272us
W20260709 13:07:14.667719 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 137392us
I20260709 13:07:14.676446 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:14.678267 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:14.685719 29038 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:14.662039  4705 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:14.687258 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:14.683931 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:14.689721 29185 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:07:14.739044 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 49395.4us
I20260709 13:07:14.682730  4646 raft_consensus.cc:3060] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:14.743696 29219 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:07:14.673756  4708 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:07:14.662397  4560 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:14.700842 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:14.705144 29111 leader_election.cc:400] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 24. Current term is 25.
I20260709 13:07:14.700359 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:14.754686 29068 raft_consensus.cc:2368] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
I20260709 13:07:14.704614 29070 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23. Ops: []
W20260709 13:07:14.710060 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:14.719966 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:14.721271 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:14.723819 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:14.691504 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:14.731880 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:14.696550 29073 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
W20260709 13:07:14.681962  4723 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.616s	user 0.006s	sys 0.014s
I20260709 13:07:14.739817 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:14.690915  4241 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:14.740548  3846 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:14.796005  3846 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:14.697073  4704 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:14.793280 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:14.687484  4761 consensus_queue.cc:1059] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.070s
I20260709 13:07:14.812696 29222 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:14.814021 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:14.784828 29067 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 24: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:14.816691 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:14.819806 29065 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:07:14.785046 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:14.787232 29217 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:07:14.757531 29903 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:07:14.796448  3846 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:14.748925 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:07:14.796692 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 105269us
I20260709 13:07:14.796871 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:14.688118  4725 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:14.799731  4753 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:14.800212  4757 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:07:14.741564  4377 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:14.839956 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:14.805704  4560 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:14.806449 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 101467us
I20260709 13:07:14.841887 29217 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:14.843554 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:14.795667  4476 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
W20260709 13:07:14.797978 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 58259.3us
I20260709 13:07:14.680444  4699 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:14.683606  4538 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:07:14.713403  4543 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:14.833443 29068 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:07:14.848874  4763 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:14.831395 29147 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:07:14.838853 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:14.837622 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:14.849989 29064 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:14.850220 29065 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:07:14.804611 29220 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 13:07:14.850839 29220 pending_rounds.cc:85] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 14
W20260709 13:07:14.695578  4350 consensus_meta.cc:287] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.627s	user 0.007s	sys 0.001s
I20260709 13:07:14.851094 29220 pending_rounds.cc:107] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 22.15
W20260709 13:07:14.843400 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 50237.3us
I20260709 13:07:14.836548  4661 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:14.851982 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:14.851392 29220 raft_consensus.cc:2889] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:14.836786  4481 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:14.864189  4481 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:14.865875  4627 consensus_meta.cc:287] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.799s	user 0.011s	sys 0.000s
I20260709 13:07:14.866408  4481 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:14.866329  4627 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.869029  4627 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:14.847783  4546 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:14.870198  4546 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:14.870577  4546 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:14.866322 29217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:07:14.873643  4705 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:14.873732 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:14.875161 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:14.859954 29213 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:07:14.839226  4718 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:14.862389  3846 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:14.884567  3846 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:14.808338  4723 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.847105  4206 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:14.896924 29143 raft_consensus.cc:1240] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 13:07:14.837484  4650 consensus_queue.cc:1048] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:07:14.832510  4565 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.853857  4767 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:14.904274 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:14.899607 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:14.900457 29222 raft_consensus.cc:2410] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 13:07:14.905493 29147 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
I20260709 13:07:14.882654  4579 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:14.913749 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:14.844702  4377 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.900264  4761 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:14.876507  4661 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.857520  4763 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:14.913946  4765 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.065s	user 0.001s	sys 0.001s
I20260709 13:07:14.985823 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:14.991286 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:14.987424 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 87927.8us
I20260709 13:07:14.896517  4206 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.893748  4718 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:14.998816 29111 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:14.979676  4658 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.004053 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:14.984297  4636 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:14.911432  4767 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:15.013667 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 184347us
I20260709 13:07:15.001663  4761 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:14.997984  4579 consensus_queue.cc:1048] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
W20260709 13:07:15.026656 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 195525us
I20260709 13:07:15.031658 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.033640 29064 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 18: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:15.021114  4241 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:15.019600  4753 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:15.010524  4658 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:15.050107 29141 raft_consensus.cc:1275] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:07:15.050422 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:15.036303  4761 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:15.025650  4543 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.024s
I20260709 13:07:15.022722  4636 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:15.062276 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 213117us
I20260709 13:07:15.056303  4704 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:15.073215  4761 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:15.075083  4543 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.075136 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3377) took 1009 ms. Trace:
I20260709 13:07:15.124095 29139 rpcz_store.cc:276] 0709 13:07:14.065218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.479474 (+414256us) service_pool.cc:224] Handling call
0709 13:07:14.479738 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:14.491736 (+ 11998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.491785 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:14.496459 (+  4674us) log.cc:844] Serialized 26 byte log entry
0709 13:07:14.699733 (+203274us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b180033ff40. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:07:14.971379 (+271646us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:14.971431 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.971532 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.074806 (+103274us) raft_consensus.cc:1685] finished
0709 13:07:15.074829 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.075071 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69577,"raft.queue_time_us":60777,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":24648,"spinlock_wait_cycles":2886016,"thread_start_us":271412,"threads_started":1,"wal-append.queue_time_us":467267}
I20260709 13:07:15.076370  4753 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:15.077792  4771 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:07:15.084509  4770 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:07:15.084403  4377 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.085104  4241 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.087152  4636 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.088105  4763 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.088598  4565 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.088976  4206 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.090615  4658 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:15.091260 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3378) took 1025 ms. Trace:
I20260709 13:07:15.093284  4476 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.118s
I20260709 13:07:15.063633  4723 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:15.169083 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 185305us
W20260709 13:07:15.172812 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 168373us
I20260709 13:07:15.180450 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:15.190748 29217 raft_consensus.cc:606] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:15.191958  4657 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:15.182075 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 194021us
I20260709 13:07:15.169129  4661 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.118537  4252 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:15.219766 29146 rpcz_store.cc:276] 0709 13:07:14.065458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.481529 (+416071us) service_pool.cc:224] Handling call
0709 13:07:14.481825 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:14.673751 (+191926us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:07:14.673803 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:14.683015 (+  9212us) log.cc:844] Serialized 26 byte log entry
0709 13:07:15.000187 (+317172us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:15.000233 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.000302 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.090991 (+ 90689us) raft_consensus.cc:1685] finished
0709 13:07:15.091013 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.091190 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75872,"raft.queue_time_us":67373,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":7019,"thread_start_us":371241,"threads_started":2,"wal-append.queue_time_us":284632}
W20260709 13:07:15.093605 29875 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:15.225616 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 175252us
I20260709 13:07:15.225656 29111 leader_election.cc:400] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
I20260709 13:07:15.226197 29222 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:07:15.238708 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:15.239851 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:15.240758 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:15.169703  4543 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.218845  4635 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:07:15.253623 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:15.244992 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:15.243516 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:15.257681 29139 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:15.258081 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:15.259459 29871 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:15.245528  4718 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.265163  4263 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:15.235971  4565 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:15.284931 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:15.285313 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:15.285755 29222 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 18.
I20260709 13:07:15.286819 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:15.287603 29222 raft_consensus.cc:3060] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:15.289194 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:15.289987 29911 raft_consensus.cc:2368] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22.
I20260709 13:07:15.248754  4743 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:15.267722  4263 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:15.245476  4767 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.308470 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:15.297154 29875 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:15.232402  4783 consensus_queue.cc:1048] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:07:15.313021 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:15.237102  4657 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:15.318663 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3384) took 1246 ms. Trace:
I20260709 13:07:15.322243 29139 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:15.321900 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:15.325160 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:15.288710  4565 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:15.322675 29140 rpcz_store.cc:276] 0709 13:07:14.072622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.491482 (+418860us) service_pool.cc:224] Handling call
0709 13:07:14.512444 (+ 20962us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:14.547334 (+ 34890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.547379 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:14.550824 (+  3445us) log.cc:844] Serialized 52 byte log entry
0709 13:07:14.893742 (+342918us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:07:14.893799 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:14.893889 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.318422 (+424533us) raft_consensus.cc:1685] finished
0709 13:07:15.318448 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.318594 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38861,"raft.queue_time_us":93509,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":50889,"thread_start_us":329412,"threads_started":1,"wal-append.queue_time_us":433576}
W20260709 13:07:15.332973 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 93203.4us
I20260709 13:07:15.335310 29188 leader_election.cc:400] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:15.335741 29188 leader_election.cc:403] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:15.244863  4252 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:15.256009  4546 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:15.354048 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:15.311296  4703 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:15.379549 29140 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:07:15.289204  4802 consensus_queue.cc:1059] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
W20260709 13:07:15.379184 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 119701us
I20260709 13:07:15.381160 29139 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:07:15.257591  4704 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.346558  4737 raft_consensus.cc:2804] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:15.339268  4565 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:15.390105 29139 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:07:15.377305  4761 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.376257  4781 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:15.310979  4800 consensus_queue.cc:1059] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 13:07:15.321301  4743 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:15.395035 29222 raft_consensus.cc:2468] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:15.400279 29139 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:07:15.416292 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:15.375810  4578 consensus_queue.cc:1048] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:07:15.430920 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:15.434266 29222 raft_consensus.cc:3060] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:15.391636  4811 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:15.420629  4781 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:15.444505  4797 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 13:07:15.447963  4809 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:07:15.452602  4560 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:15.429929  4800 consensus_queue.cc:1243] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:07:15.427548  4783 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:15.455340  4811 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.458830  4546 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:15.459740  4743 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:15.460538  4811 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:15.426069  4766 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:07:15.454113  4779 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.212s	user 0.001s	sys 0.001s
W20260709 13:07:15.462915  4730 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.898s	user 0.010s	sys 0.000s
I20260709 13:07:15.478457 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:15.413007  4761 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:15.510366 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:15.507692 29222 raft_consensus.cc:2468] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
W20260709 13:07:15.533284 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 148009us
I20260709 13:07:15.463593  4783 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:15.560323 29213 raft_consensus.cc:1240] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:15.572989 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:15.561853 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 188302us
I20260709 13:07:15.575227 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:15.576116 29185 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.037s	sys 0.033s
W20260709 13:07:15.576997 29185 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.037s	sys 0.033s
I20260709 13:07:15.510795  4730 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.549520  4783 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:07:15.579397 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 244622us
I20260709 13:07:15.548177  4543 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:15.577406 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 513377us
I20260709 13:07:15.580935 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:15.573361  4779 log.cc:927] Time spent T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:07:15.586916 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:15.588721 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 303872us
I20260709 13:07:15.590718 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:15.585497 29875 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:15.646400 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:15.646989 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 60522.9us
W20260709 13:07:15.647482 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 60696us
I20260709 13:07:15.576365  4737 raft_consensus.cc:697] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:15.648453 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:15.652055 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:15.653823 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.654917 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:15.657366  4761 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:15.658073  4820 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:15.658691  4820 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:15.662388  4781 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:15.663151  4252 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:15.665699  4730 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:15.665936 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3380) took 1600 ms. Trace:
I20260709 13:07:15.668175  4657 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:15.647446  4752 log.cc:927] Time spent T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:07:15.669421  4820 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.671609  4661 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:15.672169  4661 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:15.673327 29188 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:15.674244  4661 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.674476 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:15.663266  4637 consensus_meta.cc:287] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.089s	user 0.014s	sys 0.000s
I20260709 13:07:15.666348 29148 rpcz_store.cc:276] 0709 13:07:14.065975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.483003 (+417028us) service_pool.cc:224] Handling call
0709 13:07:14.483290 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:14.748054 (+264764us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b18004af560. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f7836b05060 00007f7836b05bfb 00007f7836b1064b
0709 13:07:14.748188 (+   134us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:07:15.218514 (+470326us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.422284 (+203770us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:15.422820 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:07:15.422981 (+   161us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:15.423024 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.423116 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.665608 (+242492us) raft_consensus.cc:1685] finished
0709 13:07:15.665710 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.665865 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79954,"prepare.queue_time_us":421993,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":92679,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":85127,"spinlock_wait_cycles":2846976,"thread_start_us":587024,"threads_started":3,"wal-append.queue_time_us":307935}
I20260709 13:07:15.676055 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:15.654213 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:15.685487 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.694454 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.695436 29068 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:15.696650 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.699457 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:15.686424 29073 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:15.701411 29188 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:15.709438 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.710227 29066 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:15.711028  4822 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:15.711357 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:15.712663  4821 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:07:15.698943  4820 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:15.716985 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:15.720161  4778 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:15.720400 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:15.720700  4778 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:15.711659  4822 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:15.722005  4822 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:15.721611  4778 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.726073  4737 consensus_queue.cc:237] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:15.728597  4821 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.729278 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:15.729146  4821 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.731282  4821 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:15.732117 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:15.733444  4821 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.735069  4821 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.736570 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.675031  4780 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.738288  4785 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.738847  4785 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.738768  4780 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.737458  4821 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:15.741837  4780 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:15.742084  4820 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:07:15.748343 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 72360.5us
I20260709 13:07:15.749498 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:07:15.752823 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 98695.2us
I20260709 13:07:15.763306 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:15.764262 29222 raft_consensus.cc:2410] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:07:15.651460  4703 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.765133 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:15.766844 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:15.651397  4718 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.769194 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:15.765058 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.777266 29222 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:15.778451 29222 raft_consensus.cc:1240] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:07:15.778411 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:15.652459  4658 raft_consensus.cc:2804] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:15.700646  4657 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:15.757627  4780 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:15.652113  4627 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.119s
I20260709 13:07:15.757118  4263 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:07:15.756085  4781 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:07:15.790853 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:15.651743  4543 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.118s
I20260709 13:07:15.756601  4637 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.768507  4820 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:15.761147  4565 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.809514  4643 raft_consensus.cc:3055] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:15.821398  4781 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:15.808981  4760 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.809263  4761 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:15.827457  4637 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.810868  4723 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:15.844302 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 3468) took 1750 ms. Trace:
I20260709 13:07:15.845283 29213 raft_consensus.cc:1217] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Deduplicated request from leader. Original: 20.16->[22.17-23.18]   Dedup: 22.17->[23.18-23.18]
I20260709 13:07:15.772787  4703 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.826831  4820 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:15.844833 29214 rpcz_store.cc:276] 0709 13:07:14.093660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.093813 (+   153us) service_pool.cc:224] Handling call
0709 13:07:15.844006 (+1750193us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54002f58d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:15.844216 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596036480}
I20260709 13:07:15.872272 29213 raft_consensus.cc:1240] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: [22.17-23.18]
I20260709 13:07:15.828019  4780 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:15.813206  4657 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.810773  4718 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:15.840224  4781 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:15.833711  4643 raft_consensus.cc:740] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:15.814917  4658 raft_consensus.cc:697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:15.906059 29213 raft_consensus.cc:1240] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:15.907574 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:15.902365 29216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3355) took 1179 ms. Trace:
I20260709 13:07:15.911553 29903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54892 (request call id 1296) took 1012 ms. Trace:
I20260709 13:07:15.777602  4822 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.852240  4830 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:15.887537  4637 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:15.860114  4792 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.909034 29216 rpcz_store.cc:276] 0709 13:07:14.723303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.479741 (+756438us) service_pool.cc:224] Handling call
0709 13:07:15.480029 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.480242 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:07:15.480660 (+   418us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.896714 (+416054us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.901870 (+  5156us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:15.901944 (+    74us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:15.902019 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.902114 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.902280 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4973,"prepare.queue_time_us":3953,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":6160,"spinlock_wait_cycles":6938368,"thread_start_us":410548,"threads_started":1}
I20260709 13:07:15.911970 29903 rpcz_store.cc:276] 0709 13:07:14.898784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.898920 (+   136us) service_pool.cc:224] Handling call
0709 13:07:14.899279 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:15.174997 (+275718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:15.175075 (+    78us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:15.176240 (+  1165us) log.cc:844] Serialized 26 byte log entry
0709 13:07:15.201023 (+ 24783us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:07:15.201231 (+   208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.902562 (+701331us) spinlock_profiling.cc:243] Waited 87.5 ms on lock 0x7b54005bc554. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:15.902635 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:15.902703 (+    68us) raft_consensus.cc:1685] finished
0709 13:07:15.902726 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:15.911483 (+  8757us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2266,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":4607,"spinlock_wait_cycles":201243392,"thread_start_us":211067,"threads_started":2,"wal-append.queue_time_us":276237}
I20260709 13:07:15.932052 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.932298 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.903016  4767 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:15.935511 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.846289  4723 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:15.936985 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:15.937168 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:15.937834 29065 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 26: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:15.940119  4718 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.940529  4476 raft_consensus.cc:3055] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:15.940759  4718 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:15.941020  4476 raft_consensus.cc:740] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:15.941596  4718 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.942142  4476 consensus_queue.cc:260] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.943315  4476 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:15.844137  4760 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.945791  4760 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:15.949115  4565 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:15.950443  4787 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.521s	user 0.001s	sys 0.000s
I20260709 13:07:15.950219  4658 consensus_queue.cc:237] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:15.953454  4707 log.cc:927] Time spent T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 1.666s	user 0.010s	sys 0.000s
I20260709 13:07:15.953760  4703 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:15.953974 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3344) took 1235 ms. Trace:
I20260709 13:07:15.954346 29220 rpcz_store.cc:276] 0709 13:07:14.718120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.718285 (+   165us) service_pool.cc:224] Handling call
0709 13:07:14.742282 (+ 23997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:14.859468 (+117186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.953737 (+1094269us) spinlock_profiling.cc:243] Waited 61.9 ms on lock 0x7b54003cd2d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:15.953913 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142455680}
I20260709 13:07:15.891258  4820 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:15.956830  4767 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:15.957439  4767 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:15.961535  4723 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:15.965301  4699 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:15.966099  4699 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:15.895133  4761 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.930171  4830 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:15.969316  4761 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.936165  4792 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:15.970863  4830 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:15.971372  4761 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:15.972790  4792 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:15.973690 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:15.934145  4661 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:15.954818  4646 consensus_meta.cc:287] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 1.169s	user 0.014s	sys 0.009s
I20260709 13:07:15.977267  4657 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:15.978006 29144 raft_consensus.cc:1275] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 13:07:15.978469 29144 pending_rounds.cc:85] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 15
I20260709 13:07:15.978672 29144 pending_rounds.cc:107] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 13:07:15.978873 29144 raft_consensus.cc:2889] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:15.980679 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3296) took 2169 ms. Trace:
I20260709 13:07:15.981019 29144 rpcz_store.cc:276] 0709 13:07:13.811210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.812041 (+   831us) service_pool.cc:224] Handling call
0709 13:07:14.754439 (+942398us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b5800003098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8eee 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:07:14.754592 (+   153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:15.979288 (+1224696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:15.980416 (+  1128us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400342454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:15.980591 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2827456512}
I20260709 13:07:15.891299  4780 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:15.922758 29213 raft_consensus.cc:1275] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:07:15.983980 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.984826 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:15.985845 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:15.986609 29073 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:15.920986 29214 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:07:15.987418 29222 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:07:15.987520 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:15.933195  4367 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.905743  4535 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:15.998376 29216 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 22: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:15.975333  4661 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:15.999190 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 249779us
I20260709 13:07:15.997113 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.020532 29039 leader_election.cc:400] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21.
W20260709 13:07:16.014000 29146 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.003s	sys 0.017s
I20260709 13:07:16.018471 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:16.022716 29146 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.004s	sys 0.017s
I20260709 13:07:16.023480 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.024991 29214 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 24. Current term is 25. Ops: []
W20260709 13:07:16.018605  4704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.004s	sys 0.017s
W20260709 13:07:15.989204  4804 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.077s	sys 0.080s
I20260709 13:07:15.923218  4627 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:15.986133  4780 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:15.892683  4783 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.039331 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.059576 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.064419 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:16.030158  4804 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.077s	sys 0.081s
W20260709 13:07:16.032250  4704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.005s	sys 0.018s
I20260709 13:07:16.085875  4657 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:16.086375 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:16.087525  4646 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:16.087909  4646 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
W20260709 13:07:16.090975 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 331766us
I20260709 13:07:16.092819 29214 raft_consensus.cc:1240] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:16.094295 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.095518 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.045707  4637 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:07:16.106084 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:16.087946 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 297181us
I20260709 13:07:16.111652 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:16.018787  4367 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.047979  4703 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:16.078509  4718 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:16.123745  4718 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:16.124598  4822 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:16.019308  4643 consensus_queue.cc:260] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.128710  4643 raft_consensus.cc:3060] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:16.118072  4646 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.132416  4481 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:16.132990  4481 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:16.133175  4646 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:16.134203  4481 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.135828  4699 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.097426  4760 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:07:16.045250  4627 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:16.136090  4657 consensus_queue.cc:1175] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" responder_term: 24 status { last_received { term: 24 index: 18 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 23. Current term is 24. Ops: [22.17-23.18]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:16.137485  4783 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:16.139123  4627 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:16.098110  4830 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.081049  4792 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:16.145331 29903 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:07:16.089907  4546 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:16.121699  4367 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.120384  4843 consensus_queue.cc:1059] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:07:16.147717 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:16.155368 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.153690 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:16.156260 29216 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:07:16.081836  4825 consensus_queue.cc:1048] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.059s
I20260709 13:07:16.158725 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.170961 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.100114  4476 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 13:07:16.145880  4760 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:16.175740 29222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3359) took 1137 ms. Trace:
I20260709 13:07:16.176239 29222 rpcz_store.cc:276] 0709 13:07:15.038685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:15.786220 (+747535us) service_pool.cc:224] Handling call
0709 13:07:15.786556 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:15.998216 (+211660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.175479 (+177263us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54001338d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:16.175680 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452825856}
I20260709 13:07:16.115665  4637 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.140383  4830 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.195123 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:16.173597  4783 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:16.142590  4703 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.191962  4704 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.158587  4546 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.150061  4792 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.196512  4843 raft_consensus.cc:3055] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:16.191359  4367 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:16.220009  4760 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:16.220386  4783 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:16.263127 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.234838  4704 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.214016  4699 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:07:16.236186  4843 raft_consensus.cc:740] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:16.296947 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 150401us
I20260709 13:07:16.236397  4844 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:16.252404  4783 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:16.308532 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.240140  4781 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.293653  4661 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:16.233032  4767 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.304949 29111 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:16.297282  4822 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.246863  4367 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:16.343856 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 311811us
W20260709 13:07:16.353524 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.306877  4844 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:16.356710  4844 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:16.298910  4760 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.370477 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:16.375511 29903 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:07:16.338328  4476 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.345923  4761 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:16.377296 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 182289us
W20260709 13:07:16.371929 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet ea48540c1af24bf3b55edec5c52ec133: Illegal state: Replica 730186a8c70f43f2a57d80163bbc37d2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 24 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:16.380299 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:16.381610 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:16.380940 29911 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:16.390820 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:16.392196 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:16.365895  4845 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.348763  4543 consensus_queue.cc:1048] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 13:07:16.386911 29037 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:16.347411  4822 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.377070  4846 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:16.397002  4761 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.391451  4699 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:16.385056  4476 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.433012  4830 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:16.401659  4637 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:16.421736  4704 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:16.374148  4843 consensus_queue.cc:260] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.470929 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.421476  4820 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.483508 29222 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:16.442272  4703 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:16.432960  4846 consensus_queue.cc:1059] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:07:16.458442  4792 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.397943  4845 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.478458 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.490836 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.497352 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.500599 29220 raft_consensus.cc:2393] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 28: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:16.449270  4761 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.508163 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:16.511461 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.510402 29070 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:16.456641  4699 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:16.505595  4850 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:07:16.523084 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.506279  4854 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:16.531898  4854 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.532491  4854 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:16.535326 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.539610 29064 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:16.506415  4843 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:16.529455 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 142626us
I20260709 13:07:16.549465 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.492161  4339 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:16.544512 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:16.514864  4855 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:16.550149 29064 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 26.
W20260709 13:07:16.537719  4849 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:07:16.526388  4781 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:16.562532 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.536448  4367 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:16.574121  4780 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.575517 29068 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19. Ops: []
I20260709 13:07:16.576740  4820 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.578454  4822 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.585758  4845 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.597003  4864 raft_consensus.cc:3055] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:07:16.597682  4864 raft_consensus.cc:740] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:16.598805  4864 consensus_queue.cc:260] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.600653  4854 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:16.601262  4847 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
W20260709 13:07:16.622462 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.536753  4767 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.628027 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:16.634150 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 242032us
I20260709 13:07:16.635823 29069 raft_consensus.cc:1240] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:07:16.636480 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:44946 (request call id 55) took 1774 ms. Trace:
W20260709 13:07:16.621353  4853 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:07:16.640509  4854 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.640795  4476 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.646586  4699 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:16.637030 29217 rpcz_store.cc:276] 0709 13:07:14.855699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.866127 (+ 10428us) service_pool.cc:224] Handling call
0709 13:07:16.621434 (+1755307us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54000bde54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b07853 00007f783bad560a 00007f783505b7d4 00007f783505b762 00007f783505b691 00007f783505b60c 00007f783505a8c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:16.621579 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733682560}
I20260709 13:07:16.655009 29218 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:07:16.655336 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:16.630451  4643 consensus_meta.cc:287] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.501s	user 0.014s	sys 0.000s
W20260709 13:07:16.657269 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 85889.2us
I20260709 13:07:16.657532 29215 raft_consensus.cc:2393] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 25: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:16.635105 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:07:16.634402  4723 consensus_meta.cc:287] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.672s	user 0.010s	sys 0.004s
W20260709 13:07:16.663746 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 316633us
W20260709 13:07:16.669795 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.673866 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:16.677644 29911 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 24: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:16.678195 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:16.679890 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:16.684955 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.681586 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:16.681358  4883 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:16.681499  4367 consensus_queue.cc:1059] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 13:07:16.698848  4883 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:16.699168  4855 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:16.698787 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:16.699571  4367 raft_consensus.cc:3055] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:16.699733  4855 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:16.700171 29910 raft_consensus.cc:2410] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:07:16.700358  4855 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:16.669778  4825 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:07:16.699126 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:16.707161  4882 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:16.707546  4851 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.402s	user 0.000s	sys 0.002s
I20260709 13:07:16.708451 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:16.681720  4854 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:07:16.709057  4646 consensus_meta.cc:287] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.575s	user 0.014s	sys 0.000s
W20260709 13:07:16.709681  4872 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:07:16.717046 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1428) took 2229 ms. Trace:
I20260709 13:07:16.672135  4723 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.681198  4761 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.721640 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:16.720156 29143 rpcz_store.cc:276] 0709 13:07:14.487836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.610361 (+122525us) service_pool.cc:224] Handling call
0709 13:07:14.660109 (+ 49748us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:14.900719 (+240610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.716727 (+1816008us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54002e8954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:16.716952 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544019328}
I20260709 13:07:16.697291  4699 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 13:07:16.700218  4367 raft_consensus.cc:740] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:16.721402  4780 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:16.698253  4339 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:07:16.724270  4854 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:16.730058  4783 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.740592  4783 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:16.741875 29217 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 24. Current term is 25. Ops: []
W20260709 13:07:16.743057  4873 log.cc:927] Time spent T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.181s	user 0.003s	sys 0.000s
I20260709 13:07:16.717073  4882 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:07:16.745492  4723 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.751607 29070 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:07:16.754385 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:16.712177  4883 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.756472 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:16.756803  4802 raft_consensus.cc:3055] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:07:16.757750 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.758620 29143 raft_consensus.cc:2410] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for term 25 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 13:07:16.757587 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:16.759083  4802 raft_consensus.cc:740] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:16.758788  4761 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:07:16.762046  4802 consensus_queue.cc:260] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.763317  4802 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:16.755179  4825 consensus_queue.cc:1059] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.014s
I20260709 13:07:16.764153  4705 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:16.764591  4705 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:16.764817  4367 consensus_queue.cc:260] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.765496  4705 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.766372  4367 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:16.770463  4882 consensus_queue.cc:1059] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:16.771143  4764 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:16.771502  4882 raft_consensus.cc:3055] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:07:16.749878  4865 log.cc:927] Time spent T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:07:16.771987  4882 raft_consensus.cc:740] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:16.772884  4882 consensus_queue.cc:260] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.774114  4882 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:16.775648  4825 raft_consensus.cc:3055] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:07:16.776083  4825 raft_consensus.cc:740] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:16.777009  4825 consensus_queue.cc:260] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.778407  4825 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:16.794654  4855 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.799599 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.800271 29217 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
W20260709 13:07:16.800603 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44579.8us
I20260709 13:07:16.801805 29112 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:16.804265 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:16.807845  4889 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:16.809440 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:16.810461  4889 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:16.806454 29112 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:16.811935 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 77849.4us
I20260709 13:07:16.821619 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:16.814481  4889 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:16.822580  4889 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:16.811185  4761 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:16.805244  4723 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:16.831326  4844 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.832933  4855 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.833540  4844 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.803508  4760 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.833954 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:16.836076 29065 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:16.838063 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:16.839339  4761 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:16.840937 29875 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:16.843591 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:16.844702 29146 raft_consensus.cc:1240] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:16.845337 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.845937 29064 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:16.846591 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:16.848037 29218 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:07:16.848754 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.840056 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:16.850404 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:16.850539 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:16.850891 29146 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:07:16.851454 29144 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:16.851581 29219 raft_consensus.cc:2393] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 18: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:16.850612 29065 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 13:07:16.804867  4802 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:07:16.851013 29910 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:07:16.855412 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:16.855687 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.856856 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:16.858834 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.861954 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:16.862525 29147 raft_consensus.cc:3060] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:16.864871 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:16.830389  4888 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:07:16.868263 29146 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:16.853559  4855 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:16.853950  4780 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:16.871178 29112 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:16.871852 29141 raft_consensus.cc:1275] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:07:16.875166 29147 raft_consensus.cc:2468] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:07:16.865815  4017 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:07:16.863211  4760 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.838855  4843 consensus_queue.cc:1059] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:07:16.864248  4882 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:16.882980  4816 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:16.883268  4834 consensus_queue.cc:1059] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 13:07:16.884289  4834 raft_consensus.cc:3055] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:16.881906  4760 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.889199  4867 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:07:16.880528  4476 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:07:16.902649 29875 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:16.884627  4834 raft_consensus.cc:740] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:16.929804  4834 consensus_queue.cc:260] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.931299  4834 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:16.912035  4826 consensus_queue.cc:1059] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:07:16.935060  4826 raft_consensus.cc:3055] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:16.935429  4826 raft_consensus.cc:740] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:16.935501  4888 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:16.935947  4888 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:16.913254  4760 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:16.936745  4888 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.938223  4888 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:16.914426 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.919310  4785 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:16.926527  4481 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:16.926664  4816 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:07:16.882052  4252 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.027s
I20260709 13:07:16.926568  4263 raft_consensus.cc:2804] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:16.927187  4579 consensus_queue.cc:1048] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 13:07:16.913304  4890 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:07:16.960525  4263 raft_consensus.cc:697] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:16.887866  4855 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:07:16.961524  4263 consensus_queue.cc:237] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.901435  4476 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:16.962627  4476 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:16.962788 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.946334  4535 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.965131 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.952482  4783 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.940038  4565 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:16.914074  4843 raft_consensus.cc:3055] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:16.965925  4783 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.966387  4843 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:16.966312  4565 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.946394 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:16.968703 29217 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:07:16.956697 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "344aa5f73f0f4124b1512832a0bca2c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:16.969767 29141 raft_consensus.cc:2393] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 25: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:16.970242  4546 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:16.970592 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:16.971342 29141 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:16.972121 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:16.959729 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:16.952132 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 98221us
I20260709 13:07:16.973647 29147 raft_consensus.cc:1275] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 18. (term mismatch)
I20260709 13:07:16.957844 29146 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:07:16.973796 29140 raft_consensus.cc:2410] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:07:16.974076 29147 pending_rounds.cc:85] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 17
I20260709 13:07:16.974354 29147 pending_rounds.cc:107] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 23.18
I20260709 13:07:16.964856  4535 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.974567 29147 raft_consensus.cc:2889] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:16.977118 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.977695 29071 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:16.972071  4538 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:16.978350 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.936460  4826 consensus_queue.cc:260] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.986310 29065 raft_consensus.cc:2393] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 24: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:16.981096 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
W20260709 13:07:16.981251 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:16.981812 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.982519 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.982792 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:16.976032 29113 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:16.983235 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:16.988206 29070 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:16.989308 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.991580 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:16.991844 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:16.992570 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:16.993127 29071 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:16.967377  4843 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:16.996851  4844 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:16.988269 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.997442  4844 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:16.997778 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:16.997231 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3290) took 3223 ms. Trace:
I20260709 13:07:17.000838 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.001488 29140 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:17.001938 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.001425 29070 raft_consensus.cc:2468] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:17.005360 29068 raft_consensus.cc:1240] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:17.005651 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:17.007160 29073 raft_consensus.cc:2393] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 22: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:17.008445 29071 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:17.009267 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:16.952493  4816 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:17.009413 29218 raft_consensus.cc:1275] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:07:17.013764 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.015334 29143 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:17.009954 29140 raft_consensus.cc:2368] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22.
I20260709 13:07:16.979278  4864 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:07:17.011258 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:17.016716 29185 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:17.026407 29069 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:17.026681  3846 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:17.027119  3846 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.027431  3846 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:17.012485 29065 raft_consensus.cc:1240] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:17.002683 29144 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:17.024385 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.028944 29147 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:17.029111 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:16.997174  4565 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:16.999688  4844 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:16.957515  4760 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:17.017525 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:16.999178 29142 rpcz_store.cc:276] 0709 13:07:13.773578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:13.773973 (+   395us) service_pool.cc:224] Handling call
0709 13:07:13.774377 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:14.644733 (+870356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:14.644786 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:14.660944 (+ 16158us) log.cc:844] Serialized 26 byte log entry
0709 13:07:14.985499 (+324555us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:15.176128 (+190629us) log.cc:844] Serialized 14 byte log entry
0709 13:07:15.178560 (+  2432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:16.980935 (+1802375us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b54000ea6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:16.980982 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:16.981027 (+    45us) raft_consensus.cc:1685] finished
0709 13:07:16.997045 (+ 16018us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:16.997179 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78385,"prepare.queue_time_us":226641,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":98847,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":3400,"spinlock_wait_cycles":1820055296,"thread_start_us":419301,"threads_started":2,"wal-append.queue_time_us":376235,"wal-append.run_cpu_time_us":2926,"wal-append.run_wall_time_us":1116033}
I20260709 13:07:17.010957 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.029870 29144 raft_consensus.cc:2376] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19. Re-sending same reply.
I20260709 13:07:17.005139  4785 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:17.010663  4778 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:07:17.039881 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.069375 29140 raft_consensus.cc:2393] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:17.040181 29072 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
I20260709 13:07:17.040625 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.040916 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.044821  4760 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.046372 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.016912  4538 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:17.048552  4816 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.058202 29111 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.081998 29111 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:17.059460 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.034001  4889 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.012993  4657 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.063937 29141 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:17.070498 29147 raft_consensus.cc:2393] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 26: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:17.000332  4635 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:07:17.016716  4802 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:17.013619  4761 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.071979  4826 raft_consensus.cc:3060] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:17.069926 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:17.072062 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:17.078684 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.083423 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:17.025008  4783 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:17.097298 29111 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:07:17.135350 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.016876  4882 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:07:17.119187 29146 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:17.124652  3846 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.126616 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:17.127171 29144 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:17.127265 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:17.130303 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.124848  4723 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:17.156157 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.072634  4844 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.031349  4804 consensus_queue.cc:1048] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.058s
W20260709 13:07:17.135680 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.136732 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:17.137879 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.175872 29141 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:17.139268 29064 raft_consensus.cc:2368] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
W20260709 13:07:17.136744 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 66912.8us
I20260709 13:07:17.131397 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:17.070879  4782 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:07:17.068799  4778 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:17.157008 29147 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 26: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:17.129333  4802 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.245222 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.245517  4802 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:17.159673  3846 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:17.162266 29069 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 26.
I20260709 13:07:17.168202 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:07:17.170243 29221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42964 (request call id 3348) took 2431 ms (client timeout 1501 ms). Trace:
W20260709 13:07:17.252166 29221 rpcz_store.cc:269] 0709 13:07:14.721212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:14.721367 (+   155us) service_pool.cc:224] Handling call
0709 13:07:16.980516 (+2259149us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b540022ed20. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836a84dc0 00007f7836b10e18 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6
0709 13:07:17.152569 (+172053us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54003cd7d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:17.152713 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2968451200}
I20260709 13:07:17.137823  4816 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:17.138984  4761 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.258399  4761 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:17.132184  4643 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.260581  4643 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.142995  4646 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:07:17.151906  4870 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:07:17.215374 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.264186  4646 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:17.264617  4646 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:17.150197  4650 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:17.265282  4650 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:17.265738  4743 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:17.266266  4650 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.266569  4743 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:17.172489  4898 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:07:17.269382 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.099259  4785 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.270130 29147 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:17.270728 29142 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:17.271080 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.271840  4785 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:17.272759  4785 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.273548  4785 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.277558  4889 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:17.280375 29142 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:17.242856  4760 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:17.185246  4844 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.284412  4844 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:17.285133  4778 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.286815  4778 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:17.288651  4476 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.263509  4643 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:17.089334  4538 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.128237  4657 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.271888 29147 raft_consensus.cc:2410] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:07:17.306341 29144 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:17.221016  4723 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.301594 29143 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:17.310487 29141 raft_consensus.cc:2468] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:17.311470 29067 raft_consensus.cc:2410] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:07:17.312521 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.319321 29147 raft_consensus.cc:2410] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:07:17.301556  4761 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:17.321218 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.322459 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:17.311492 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.327164 29143 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:17.328253 29141 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 26: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:17.331802 29903 raft_consensus.cc:2410] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 13:07:17.314782  4725 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:17.342191 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.342984 29143 raft_consensus.cc:2376] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24. Re-sending same reply.
W20260709 13:07:17.344128 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 261096us
W20260709 13:07:17.346120 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:17.352139 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.362514  3846 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:17.363332  3846 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.366232  3846 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:17.320387  4888 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.369138 29186 leader_election.cc:400] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22.
I20260709 13:07:17.369428 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.371888 29186 leader_election.cc:403] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:07:17.372195 29141 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:17.328214  4785 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:17.313691  4723 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.374788 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.375588 29143 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:17.378250  3846 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22.
I20260709 13:07:17.383579 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.380343  4888 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.384423 29141 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:17.388689 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.385707  4538 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:17.392238 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.393007 29141 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 22: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:17.386207  4743 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:17.386333 29186 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.394345 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.400362 29143 raft_consensus.cc:2376] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24. Re-sending same reply.
I20260709 13:07:17.404124  3846 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:17.415262 29146 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:17.449865 29186 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.416661  4785 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.414849  4753 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:17.460371 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:17.463652 29146 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
W20260709 13:07:17.459700 29188 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Timed out: RequestConsensusVote RPC to 127.27.71.130:35337 timed out after 1.738s (SENT)
I20260709 13:07:17.461589 29903 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:17.466279 29188 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:17.473742 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:17.474239 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.475360  3846 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:17.478960 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 342290us
I20260709 13:07:17.465392  4783 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:17.479743 29111 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.482563 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.482039 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.484715  4476 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:17.492230 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:17.495354 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:17.494302 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.498302 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:17.498418 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.500973  4743 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:17.465921  4785 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.490234  4657 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:17.489034  4827 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:17.494323  4783 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:17.493906  4907 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:17.509837 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.512799  3846 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:17.516037  3846 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.501987  4814 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:17.497869  4565 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:17.518158 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:17.518347 29871 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.503968  4826 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:17.514562  4476 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:17.527571 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:17.528340  4565 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.531008  4705 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:17.531248  4565 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:17.527566  4910 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:17.536576  4910 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:17.536955  4910 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:17.537744  4796 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:17.538177  4796 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:17.539012  4796 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.540494 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:17.543025  4902 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.092s	user 0.000s	sys 0.003s
I20260709 13:07:17.520874  4657 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:17.552114 29139 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
I20260709 13:07:17.554965 29910 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:07:17.555968 29907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:17.556581 29037 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:17.558765 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.566323  3846 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:17.609830 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52426.5us
W20260709 13:07:17.611306 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.612262  3846 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:17.612840  3846 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.620878 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:17.579324  4565 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:17.623100  4565 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.626624  4565 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:17.628250 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 69540.1us
I20260709 13:07:17.629191 29111 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.630623 29111 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:17.630694 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:17.632299 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:17.524427  4814 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:17.637818 29188 leader_election.cc:304] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:17.634384 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:17.635883  3846 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:17.640661 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.641678  4725 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:17.642222  4725 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:17.643561 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:17.645296 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:17.619253  4882 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:17.646567 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:17.646850  3846 raft_consensus.cc:2749] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:17.630388  4705 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:17.650782 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.631700  4907 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:17.515170  4827 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.652467 29039 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:17.690719 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.691071 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.691385 29072 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:17.691728 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:17.636262  4657 raft_consensus.cc:2804] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:17.690644 29069 raft_consensus.cc:2393] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:17.635047  4796 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:17.692359 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.696086 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.697026 29071 raft_consensus.cc:2410] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:07:17.640343  4814 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:17.690335 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.694849 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:17.697544 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:17.691758 29073 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
I20260709 13:07:17.647491  4764 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:17.646950  4864 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:17.637866  4723 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:17.651440  4889 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:17.705217 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.706972  3846 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:17.629693  4802 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.718765 29066 raft_consensus.cc:2410] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:07:17.709196 29188 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.710331 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:17.693989  4377 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:07:17.689603  4813 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:17.722658 29112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.723440 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.727831 29111 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.729106 29188 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.706229  4830 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:17.704236  4657 raft_consensus.cc:697] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:17.689906  4783 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:07:17.735430 29140 raft_consensus.cc:2368] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:07:17.736124 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:17.739626 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.717597  4889 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:17.719509  4882 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:17.736224 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.740502 29147 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 18.
I20260709 13:07:17.740644 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:17.689527  4565 consensus_queue.cc:1048] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 13:07:17.692464  4910 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.643200  4725 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.747076 29066 raft_consensus.cc:2393] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 20: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:17.754992  3846 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:17.728963  4753 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:17.757226 29039 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:17.730948  4635 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:17.741345  4377 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:17.733278  4830 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:17.741742  4855 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:07:17.772487  4908 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:07:17.772094  4657 consensus_queue.cc:237] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.773186  4802 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:17.769280  4753 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:17.745985  4889 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:17.746433  4821 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:17.763556  4864 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:07:17.779652 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52616us
W20260709 13:07:17.770394  4646 consensus_meta.cc:287] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.000s
I20260709 13:07:17.764797  4635 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:17.769105  4890 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:17.782907 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.768031  4377 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:17.782261 29188 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.774967  4830 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:17.779363  4821 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:17.782271  4785 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:17.783828  4635 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:17.792601 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.794037 29147 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:17.785938  4377 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:17.795987  4827 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:17.787612  4783 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:17.808149  4783 raft_consensus.cc:697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:17.791654  4821 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:17.783547  4646 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.817579 29147 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:07:17.818528 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.821379  4646 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:17.822248 29141 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:17.825459 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.826306 29141 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
W20260709 13:07:17.827486  4914 log.cc:927] Time spent T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:07:17.827378 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.784765  4753 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:17.834111 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.838294  4888 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:17.850123  4778 consensus_meta.cc:287] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.000s
W20260709 13:07:17.852711 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 124970us
I20260709 13:07:17.854508 29039 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:17.857810 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:17.858541 29070 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:07:17.860373 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:17.859549  4730 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:17.828194  4783 consensus_queue.cc:237] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.857373  4802 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:17.864092 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:17.864560 29065 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:17.872471 29070 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:07:17.865547  4764 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:17.883666 29904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:34200
I20260709 13:07:17.884588 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.870532  4910 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:17.885471 29144 raft_consensus.cc:2393] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:17.888391 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.890156 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.890834  4843 raft_consensus.cc:3060] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:17.891388 29144 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:17.892364 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:17.893774  4918 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.097s	user 0.000s	sys 0.002s
I20260709 13:07:17.893714  4565 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:07:17.885959 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.895841  4867 consensus_queue.cc:1059] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:17.903574 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:17.904652 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.904999 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:17.909181 29218 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:07:17.909380 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:17.917054 29144 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:17.912722 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:17.916285 29065 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:07:17.917130 29214 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:17.918155 29219 raft_consensus.cc:2410] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:07:17.917347 29213 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:17.919174 29215 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 27. Current term is 28. Ops: []
I20260709 13:07:17.919323 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:17.919577 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:17.920181 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:17.920560 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:17.920677 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:17.920893 29215 raft_consensus.cc:2393] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 24: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:17.921593 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:17.922482 29218 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 17.
W20260709 13:07:17.922619 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:17.923405 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:17.924582 29221 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:17.926005 29065 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:17.927148 29213 raft_consensus.cc:2410] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:07:17.927326 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.924955  4565 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:17.929984 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.930153 29064 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:07:17.930608 29140 raft_consensus.cc:2376] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22. Re-sending same reply.
I20260709 13:07:17.931092  4565 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:17.931375 29064 raft_consensus.cc:1240] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:17.931485 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.947007  4350 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:17.946225 29140 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:17.888633  4778 consensus_queue.cc:1059] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:07:17.932698 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.935081  4813 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:17.936163  4890 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:17.937414  4919 consensus_queue.cc:1059] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:17.938719  4830 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:17.939841 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.940685 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.941370 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:17.962059 29144 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:07:17.943658 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.935575 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.962234 29111 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:17.953737 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.954984 29214 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 19.
I20260709 13:07:17.949616 29068 raft_consensus.cc:2468] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:07:17.921782  4476 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:17.985287 29904 raft_consensus.cc:606] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.985853 29904 raft_consensus.cc:614] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:17.965308 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.986574 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:17.986799 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.948633  4350 raft_consensus.cc:3055] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:17.987617 29214 raft_consensus.cc:3060] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:17.987779 29144 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:17.987722 29070 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:07:17.989065 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:17.967186 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:17.988996 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:17.965615 29111 leader_election.cc:400] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 18. Current term is 19.
I20260709 13:07:17.991147 29144 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:17.966285 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:17.992121 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:17.992923 29144 raft_consensus.cc:2368] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22.
I20260709 13:07:17.993139 29111 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:17.994164 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:17.995898 29067 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:07:17.997165 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 61710.2us
I20260709 13:07:17.998193 29188 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:17.997479 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:18.000873 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:18.006114 29111 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:18.007592 29111 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.008409 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:18.009261 29066 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:07:18.020965 29188 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.988878  4350 raft_consensus.cc:740] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:18.023008  3846 raft_consensus.cc:2804] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:18.023578 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:17.988451  4844 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:17.960417  4806 consensus_queue.cc:1059] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 13:07:18.033200  3846 raft_consensus.cc:697] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:17.984944  4910 raft_consensus.cc:2764] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:07:18.032804 29147 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 28.
I20260709 13:07:18.034962 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:18.037478 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:18.016057 29140 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:18.041932 29113 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.044628 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:18.017618  4757 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:18.048408 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:18.052659 29214 raft_consensus.cc:2468] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:07:18.057392 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:18.063889 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:18.066738 29113 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.028249  4802 raft_consensus.cc:2804] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:18.028584  4657 raft_consensus.cc:2804] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:18.022101  4909 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:18.069996  3846 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.030539  4919 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:18.049429  4650 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:18.033536  4844 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.061776  4757 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.088089  4814 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.090600  4830 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:18.092391  4921 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:07:18.096144  4910 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:18.097182 29113 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.097832  4657 raft_consensus.cc:697] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:18.098510  4802 raft_consensus.cc:697] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:18.101907 29140 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:18.068837  4907 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:07:18.116818  4920 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:07:18.119397  4377 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.038546  4476 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:18.082216 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:18.176581 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 94468.1us
I20260709 13:07:18.081620 29213 raft_consensus.cc:1240] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:18.071596  4782 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:18.073549  4637 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:18.077526  4723 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.139561 29113 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:18.186866 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 47388.8us
I20260709 13:07:18.188411 29113 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:18.137138 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 102280us
I20260709 13:07:18.189814 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.132076  4844 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:18.172796 29222 raft_consensus.cc:1240] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:18.134995  4757 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:18.194301 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:18.194978 29113 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.200513 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.147022  4909 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:18.097844  4821 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.206686 29188 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.192750  4806 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:18.197151  4769 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:07:18.218374 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.204326  4830 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:18.223608 29070 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:18.221522 29188 leader_election.cc:400] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22.
I20260709 13:07:18.224370 29219 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:18.197645  4910 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:18.205888  4725 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:18.215389  4782 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:18.205932  4919 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:18.136750  4350 consensus_queue.cc:260] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.175011  4889 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.225701 29188 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:18.218628  4637 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.195031  4476 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.197075  4723 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:18.269729 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 44106.7us
I20260709 13:07:18.270870  3846 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:18.334515 29070 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:07:18.289004 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:18.307850 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:18.298524 29068 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:07:18.338192 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 49310.7us
I20260709 13:07:18.309406 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:18.330030 29067 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
I20260709 13:07:18.341215 29147 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
I20260709 13:07:18.285187 29066 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:18.342902  3846 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:18.343617  3846 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.223954  4830 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:18.282872  4903 log.cc:927] Time spent T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.897s	user 0.007s	sys 0.007s
W20260709 13:07:18.345945 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.232636  4783 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:07:18.298398  4924 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:07:18.358740  3846 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:18.274389  4782 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:18.297041  4565 raft_consensus.cc:3055] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:18.222932  4930 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:18.304529  4778 raft_consensus.cc:3055] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:18.272209  4350 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:18.298666  4761 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.284605  4821 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:18.302047  4814 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:18.256366  4802 consensus_queue.cc:237] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.351663  4910 raft_consensus.cc:2764] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:07:18.282038  4889 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:18.353122  4830 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:18.368683 29188 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.350695  4769 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:18.350387  4725 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:18.352583  4535 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:18.304193  4635 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.275933  4377 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:18.379750 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:18.396989 29147 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:18.290697  4637 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.363480  4907 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:18.366214  4565 raft_consensus.cc:740] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:18.405795  4565 consensus_queue.cc:260] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.407359  4565 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:18.408073 29147 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
I20260709 13:07:18.409083 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:18.409793 29147 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:18.410673 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:18.411202 29147 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:18.411665  4635 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:18.414054  4637 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:18.414449  4783 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:18.415144  4783 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:18.415235 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1450) took 1570 ms. Trace:
I20260709 13:07:18.415877 29148 rpcz_store.cc:276] 0709 13:07:16.844715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.853831 (+  9116us) service_pool.cc:224] Handling call
0709 13:07:16.854109 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:16.858776 (+  4667us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 13:07:16.909294 (+ 50518us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.451179 (+541885us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:17.457487 (+  6308us) log.cc:844] Serialized 26 byte log entry
0709 13:07:17.457648 (+   161us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:07:17.457684 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.457750 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:18.282249 (+824499us) spinlock_profiling.cc:243] Waited 69.3 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:18.414829 (+132580us) raft_consensus.cc:1685] finished
0709 13:07:18.414980 (+   151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.415158 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5491,"prepare.queue_time_us":47997,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":109148,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":187,"spinlock_wait_cycles":159406336,"thread_start_us":505312,"threads_started":3,"wal-append.queue_time_us":424639}
I20260709 13:07:18.419098 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:18.230383  4657 consensus_queue.cc:237] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.428231 29217 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: []
W20260709 13:07:18.431337 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.431516 29147 raft_consensus.cc:2468] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:18.432106  4723 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:18.432494 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:18.433066 29147 raft_consensus.cc:3060] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:18.433262  4723 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.433703  4723 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.353159  4646 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:07:18.437371 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.440505 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42044 (request call id 3407) took 1438 ms. Trace:
I20260709 13:07:18.441058 29146 rpcz_store.cc:276] 0709 13:07:17.002324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.167803 (+165479us) service_pool.cc:224] Handling call
0709 13:07:18.440270 (+1272467us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b5400190ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:18.440425 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384748928}
I20260709 13:07:18.367517  4778 raft_consensus.cc:740] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:18.445219  4778 consensus_queue.cc:260] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.446579  4778 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:18.448778  4867 raft_consensus.cc:3055] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:18.449223  4867 raft_consensus.cc:740] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:18.365563  4705 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:18.368448  4930 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:18.452196  4930 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:18.378856  4761 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:18.454053 29145 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:18.399165  4814 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:07:18.455237  4814 raft_consensus.cc:697] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:18.455713 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3333) took 1494 ms. Trace:
I20260709 13:07:18.456420  4814 consensus_queue.cc:237] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.452209  4705 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:18.458690  4924 log.cc:927] Time spent T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:07:18.391156 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:18.459383  4705 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:18.459630 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 68555.9us
I20260709 13:07:18.399705  4725 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.460156  4377 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:18.460592 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:18.460701  4377 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:18.404335  4796 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.706s	user 0.012s	sys 0.000s
I20260709 13:07:18.452755 29147 raft_consensus.cc:2468] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
I20260709 13:07:18.459820 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3321) took 1502 ms. Trace:
I20260709 13:07:18.462229  4725 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:18.463033  4535 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:07:18.463677  4535 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.464102  4535 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:18.402985  4650 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:18.465251  4650 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:18.466245 29216 raft_consensus.cc:2410] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:07:18.462759 29142 rpcz_store.cc:276] 0709 13:07:16.957559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.405010 (+447451us) service_pool.cc:224] Handling call
0709 13:07:17.405440 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:17.413236 (+  7796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:17.413267 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:17.413734 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:07:17.899508 (+485774us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b180000e0e0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:07:18.046145 (+146637us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:18.082276 (+ 36131us) log.cc:844] Serialized 14 byte log entry
0709 13:07:18.100253 (+ 17977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.100332 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:18.459516 (+359184us) raft_consensus.cc:1685] finished
0709 13:07:18.459639 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.459762 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28400,"prepare.queue_time_us":22091,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":38,"raft.queue_time_us":26942,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":6661,"spinlock_wait_cycles":3110144,"thread_start_us":146412,"threads_started":1,"wal-append.queue_time_us":753926}
I20260709 13:07:18.468400 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:18.468997 29219 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:18.469764 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:18.470484 29219 raft_consensus.cc:2410] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 27 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 13:07:18.404079  4476 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:18.456140 29145 rpcz_store.cc:276] 0709 13:07:16.961224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.462563 (+501339us) service_pool.cc:224] Handling call
0709 13:07:18.453864 (+991301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.454463 (+   599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.455506 (+  1043us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b5400190ad0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:18.455654 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2263546240}
I20260709 13:07:18.491909 29188 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:18.478420 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:18.492986 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.485601 29148 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:18.494451 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:18.497388 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:18.502089 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:18.504316 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:18.504578 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:18.507462 29068 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:07:18.507961 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:18.463161  4377 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:18.495324 29188 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:18.510030 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:18.510772 29145 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:18.511518  4252 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.512365 29188 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:18.513492 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:18.512127  4252 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.516449 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:18.519712 29068 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:18.435766  4723 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:18.521059 29064 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
W20260709 13:07:18.490543 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.525224 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:18.472396  4743 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:18.528002 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:18.530685 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:18.531832 29073 raft_consensus.cc:2393] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:18.493927  4637 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:18.507309  4650 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:18.541092 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:18.397529  4769 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.506819  4796 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:18.538235 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:18.404480  4907 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.536908 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:18.518096  4804 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.471993  4844 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.527633  4947 raft_consensus.cc:2804] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:18.528119  4635 raft_consensus.cc:2804] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:18.546248 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:18.563125 29145 raft_consensus.cc:2468] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
W20260709 13:07:18.565641 29139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60830 (request call id 1454) took 1719 ms (client timeout 1854 ms). Trace:
W20260709 13:07:18.566193 29139 rpcz_store.cc:269] 0709 13:07:16.845889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:16.846363 (+   474us) service_pool.cc:224] Handling call
0709 13:07:18.565441 (+1719078us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b5400191754. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:18.565574 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1621125376}
I20260709 13:07:18.450078  4867 consensus_queue.cc:260] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.577728 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1463) took 1094 ms. Trace:
I20260709 13:07:18.578559 29139 rpcz_store.cc:276] 0709 13:07:17.483459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.576552 (+1093093us) service_pool.cc:224] Handling call
0709 13:07:18.577014 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.577257 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.577300 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:18.577362 (+    62us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:07:18.577412 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.577492 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.577644 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:18.581918 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1464) took 1098 ms. Trace:
I20260709 13:07:18.583484 29139 rpcz_store.cc:276] 0709 13:07:17.483653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.580790 (+1097137us) service_pool.cc:224] Handling call
0709 13:07:18.581229 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.581465 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.581506 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:18.581569 (+    63us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:07:18.581618 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.581696 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.581844 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:18.430718  4782 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.591187 29219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:07:18.592491 29144 raft_consensus.cc:2368] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:18.552201  4796 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:18.558045  4650 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:18.604215  4252 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:18.592774  4635 raft_consensus.cc:697] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:18.559732  4804 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.592422  4846 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:18.547045  4637 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.629510  4783 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.627013  4535 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.647746 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:07:18.719293 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 71668.4us
I20260709 13:07:18.720752 29113 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:18.658752 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 224851us
I20260709 13:07:18.551844  4764 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 13:07:18.678174  4796 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:18.572443  4947 raft_consensus.cc:697] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:18.591782  4888 consensus_queue.cc:1059] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 13:07:18.674494 29219 raft_consensus.cc:606] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:18.676192 29039 leader_election.cc:400] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:07:18.732751 29039 leader_election.cc:403] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:07:18.676839 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 91599.4us
I20260709 13:07:18.666715  4867 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:18.696800  4846 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.686416 29186 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.694197 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:18.693521 29142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1461) took 1629 ms. Trace:
I20260709 13:07:18.737115 29142 rpcz_store.cc:276] 0709 13:07:17.064494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.491232 (+1426738us) service_pool.cc:224] Handling call
0709 13:07:18.513562 (+ 22330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.574322 (+ 60760us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.574391 (+    69us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:18.574456 (+    65us) raft_consensus.cc:1645] Marking committed up to 21
0709 13:07:18.574508 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.693177 (+118669us) spinlock_profiling.cc:243] Waited 60.4 ms on lock 0x7b5400041354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:18.693247 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.693440 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139021568}
I20260709 13:07:18.702111 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42054 (request call id 3346) took 1026 ms. Trace:
I20260709 13:07:18.703470 29188 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:18.722380 29113 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:18.738970 29144 rpcz_store.cc:276] 0709 13:07:17.675893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.997262 (+321369us) service_pool.cc:224] Handling call
0709 13:07:18.701876 (+704614us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b540029f254. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:18.702033 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1361359616}
I20260709 13:07:18.723721 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:18.677937  4377 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.729496 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:18.740124 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:18.740481 29141 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:18.716473  4743 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:18.716696  4802 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.014s
I20260709 13:07:18.734299 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:18.735419 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 59474.9us
I20260709 13:07:18.718508  4658 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.718629  4806 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:18.669907  4761 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.748893  4723 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:18.742740  4846 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:18.698509 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1462) took 1215 ms. Trace:
I20260709 13:07:18.746534  4657 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:18.757225  4657 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:18.756371 29145 rpcz_store.cc:276] 0709 13:07:17.483134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.571734 (+1088600us) service_pool.cc:224] Handling call
0709 13:07:18.572121 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.590252 (+ 18131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.590301 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:18.590359 (+    58us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:18.590401 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.698196 (+107795us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b54001bc6d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:18.698262 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.698435 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41040256}
I20260709 13:07:18.757858  4657 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:18.746675  4377 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.760478  4844 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:18.761471  4377 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:18.746646 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:18.728417  4705 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:18.745611 29188 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:18.769979 29188 leader_election.cc:400] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:18.749771  4919 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:18.749866  4806 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.751506  4843 raft_consensus.cc:3055] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:07:18.728129  4476 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:18.744633  4661 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 18. Current term is 19.
I20260709 13:07:18.753984  4947 consensus_queue.cc:237] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.754761  4888 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:18.755347 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:18.796418 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 110162us
I20260709 13:07:18.796959 29066 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 26.
I20260709 13:07:18.692320  4889 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:07:18.796468 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:18.763948 29148 raft_consensus.cc:1240] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 26. Current term is 27. Ops: []
I20260709 13:07:18.765043 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:18.765791 29141 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:18.765991 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:18.798678 29065 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:18.772202 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1465) took 1234 ms. Trace:
I20260709 13:07:18.777377 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:18.800504 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1459) took 1835 ms. Trace:
I20260709 13:07:18.778712  3846 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:07:18.778437 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 54804.4us
I20260709 13:07:18.741757  4263 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.802119  3846 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:18.802556  3846 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:07:18.803856 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 54448.4us
I20260709 13:07:18.800937 29146 rpcz_store.cc:276] 0709 13:07:16.964666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.442345 (+1477679us) service_pool.cc:224] Handling call
0709 13:07:18.442783 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.443259 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.443301 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.471066 (+ 27765us) log.cc:844] Serialized 26 byte log entry
0709 13:07:18.671164 (+200098us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:18.795558 (+124394us) log.cc:844] Serialized 14 byte log entry
0709 13:07:18.800176 (+  4618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.800265 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:18.800304 (+    39us) raft_consensus.cc:1685] finished
0709 13:07:18.800319 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.800443 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20358,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":92,"thread_start_us":231340,"threads_started":2,"wal-append.queue_time_us":235164}
I20260709 13:07:18.806704 29144 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:07:18.807513 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3342) took 1132 ms. Trace:
I20260709 13:07:18.795850  4377 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:18.807986 29144 rpcz_store.cc:276] 0709 13:07:17.674813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.806090 (+1131277us) service_pool.cc:224] Handling call
0709 13:07:18.806464 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.807192 (+   728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.807449 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:18.809587  4535 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:18.794315  4843 raft_consensus.cc:740] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:18.811422 29144 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
W20260709 13:07:18.794866 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 77101.2us
I20260709 13:07:18.795951 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:18.763562 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:18.814687 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3424) took 1129 ms. Trace:
I20260709 13:07:18.815136 29144 rpcz_store.cc:276] 0709 13:07:17.685125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.809006 (+1123881us) service_pool.cc:224] Handling call
0709 13:07:18.809338 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.811890 (+  2552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.814626 (+  2736us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:18.800750 29139 rpcz_store.cc:276] 0709 13:07:17.537410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.673143 (+1135733us) service_pool.cc:224] Handling call
0709 13:07:18.673611 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.733792 (+ 60181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.733826 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:18.733866 (+    40us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:18.733900 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.771828 (+ 37928us) spinlock_profiling.cc:243] Waited 60 ms on lock 0x7b54001bd5d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:18.771934 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.772122 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137934080}
I20260709 13:07:18.782807 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:07:18.817641 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:18.805363  4778 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:18.778143  4919 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:18.822044  4919 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:18.777918  4705 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:18.822593  4705 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:18.823840 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:18.824759 29144 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:07:18.828229 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 45539.8us
I20260709 13:07:18.798777  4948 raft_consensus.cc:2804] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:18.832988 29065 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:07:18.832170  3846 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.795749  4909 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:18.812877  4932 log.cc:927] Time spent T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.420s	user 0.001s	sys 0.003s
I20260709 13:07:18.833806 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:18.840375 29218 raft_consensus.cc:3060] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:18.822304  4843 consensus_queue.cc:260] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.762126  4658 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.778132  4844 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.855137 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:18.841253 29904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8"
dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
mode: GRACEFUL
new_leader_uuid: "a936878f59c840a6bd176a1d2d7347b1"
 from {username='slave'} at 127.0.0.1:34200
I20260709 13:07:18.867228  3846 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:18.823186  4377 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:18.837177  4661 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:18.748333  4743 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.819008  4888 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:18.841832  4890 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:18.892221  4890 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.894735  4890 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:18.895388  4890 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:18.756071  4650 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.897632  4658 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:18.818692  4263 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:18.869365  4945 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 13:07:18.833213  4769 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:18.806063  4635 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.888626 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:18.930141 29145 raft_consensus.cc:2368] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
W20260709 13:07:18.901275  4757 consensus_meta.cc:287] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.695s	user 0.008s	sys 0.003s
W20260709 13:07:18.904604 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 65687us
I20260709 13:07:18.905524 29144 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:18.934917 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:18.935560 29144 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:18.936689 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:18.841373  4821 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:07:18.853436  4778 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:07:18.904031  4661 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:18.904209  4843 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:07:18.933562  4915 log.cc:927] Time spent T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 1.158s	user 0.000s	sys 0.004s
I20260709 13:07:18.878918  4962 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:07:18.959232  4944 log.cc:927] Time spent T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.441s	user 0.003s	sys 0.000s
I20260709 13:07:18.960204  4806 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:18.961107  4263 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:18.983491 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet 2e40d8d1f334428db02ee54c8467d058: Illegal state: Replica a936878f59c840a6bd176a1d2d7347b1 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:07:18.961319  4830 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.000s
I20260709 13:07:18.968163  4804 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:18.968935  4951 log.cc:927] Time spent T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:07:18.970952 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60830 (request call id 1460) took 1998 ms. Trace:
I20260709 13:07:18.990152 29147 rpcz_store.cc:276] 0709 13:07:16.972639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.463830 (+1491191us) service_pool.cc:224] Handling call
0709 13:07:18.464214 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.464800 (+   586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.464828 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.465401 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 13:07:18.778227 (+312826us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:07:18.778266 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.778328 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:18.970744 (+192416us) raft_consensus.cc:1685] finished
0709 13:07:18.970768 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:18.970896 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":68,"thread_start_us":312319,"threads_started":1,"wal-append.queue_time_us":397008}
I20260709 13:07:18.858345  4909 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.972007  4783 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:18.972218  4377 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:18.876986  4844 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:18.975618  4930 consensus_meta.cc:287] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.000s
I20260709 13:07:18.979854 29144 raft_consensus.cc:1240] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 27. Current term is 28. Ops: []
I20260709 13:07:18.980726 29188 leader_election.cc:400] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:19.019143 29188 leader_election.cc:403] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:07:19.019405 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3425) took 1334 ms. Trace:
I20260709 13:07:19.019838 29144 rpcz_store.cc:276] 0709 13:07:17.685395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.955712 (+1270317us) service_pool.cc:224] Handling call
0709 13:07:18.979135 (+ 23423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.019141 (+ 40006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.019341 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:19.021744 29144 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:07:19.022449 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3428) took 1302 ms. Trace:
I20260709 13:07:19.022831 29144 rpcz_store.cc:276] 0709 13:07:17.720249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.020905 (+1300656us) service_pool.cc:224] Handling call
0709 13:07:19.021294 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.022168 (+   874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.022386 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:07:18.968230  4954 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:07:18.953874  4889 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:07:18.926190 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 194305us
I20260709 13:07:19.031091 29875 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:18.954885  4813 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:18.895766  4657 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:18.989215  4948 raft_consensus.cc:697] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:18.940666 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:19.036531 29145 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
W20260709 13:07:18.981987  4350 consensus_meta.cc:287] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.000s
I20260709 13:07:19.043213 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:19.027181 29144 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: [22.11-24.12]
W20260709 13:07:19.032195 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 51828.4us
I20260709 13:07:19.034554 29142 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:19.012290  4962 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:19.034240 29072 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40862 (request call id 1281) took 1077 ms. Trace:
I20260709 13:07:18.942013  4757 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.058548 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3319) took 2101 ms. Trace:
I20260709 13:07:18.971860  4796 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.051734 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:19.061246 29902 raft_consensus.cc:3060] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:07:19.040107 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:19.040305 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:18.982448  4888 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.046389 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:19.049268 29145 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:19.063405 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3430) took 1342 ms. Trace:
I20260709 13:07:19.063897 29145 rpcz_store.cc:276] 0709 13:07:17.720781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.042811 (+1322030us) service_pool.cc:224] Handling call
0709 13:07:19.049101 (+  6290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.063070 (+ 13969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.063324 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:19.050998 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3429) took 1330 ms. Trace:
I20260709 13:07:19.051735  4661 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.048902 29072 rpcz_store.cc:276] 0709 13:07:17.957306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.249328 (+292022us) service_pool.cc:224] Handling call
0709 13:07:18.253839 (+  4511us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:18.265865 (+ 12026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.265908 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:18.280899 (+ 14991us) log.cc:844] Serialized 52 byte log entry
0709 13:07:18.388382 (+107483us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:18.388432 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:18.388502 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.033963 (+645461us) raft_consensus.cc:1685] finished
0709 13:07:19.033989 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.034165 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":214,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":8590,"thread_start_us":107071,"threads_started":1,"wal-append.queue_time_us":254831}
I20260709 13:07:19.053643  4948 consensus_queue.cc:237] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.030308  4743 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.057631  4657 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:19.047511 29188 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:19.030617  4889 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.066172 29144 rpcz_store.cc:276] 0709 13:07:17.720535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.023753 (+1303218us) service_pool.cc:224] Handling call
0709 13:07:19.026821 (+  3068us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:19.050645 (+ 23824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.050938 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:19.058935 29143 rpcz_store.cc:276] 0709 13:07:16.956940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:17.401047 (+444107us) service_pool.cc:224] Handling call
0709 13:07:17.423942 (+ 22895us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:17.450122 (+ 26180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:17.450153 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:17.468809 (+ 18656us) log.cc:844] Serialized 52 byte log entry
0709 13:07:17.648576 (+179767us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:17.700618 (+ 52042us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.850282 (+149664us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b1800101e60. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f783399ac0a 00007f783d0da219 00007f7833a22a40 00007f7833a1d4e4 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287 00007f7836b28208
0709 13:07:17.850712 (+   430us) log.cc:844] Serialized 14 byte log entry
0709 13:07:17.861880 (+ 11168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:17.861984 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:18.777471 (+915487us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:19.058328 (+280857us) raft_consensus.cc:1685] finished
0709 13:07:19.058351 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.058493 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16133,"prepare.queue_time_us":53205,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":186881,"raft.run_cpu_time_us":400,"raft.run_wall_time_us":12837,"spinlock_wait_cycles":33640576,"thread_start_us":158438,"threads_started":1,"wal-append.queue_time_us":255027}
I20260709 13:07:19.000061  4830 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.080345 29144 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: [17.11-21.13]
I20260709 13:07:19.081351 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3432) took 1360 ms. Trace:
I20260709 13:07:19.081813 29144 rpcz_store.cc:276] 0709 13:07:17.721270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.074364 (+1353094us) service_pool.cc:224] Handling call
0709 13:07:19.080054 (+  5690us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:07:19.080997 (+   943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.081269 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:19.041767  4804 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:19.082921 29908 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:07:19.090235 29142 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 22.
I20260709 13:07:19.091611 29910 raft_consensus.cc:1275] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:07:19.091696 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:19.093613 29908 raft_consensus.cc:2410] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 27 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 13:07:19.059219  4890 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:19.096921 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3426) took 1398 ms. Trace:
I20260709 13:07:19.031323  4930 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.082029  4889 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:19.042189  4813 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.066937  4967 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.070s
W20260709 13:07:19.098189 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 293371us
I20260709 13:07:19.130045 29146 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 13:07:19.119405 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:19.120632 29220 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:19.155485 29146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3351) took 1299 ms. Trace:
I20260709 13:07:19.156605 29146 rpcz_store.cc:276] 0709 13:07:17.855674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.129372 (+1273698us) service_pool.cc:224] Handling call
0709 13:07:19.129756 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.155197 (+ 25441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.155417 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:07:19.099239 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:19.127049 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:19.097339 29139 rpcz_store.cc:276] 0709 13:07:17.698780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.963267 (+1264487us) service_pool.cc:224] Handling call
0709 13:07:18.963807 (+   540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.977159 (+ 13352us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:07:18.983518 (+  6359us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.096586 (+113068us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:19.096641 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:19.096682 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.096751 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.096865 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12120,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":1730,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":16823,"thread_start_us":112191,"threads_started":1}
I20260709 13:07:19.079236  4538 raft_consensus.cc:2804] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:07:19.094404  4952 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:07:19.168202 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 47282us
I20260709 13:07:19.180559 29218 raft_consensus.cc:2468] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:07:19.079592  4846 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.189790 29217 raft_consensus.cc:1240] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:19.191377 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:19.097816  4890 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.118769  4650 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:19.207666 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.092319  4804 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:07:19.185803  4964 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:07:19.197363  4965 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:19.226164 29904 raft_consensus.cc:606] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Received request to transfer leadership to TS a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:19.183298  4783 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:19.237102  4909 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.177132  4967 consensus_queue.cc:1243] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 730186a8c70f43f2a57d80163bbc37d2 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:07:19.238691  4813 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.197984  4890 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:19.190968  4538 raft_consensus.cc:697] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:19.241146  4538 consensus_queue.cc:237] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.246361  4846 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.251443  4796 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:19.235592  4963 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:07:19.236653 29145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3431) took 1515 ms. Trace:
I20260709 13:07:19.239615  4967 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:19.232468 29904 raft_consensus.cc:614] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:19.255031 29902 raft_consensus.cc:2468] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:07:19.254673 29145 rpcz_store.cc:276] 0709 13:07:17.721030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.067048 (+1346018us) service_pool.cc:224] Handling call
0709 13:07:19.070746 (+  3698us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.071190 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.071232 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.080160 (+  8928us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.185761 (+105601us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:19.185845 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.185979 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.236406 (+ 50427us) raft_consensus.cc:1685] finished
0709 13:07:19.236429 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.236584 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1035,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":3515,"thread_start_us":57556,"threads_started":1,"wal-append.queue_time_us":136643}
I20260709 13:07:19.258209 29145 raft_consensus.cc:1240] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
W20260709 13:07:19.258684 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 102758us
I20260709 13:07:19.274879 29145 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:07:19.275754 29185 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:19.281469 29145 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:07:19.282480  4844 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.282662  4965 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:19.283551  4965 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:19.280489  4947 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:07:19.287323  4705 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.289057  4757 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:19.226235  4804 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:19.289757 29147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3427) took 1569 ms. Trace:
I20260709 13:07:19.290207 29147 rpcz_store.cc:276] 0709 13:07:17.719891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.992113 (+1272222us) service_pool.cc:224] Handling call
0709 13:07:18.992579 (+   466us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:18.993036 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.993075 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:07:19.012698 (+ 19623us) log.cc:844] Serialized 52 byte log entry
0709 13:07:19.194663 (+181965us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:07:19.247292 (+ 52629us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.248599 (+  1307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.248707 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.289519 (+ 40812us) raft_consensus.cc:1685] finished
0709 13:07:19.289534 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.289682 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49386,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":406,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":17099,"thread_start_us":138310,"threads_started":1,"wal-append.queue_time_us":242468}
I20260709 13:07:19.292156 29147 raft_consensus.cc:1275] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:07:19.293118 29147 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:07:19.293778 29145 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:07:19.228190  4650 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.272100  4846 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:07:19.289567  4965 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:19.306664 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:19.308126 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:19.308637 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:19.308724 29065 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 26.
I20260709 13:07:19.309051 29064 raft_consensus.cc:2368] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24.
I20260709 13:07:19.309330 29071 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:19.275779  4377 consensus_queue.cc:1059] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.028s
I20260709 13:07:19.310509 29064 raft_consensus.cc:1275] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:07:19.310823 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:19.311388 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:19.312750 29065 raft_consensus.cc:3060] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:19.311564 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:19.313452 29067 raft_consensus.cc:2368] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:19.312666 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:19.314049 29068 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:07:19.317731 29066 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:07:19.317528  4965 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.227265  4962 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.325878 29065 raft_consensus.cc:2468] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:19.329344  4930 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:19.331164  4890 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.310824  4705 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.334022  4637 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:19.338477 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:19.339315  4919 consensus_meta.cc:287] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.000s
I20260709 13:07:19.339771  4919 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.342208 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.342746 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.343183 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:19.343943 29215 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:07:19.344864 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:19.345084 29216 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:19.348755 29214 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:07:19.350636 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.351121 29214 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:07:19.351619  4635 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.352348 29213 raft_consensus.cc:1240] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:19.352157  4635 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.353313 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.356164 29070 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 25. Current term is 26. Ops: [23.13-25.14]
I20260709 13:07:19.364482  4907 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:19.364933  4890 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.366618  4830 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:19.367697 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:19.368309 29219 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:19.368335  4846 raft_consensus.cc:3055] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:19.368924  4919 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:19.371505  4650 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:19.374208 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.374701 29216 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:07:19.374904 29213 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 26.
W20260709 13:07:19.376139 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet e9334a7c40c744cbbff4dec0358f721b: Illegal state: Replica 730186a8c70f43f2a57d80163bbc37d2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 24 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:19.378666 29141 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:07:19.381199 29141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3340) took 1707 ms. Trace:
I20260709 13:07:19.381523 29141 rpcz_store.cc:276] 0709 13:07:17.674265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.798888 (+1124623us) service_pool.cc:224] Handling call
0709 13:07:19.378429 (+579541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.379111 (+   682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.381010 (+  1899us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b540029ed50. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0c8ad 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:19.381139 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1309309824}
I20260709 13:07:19.383993  4889 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.385102 29071 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:19.385807 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.386937 29144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3350) took 1531 ms. Trace:
I20260709 13:07:19.389159 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:19.389384 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:19.389745 29142 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:19.389971 29145 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:07:19.390115 29147 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:19.391388 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:19.392269 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.389467 29144 rpcz_store.cc:276] 0709 13:07:17.855522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.083272 (+1227750us) service_pool.cc:224] Handling call
0709 13:07:19.083595 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:19.094533 (+ 10938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:19.094581 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:19.095145 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.271285 (+176140us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:07:19.291972 (+ 20687us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.296913 (+  4941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.296964 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.386774 (+ 89810us) raft_consensus.cc:1685] finished
0709 13:07:19.386795 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.386892 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10888,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":94,"raft.queue_time_us":9917,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":7385,"thread_start_us":115880,"threads_started":1,"wal-append.queue_time_us":223778}
I20260709 13:07:19.395584 29219 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:19.396659 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:19.400871 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:19.404806 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:19.407051 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:19.408012 29145 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:19.410492 29064 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:07:19.412719 29147 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 26.
I20260709 13:07:19.412955 29142 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:19.420796 29141 raft_consensus.cc:1217] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Deduplicated request from leader. Original: 17.11->[19.12-24.15]   Dedup: 19.12->[20.13-24.15]
I20260709 13:07:19.415480 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:19.367002  4635 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:19.425074 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:19.425823 29145 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:19.384984 29066 consensus_queue.cc:237] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:07:19.421936 29141 pending_rounds.cc:85] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 12
I20260709 13:07:19.427815 29141 pending_rounds.cc:107] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:07:19.428136 29141 raft_consensus.cc:2889] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:19.416839 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:19.406535  4947 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:07:19.430706 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:19.424248  4782 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:07:19.406057  4930 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:07:19.368918  4846 raft_consensus.cc:740] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:19.405856  4890 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:07:19.437364  4846 consensus_queue.cc:260] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.438782  4846 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:19.404678  4637 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:07:19.437389  4966 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:19.440485  4966 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.440855  4966 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:19.385039  4804 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.401415 29144 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 19.
I20260709 13:07:19.451516 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:19.453857 29144 raft_consensus.cc:2368] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24.
I20260709 13:07:19.455639 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:19.458847 29144 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:19.459971 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:19.470505 29145 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:19.483379  4984 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:07:19.485504 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 70542.5us
I20260709 13:07:19.486418 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:07:19.490361 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 64330.6us
I20260709 13:07:19.491101 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:19.489046  4947 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:19.496455  4947 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.498023  4947 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:19.498862 29871 leader_election.cc:400] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24.
I20260709 13:07:19.499228 29871 leader_election.cc:403] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:19.499969 29871 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:19.500872  4959 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:19.405215  4830 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:07:19.501983 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:19.503005  4813 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:19.503536  4813 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.509248 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3337) took 1835 ms. Trace:
I20260709 13:07:19.509794 29148 rpcz_store.cc:276] 0709 13:07:17.673412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.713436 (+1040024us) service_pool.cc:224] Handling call
0709 13:07:18.713785 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:18.798264 (+ 84479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.508992 (+710728us) spinlock_profiling.cc:243] Waited 50 ms on lock 0x7b540027ced4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:19.509181 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114899328}
W20260709 13:07:19.512128  4796 consensus_queue.cc:1175] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "730186a8c70f43f2a57d80163bbc37d2" responder_term: 22 status { last_received { term: 19 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: [21.12-21.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:07:19.501425  4974 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24.
I20260709 13:07:19.513340  4764 consensus_queue.cc:1059] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:19.415445  4377 consensus_queue.cc:1059] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:07:19.485028  4730 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:07:19.515794  4806 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.515931 29144 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:19.516398  4806 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.485786  4637 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:19.517802  4637 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.518553  4806 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.518664 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:19.519695  4844 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.501368  4959 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.520354  4959 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:19.520157  4844 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.520668  4907 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:19.521320  4764 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:19.522290  4844 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:19.523131  4764 raft_consensus.cc:3055] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:19.523077 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.523600  4764 raft_consensus.cc:740] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:19.523840 29215 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:19.524606 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.527233 29113 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:19.532876  4778 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:19.515817  4965 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:19.517271 29144 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 25. Current term is 26. Ops: [23.13-25.14]
I20260709 13:07:19.541453 29144 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:07:19.540866 29112 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:19.539376 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:19.543620 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:19.580996 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:19.582747 29112 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:19.544204 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:19.517321  4813 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:19.589620  4989 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:19.590135  4989 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:19.591101  4989 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.592950  4989 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:19.596225  4802 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:19.601495 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:19.603379 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:19.610644 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:19.615280 29219 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:19.617890 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:19.618867 29144 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:19.621059  4802 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.621762  4802 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.629603 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.630455 29213 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:19.632122 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:19.633394  4723 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:19.633872  4723 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.634294  4723 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:19.640552  4948 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:19.641350  4948 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.644737  4802 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:19.646570  4948 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:19.654731 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:19.660784 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:19.662086 29906 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:07:19.662495 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:19.663882 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:19.664755 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:19.665139 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:19.538187  4806 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:19.535426  4637 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:19.671736  4769 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:19.686527  4723 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.687991 29144 raft_consensus.cc:2368] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 25. Current term is 26.
I20260709 13:07:19.539896  4778 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:07:19.676925 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 238665us
I20260709 13:07:19.673758 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:19.715445 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 228620us
I20260709 13:07:19.545473  4907 raft_consensus.cc:2804] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:07:19.735637 29144 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:19.587512  4909 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:19.745385 29875 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:19.748674  4723 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:19.749485  4986 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:07:19.750749  4974 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:19.751293  4974 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:19.751694  4974 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:19.541813  4705 consensus_queue.cc:1048] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.161s
I20260709 13:07:19.541728  4844 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.192s
I20260709 13:07:19.736410  4830 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.772925  4830 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.772109  4974 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.774487  4959 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.742004 29039 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:07:19.778647 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.764356 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:19.780357 29902 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:19.772042 29148 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 26.
I20260709 13:07:19.783274 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:19.784142 29213 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:19.789896 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:19.790721 29144 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:19.801916 29140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3336) took 2128 ms. Trace:
I20260709 13:07:19.802045 29875 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:19.802659 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:19.802572 29140 rpcz_store.cc:276] 0709 13:07:17.673104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.714113 (+1041009us) service_pool.cc:224] Handling call
0709 13:07:18.714421 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:18.715623 (+  1202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:18.715663 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:18.716136 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:07:19.355276 (+639140us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b18000f21a0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a02e07 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b 00007f78366acd23 00007f78366ad0d1 00007f7836ac8a0f
0709 13:07:19.538071 (+182795us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:07:19.604092 (+ 66021us) log.cc:844] Serialized 14 byte log entry
0709 13:07:19.604496 (+   404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:19.604569 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:19.792709 (+188140us) raft_consensus.cc:1685] finished
0709 13:07:19.801666 (+  8957us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:19.801852 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5504,"prepare.queue_time_us":62672,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":1005,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":6652,"spinlock_wait_cycles":6529920,"thread_start_us":205738,"threads_started":2,"wal-append.queue_time_us":1035813}
I20260709 13:07:19.808176 29140 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: [22.14-24.15]
I20260709 13:07:19.811444 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:19.674842  4743 consensus_queue.cc:1048] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.117s
I20260709 13:07:19.815400 29875 leader_election.cc:400] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 25. Current term is 26.
I20260709 13:07:19.775723  4830 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:19.744410  4992 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.792809  4806 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:19.769299  4804 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.771281  4889 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.818735 29875 leader_election.cc:403] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:07:19.799015  4864 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:19.805150  4957 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.836375 29213 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 26.
I20260709 13:07:19.814565  4263 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:19.840256 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:19.815343  4907 raft_consensus.cc:697] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:19.818889  4989 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.768774  4778 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.733402  4764 consensus_queue.cc:260] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:19.849959 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 310651us
I20260709 13:07:19.837911  4974 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:19.857194 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 138449us
I20260709 13:07:19.861711 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:19.862578 29215 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:19.837333  4637 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:07:19.849764  4864 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:19.862013 29144 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:19.864049 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:19.867209 29213 raft_consensus.cc:2368] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 25. Current term is 26.
I20260709 13:07:19.850620  4806 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:19.832331  4992 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.842332  4957 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.856926  4705 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:19.857851  4723 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:19.860370  4959 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:19.876010  4889 consensus_queue.cc:1059] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:07:19.877264  4661 consensus_queue.cc:1059] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:19.878283  5001 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.898217 29902 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:19.879220  4975 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:07:19.879870 29874 leader_election.cc:400] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 25. Current term is 26.
I20260709 13:07:19.899381 29142 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:07:19.887780 29217 raft_consensus.cc:2410] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:07:19.889017 29215 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:07:19.863801  4764 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:19.894965 29144 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:07:19.850937  4989 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.902169 29874 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:19.886888  4864 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:19.905280  5001 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:19.937960  4263 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:19.909861  5004 raft_consensus.cc:3055] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:19.953580 29145 raft_consensus.cc:1275] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:07:19.925912 29904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34200 (request call id 1) took 1103 ms. Trace:
I20260709 13:07:19.955711 29904 rpcz_store.cc:276] 0709 13:07:18.822213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.823536 (+  1323us) service_pool.cc:224] Handling call
0709 13:07:19.925675 (+1102139us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54005e92d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b07853 00007f783bad560a 00007f783505b7d4 00007f783505b762 00007f783505b691 00007f783505b60c 00007f783505a8c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:19.925808 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831087360}
I20260709 13:07:19.903285  4975 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:19.876073  4967 raft_consensus.cc:3055] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:19.896114  4974 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:19.948329  4966 consensus_meta.cc:287] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.003s
I20260709 13:07:19.958752 29139 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:19.960040 29145 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:07:19.978943 29139 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 13:07:19.979509 29145 raft_consensus.cc:1240] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: [22.11-24.12]
I20260709 13:07:19.907148  4992 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:19.924453  4997 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:07:19.924253  4960 log.cc:927] Time spent T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.940s	user 0.006s	sys 0.006s
W20260709 13:07:19.943166 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:19.985286 29185 leader_election.cc:400] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 18 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 18. Current term is 19.
I20260709 13:07:19.985394  5001 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.985641  4957 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.986537 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:19.964572 29144 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 13:07:19.964658 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 148608us
I20260709 13:07:19.967113  4890 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.988365 29112 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:19.989995 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.990371 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:19.991269 29214 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:19.967177  4761 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:19.992832 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:19.968977 29142 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:19.968922 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:19.970417 29140 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:07:19.931273  4959 raft_consensus.cc:2804] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:19.934526  4988 raft_consensus.cc:3055] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:19.923952  4989 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:19.976562 29113 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:20.000913 29903 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 13:07:19.952054  5004 raft_consensus.cc:740] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:19.971487  4967 raft_consensus.cc:740] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:19.997252  4930 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:19.997524  4830 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:20.003623 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:20.013762  4538 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:07:20.014971  4650 consensus_queue.cc:1048] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:20.020207  4975 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.023600 29144 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:07:20.024309  5010 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:20.025266 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:07:20.044860 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 180760us
I20260709 13:07:19.904567  4907 consensus_queue.cc:237] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.004417  4988 raft_consensus.cc:740] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:20.009577  4948 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:07:19.962144  4263 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.009682  4743 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 13:07:20.005102  4957 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:07:20.027432  4947 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.529s	user 0.004s	sys 0.009s
I20260709 13:07:20.027621 29141 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:07:20.051117 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:20.062902 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:20.063704 29065 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:20.065124 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:20.065670 29068 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:20.039971  5012 raft_consensus.cc:2804] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:20.067844 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:20.069818 29213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f"
dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
mode: GRACEFUL
new_leader_uuid: "730186a8c70f43f2a57d80163bbc37d2"
 from {username='slave'} at 127.0.0.1:44946
I20260709 13:07:20.070000 29065 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:20.070536 29213 raft_consensus.cc:606] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Received request to transfer leadership to TS 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:20.071022 29213 raft_consensus.cc:614] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:07:19.987506  4974 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.068925 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:19.992513  4966 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.074191 29215 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:07:20.075690 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:20.068301 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:20.087324 29067 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:07:20.004653  4890 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.081568 29072 raft_consensus.cc:3060] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:20.089370 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:20.093611 29215 raft_consensus.cc:1240] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:07:20.095582 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:20.129923 29221 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:20.136363 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:20.059988  4957 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.062469  4984 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:07:20.162750 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 160186us
I20260709 13:07:20.167814 29068 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 27.
I20260709 13:07:20.045755  4830 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:20.139534  4954 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:07:20.056493  4970 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
W20260709 13:07:20.158715  4991 log.cc:927] Time spent T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:07:20.043937  4930 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.163899 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:07:20.173100 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 101511us
I20260709 13:07:20.133808  4890 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.160830  4957 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:20.171907 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:20.137737  4263 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:20.066617  4947 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.102201  5012 raft_consensus.cc:697] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:20.104404  4975 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:20.100361  4959 raft_consensus.cc:697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:20.193017 29216 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 19.
I20260709 13:07:20.103454  4757 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.077s
I20260709 13:07:20.131870  4538 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.086s
I20260709 13:07:20.172523  4984 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:20.207367 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.208179 29111 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:20.213380 29068 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 13:07:20.215802 29068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:20.173735  4761 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.030s
I20260709 13:07:20.218590 29068 raft_consensus.cc:1275] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:07:20.220077 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:20.191126  4813 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.223407 29064 raft_consensus.cc:1275] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:07:20.224032  4947 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:20.224377  4813 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:20.218858  5009 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.225538  5009 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.227609  5009 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:20.216902  4764 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:20.209980  4757 consensus_queue.cc:1243] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:07:20.229372  4764 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.230201 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:20.230800 29146 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:20.231073 29141 raft_consensus.cc:1240] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:20.231448  4764 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:20.225214  5012 consensus_queue.cc:237] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.232466  4959 consensus_queue.cc:237] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.221973 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:20.236778 29065 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:20.237433 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:20.239848 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:20.240487 29064 raft_consensus.cc:3060] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:20.079512  4967 consensus_queue.cc:260] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.240258 29072 raft_consensus.cc:2468] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:07:20.167934  4988 consensus_queue.cc:260] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.244930  4988 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:20.241005  5015 consensus_queue.cc:1059] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 13:07:20.249012  4761 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:20.249497  4761 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:20.249898  4761 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:20.103224  5004 consensus_queue.cc:260] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.253041  5013 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:07:20.253460  5013 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:20.256093 29146 raft_consensus.cc:2468] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:07:20.257509 29875 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:20.262188 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:20.265038  4816 raft_consensus.cc:2804] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:20.265631  4816 raft_consensus.cc:697] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:20.267575 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:20.268175 29144 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:20.270536 29141 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 22: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:20.273610  4816 consensus_queue.cc:237] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.276664 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:20.277637 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:20.266656  4967 raft_consensus.cc:3060] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:20.283504 29909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:20.266760  4806 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 25. Current term is 26.
I20260709 13:07:20.288602 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:20.289247 29911 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:20.300685  4830 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.254251  5013 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.279172  5000 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:20.303508  5000 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.284014  4984 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.307560  4966 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:20.308897 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.312861 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:20.313297 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:20.313968 29911 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:20.314327  5027 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.314891  5027 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.316308  4830 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.313483 29221 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:20.316773  4830 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.318845  4830 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:20.319984 29144 raft_consensus.cc:2468] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
W20260709 13:07:20.452140 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.457758 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.465579 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.473821 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.482385 29114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:20.485867  4705 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.005s	sys 0.014s
I20260709 13:07:20.339231 29220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3404) took 1855 ms. Trace:
I20260709 13:07:20.384084 29221 raft_consensus.cc:2468] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 27.
I20260709 13:07:20.497503  5010 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:20.364913 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:20.347997  4796 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:07:20.458652 29037 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b55b583a63b64ca6832b73f813985e49. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:20.498776  5010 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.342875  4778 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:20.345407  4804 raft_consensus.cc:3055] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:20.501155  4804 raft_consensus.cc:740] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:07:20.515638 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.351967 29222 raft_consensus.cc:1240] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:20.374959 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:20.499718 29902 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 18.
W20260709 13:07:20.485242 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.546759  4843 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.547847  4843 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:20.547958 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 280206us
W20260709 13:07:20.495582  4705 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.006s	sys 0.014s
W20260709 13:07:20.500150 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44881.5us
I20260709 13:07:20.486349 29217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42970 (request call id 3521) took 1144 ms. Trace:
W20260709 13:07:20.558033  5039 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.136s	user 0.003s	sys 0.000s
I20260709 13:07:20.322611  5027 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.321880  4930 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.567847 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:20.571115  5010 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.492501  4730 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:20.573196  4730 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.320519  4761 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.577569 29037 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:20.578846  4705 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:20.581239  4705 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.346341  4948 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:20.584671  4948 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:20.585017 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:20.585912 29903 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 18.
W20260709 13:07:20.588017 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.332454  4957 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.602620  4796 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.496953 29220 rpcz_store.cc:276] 0709 13:07:18.484184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:18.517678 (+ 33494us) service_pool.cc:224] Handling call
0709 13:07:18.521538 (+  3860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.179435 (+657897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.338916 (+1159481us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540023c154. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:20.339152 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1377318272}
I20260709 13:07:20.613838 29070 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3375) took 1301 ms. Trace:
I20260709 13:07:20.614313 29070 rpcz_store.cc:276] 0709 13:07:19.312147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.312266 (+   119us) service_pool.cc:224] Handling call
0709 13:07:19.312621 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:07:19.356592 (+ 43971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.613064 (+1256472us) spinlock_profiling.cc:243] Waited 43.3 ms on lock 0x7b5400321ed4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:20.613569 (+   505us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99563392}
I20260709 13:07:20.554884 29217 rpcz_store.cc:276] 0709 13:07:19.341534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.342514 (+   980us) service_pool.cc:224] Handling call
0709 13:07:20.485600 (+1143086us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b5400283cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:07:20.486061 (+   461us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1251585920}
I20260709 13:07:20.593214  4804 consensus_queue.cc:260] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:20.634394 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.631301  5010 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:20.586642  4948 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.637653 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:20.641515 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 64318.5us
I20260709 13:07:20.632566  4970 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.646318  5025 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:20.646984  5025 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.647058  4888 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:20.637573  4855 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.647183  4804 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:20.643586  4637 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.649464  4637 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.647629  4888 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.637643  5010 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.652444  4888 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:20.653502  4957 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:20.647759  5004 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:20.654718  5010 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.656844  4637 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:20.655488 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 249291us
I20260709 13:07:20.647529  5025 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:20.648243  4855 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.333953  4844 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.325s
I20260709 13:07:20.657997  4730 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:20.671018 29874 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:20.679530  5035 log.cc:927] Time spent T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.258s	user 0.003s	sys 0.000s
I20260709 13:07:20.670380  4970 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.684336  5001 raft_consensus.cc:2804] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:07:20.684993 29141 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 26. Current term is 27. Ops: []
I20260709 13:07:20.686028  4970 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:20.681243  4855 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:20.685034  5001 raft_consensus.cc:697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 27 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:20.692101 29186 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:20.692394  4843 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:20.681540  4761 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.693037  4948 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:20.693524 29186 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:20.689062 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 90304.9us
I20260709 13:07:20.718520 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
W20260709 13:07:20.699348 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.721935 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:20.704744  4974 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:20.705701  4970 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:20.722537 29904 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 26.
I20260709 13:07:20.708971  5000 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:20.719372  4907 consensus_queue.cc:1048] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:07:20.691610  5001 consensus_queue.cc:237] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.699659  4796 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.681533  4637 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:07:20.723155  5010 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:20.724226  4830 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:20.737686 29064 raft_consensus.cc:2468] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:07:20.738909  4263 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 26.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 13:07:20.739413  4909 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:07:20.741245  5024 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:07:20.741803  5024 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:20.742071  4830 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.742808  4830 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:20.743607 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:20.743224  5024 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.745177  5024 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:20.748637 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:20.727211  4970 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:20.752341 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:20.735209  5010 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.722632  4761 consensus_queue.cc:1048] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:07:20.761049 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:20.754570  4970 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:20.756798  5010 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:20.762053 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:20.770291 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48582.1us
I20260709 13:07:20.774619 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:20.774940 29070 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:20.775782 29141 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:20.761292  4843 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:20.777490  4843 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.779318 29143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42044 (request call id 3433) took 3057 ms. Trace:
I20260709 13:07:20.784133 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:20.793300 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:20.795323  5006 log.cc:927] Time spent T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.189s	user 0.001s	sys 0.001s
I20260709 13:07:20.795499  4565 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:20.804729 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:20.805980 29222 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 19.
I20260709 13:07:20.808113 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:20.809507 29070 raft_consensus.cc:2410] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:07:20.779718 29143 rpcz_store.cc:276] 0709 13:07:17.721535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.074685 (+1353150us) service_pool.cc:224] Handling call
0709 13:07:19.098742 (+ 24057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:19.779337 (+680595us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:07:19.812182 (+ 32845us) log.cc:844] Serialized 14 byte log entry
0709 13:07:20.571350 (+759168us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b18004eb3e0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78366b54bc 00007f7833a0348b 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f7836b05060 00007f7836b17716 00007f7836b28287
0709 13:07:20.573204 (+  1854us) log.cc:844] Serialized 14 byte log entry
0709 13:07:20.604346 (+ 31142us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b18004eb3e0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1
0709 13:07:20.605173 (+   827us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b18004eb3e0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f7833a203f3 00007f7833a227c1 00007f7833a22779 00007f7833a22709 00007f7833a226d1 00007f7833a219cd 000000000038f354 000000000038f199 00007f7833a0383a
0709 13:07:20.620748 (+ 15575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:07:20.623263 (+  2515us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:20.625837 (+  2574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:20.778954 (+153117us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54003cc3d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:20.778995 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:20.779140 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59202,"prepare.queue_time_us":39267,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":97,"raft.queue_time_us":50153,"raft.run_cpu_time_us":538,"raft.run_wall_time_us":19742,"spinlock_wait_cycles":1743734144,"thread_start_us":196622,"threads_started":2,"wal-append.queue_time_us":166039}
I20260709 13:07:20.815644 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "128a7406f55d4400bfd5fadcbc14d065" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:20.817690 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:20.818543 29216 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:20.820401  5013 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:20.821828  4888 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.792176  4843 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.796063  4565 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.830004  4565 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:20.833040 29188 leader_election.cc:400] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24.
I20260709 13:07:20.833513 29188 leader_election.cc:403] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:07:20.834709  4965 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24.
I20260709 13:07:20.842260 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d4b885cd86484fbaa6c528fc2ed1bc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:20.846781  4864 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:20.847038 29113 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:20.847519  4864 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.850189  4864 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.850843 29910 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:07:20.850927  4909 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:20.851491 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:20.854497 28964 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 13 to 18, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 18 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:20.863441 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:20.864460 29221 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 19: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:20.865118 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:20.866487 29214 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 24: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:20.868183 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:20.869800 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:20.861515 29113 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:20.871093  4888 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.871874 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:20.872192  4538 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:20.872394  4637 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:07:20.872967  4637 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.872809  4538 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.873492  4637 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:20.873718 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:20.874879  4782 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:20.875203  4538 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.875602  4782 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:20.875772  4778 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:07:20.877126 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:20.876911  4782 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:20.882793 29872 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.884265 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:20.884807 28964 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 14 to 18, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 18 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:20.884316 29113 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:20.888576 29113 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:20.888196 28964 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:20.889633  4909 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:20.890636  4909 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:20.891219  4909 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.891664  4909 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:20.846215 29064 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:20.890874 28964 catalog_manager.cc:5697] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 10 to 13, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 13 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:20.843180 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:20.898711 29874 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:20.903535 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:20.904258 29113 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:20.902978 29220 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:07:20.896591  4888 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.905495 29910 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:07:20.908720 29064 raft_consensus.cc:2368] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24.
I20260709 13:07:20.913441  4864 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:20.910648 29188 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:20.915252  4843 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:20.915517  4864 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 26 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:20.916754  4796 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:07:20.917336  4796 raft_consensus.cc:697] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:20.918557  4796 consensus_queue.cc:237] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.920156  4778 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:20.920547 29216 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:20.921854  4635 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:07:20.922264  4635 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:20.916895  4864 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.923280  4635 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.926049  4635 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:20.927117 29903 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 27.
I20260709 13:07:20.932044  4844 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:20.931928  4743 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:20.900377  4538 consensus_queue.cc:1059] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:07:20.922286  4816 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:20.933459  4830 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.941241 29217 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 23: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:07:20.943385 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:20.909318  4761 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.901671  5025 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:20.954175 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:20.945055 28965 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 264) took 100787 ms (client timeout 15000 ms). Trace:
I20260709 13:07:20.956571  4965 raft_consensus.cc:3055] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:07:20.957556  4965 raft_consensus.cc:740] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:20.937420  4778 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:07:20.958935  4965 consensus_queue.cc:260] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.967861 29906 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:20.943001  4844 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:07:20.949954  4565 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.970901 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:20.974948  4783 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:20.969638  4637 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:20.976364  5041 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:07:20.976243  4535 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:20.978148  4637 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:20.978981 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:20.979636 29222 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:20.981680  5025 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:20.979758  4535 consensus_queue.cc:1048] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:07:20.985325  4965 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:20.988978 29217 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:07:20.990097  4565 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:20.990176  4761 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.992250 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:20.992399 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:20.992661  4761 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:20.991591 28964 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 11 to 21. New cstate: current_term: 21 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:20.996392  5013 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:20.993441  4761 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:20.997412  4761 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.997638  4909 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:20.997994  5013 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:07:20.998729 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 207415us
I20260709 13:07:21.000033  4761 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.000334 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:21.000948 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:21.022066  5013 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.001570 28964 catalog_manager.cc:5697] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:21.002661 29071 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 21. Current term is 22. Ops: []
I20260709 13:07:21.024147 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:21.018993 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 43686.2us
I20260709 13:07:21.029366 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:21.032862 29222 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:07:20.968981  4888 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.039706 29064 raft_consensus.cc:1275] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
W20260709 13:07:21.041083 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.025916  5013 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:21.044974 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.048089 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:07:21.048480  5045 log.cc:927] Time spent T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.063s	user 0.003s	sys 0.000s
I20260709 13:07:21.049023 29071 raft_consensus.cc:1275] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:07:21.049366  5004 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:21.034550  4637 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.031587  4761 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.053850 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:07:21.057845  5046 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 13:07:21.058425  5047 log.cc:927] Time spent T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.083s	user 0.002s	sys 0.000s
I20260709 13:07:21.058338 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.060343 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:21.055140  5004 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:21.064996 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 115647us
I20260709 13:07:21.067137 29112 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:21.067389 29906 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
W20260709 13:07:20.957324 28965 rpcz_store.cc:269] 0709 13:05:40.099656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.128767 (+ 29111us) service_pool.cc:224] Handling call
0709 13:07:20.883821 (+100755054us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:12.100915 (+     0us) spinlock_profiling.cc:243] Waited 7.84 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f7833a030ac 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1d455 00007f783a54216c
0709 13:07:12.250303 (+149388us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:12.250971 (+   668us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:12.251020 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:07:12.251045 (+    25us) tablet.cc:662] Decoding operations
0709 13:07:12.251443 (+   398us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:12.251531 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:12.251558 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:07:12.251884 (+   326us) tablet.cc:701] Row locks acquired
0709 13:07:12.251963 (+    79us) write_op.cc:260] PREPARE: finished
0709 13:07:12.252133 (+   170us) write_op.cc:270] Start()
0709 13:07:12.252359 (+   226us) write_op.cc:276] Timestamp: P: 1783602432252068 usec, L: 0
0709 13:07:12.252411 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 13:07:12.253141 (+   730us) log.cc:844] Serialized 536 byte log entry
0709 13:07:12.628688 (+375547us) op_driver.cc:464] REPLICATION: finished
0709 13:07:12.912889 (+284201us) write_op.cc:301] APPLY: starting
0709 13:07:12.925446 (+ 12557us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:12.927452 (+  2006us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:12.927505 (+    53us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:13.075844 (+148339us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:13.161611 (+ 85767us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:07:13.180797 (+ 19186us) write_op.cc:312] APPLY: finished
0709 13:07:13.213024 (+ 32227us) log.cc:844] Serialized 25 byte log entry
0709 13:07:13.213238 (+   214us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:13.213342 (+   104us) write_op.cc:454] Released schema lock
0709 13:07:13.213415 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:13.213889 (+   474us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:13.213928 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":597699,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":243995,"apply.run_cpu_time_us":5629,"apply.run_wall_time_us":312452,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8307,"num_ops":1,"prepare.queue_time_us":160655,"prepare.run_cpu_time_us":7009,"prepare.run_wall_time_us":51668,"raft.queue_time_us":192989,"raft.run_cpu_time_us":33737,"raft.run_wall_time_us":165316,"replication_time_us":376115,"spinlock_wait_cycles":18041216,"thread_start_us":368177,"threads_started":4,"wal-append.queue_time_us":73016,"wal-append.run_cpu_time_us":12727,"wal-append.run_wall_time_us":1910907}]]}
I20260709 13:07:21.070379  4761 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.073314 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:21.076094 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.041761  4830 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:21.065021 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.086860 29143 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:07:21.072754 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:21.050081  4565 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:21.092240 29144 raft_consensus.cc:2393] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:21.095605 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:07:21.104171 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 50387.3us
I20260709 13:07:21.088184  5013 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:07:21.113790  5040 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:21.114542  5040 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.108325 29069 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:21.120615  4565 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:21.123312 28965 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 244) took 17558 ms (client timeout 15000 ms). Trace:
W20260709 13:07:21.123845 28965 rpcz_store.cc:269] 0709 13:07:03.569414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.123027 (+17553613us) service_pool.cc:211] Skipping call since client already timed out
0709 13:07:21.123223 (+   196us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:07:21.123296  5051 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 13:07:21.126699 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.133970 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.099520  4637 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.134670 29148 raft_consensus.cc:2468] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:21.062234  5004 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.136650 29147 raft_consensus.cc:2468] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 28.
I20260709 13:07:21.149341 29188 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:21.150202  4796 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:21.150401  4635 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:07:21.151844  4796 raft_consensus.cc:2804] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:21.152554  4796 raft_consensus.cc:697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:21.153717  4796 consensus_queue.cc:237] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.149221  4535 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:07:21.131799 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 58375.3us
I20260709 13:07:21.090032  4907 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:21.160401  4761 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.161316 29147 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:07:21.161559  5040 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.161877 29064 raft_consensus.cc:3055] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:07:21.162276 29064 raft_consensus.cc:740] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:21.162786  4637 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.165905 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:21.166880  4804 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:21.163151 29064 consensus_queue.cc:260] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:07:21.179476 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:21.181244 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:21.181645 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.181967 29069 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:21.183032 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:21.160501  4907 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.189546 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:21.190341 29065 raft_consensus.cc:2368] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21.
I20260709 13:07:21.190546  4907 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.191257 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:21.192243 29064 raft_consensus.cc:3060] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:21.192685  4377 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:07:21.197609 29065 raft_consensus.cc:2468] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:21.128294  4565 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:07:21.196717 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:21.195003  4377 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:07:21.181288 29066 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:07:21.191293  4970 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.207521 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:21.196491 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:21.186249  5010 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.208758 29217 raft_consensus.cc:3060] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:21.208877 29145 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:21.204694 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:21.211598 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:21.214198 28964 catalog_manager.cc:5697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 7 to 17. New cstate: current_term: 17 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:21.217376 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:21.224416 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:21.225881 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:21.226657 29903 raft_consensus.cc:2368] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21.
I20260709 13:07:21.229892 29217 raft_consensus.cc:2468] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:21.245800 29064 raft_consensus.cc:1275] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:07:21.246347 29064 pending_rounds.cc:85] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Aborting all ops after (but not including) 13
I20260709 13:07:21.250015 29216 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 25: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:07:21.254870 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 72005.5us
I20260709 13:07:21.266194  4782 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.211337  4970 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:21.273252 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.274225 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:21.278211 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.279335  4909 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.279687 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:21.280324 29069 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:07:21.276916 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:21.282512 29065 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
W20260709 13:07:21.289520 29874 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.250996  5010 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.246984 29064 pending_rounds.cc:107] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 21.14
I20260709 13:07:21.309396 29064 raft_consensus.cc:2917] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Aborting config change with OpId 21.14: Aborted: Op aborted by new leader
I20260709 13:07:21.309971 29064 raft_consensus.cc:2889] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:21.302245 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.315147 29142 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:21.324383 29186 leader_election.cc:400] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21.
I20260709 13:07:21.324846 29186 leader_election.cc:403] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:07:21.282783  4782 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.333843  4782 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:21.338217 29070 raft_consensus.cc:1275] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:07:21.351270 29874 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:21.373901 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 126401us
I20260709 13:07:21.376219 29065 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
W20260709 13:07:21.376598 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.377609 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.377975 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 62662.1us
W20260709 13:07:21.377986 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.325515  4907 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:21.335516  5013 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:21.384179 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:21.348626  4565 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.383774 29064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:48862 (request call id 1) took 1994 ms. Trace:
I20260709 13:07:21.380429 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.337258  4888 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:21.388841 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:21.391300  4761 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:21.387894  4377 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.392410  4377 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.396456 29871 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:21.376353  5025 raft_consensus.cc:2804] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:21.401371 29903 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:07:21.404843 29141 raft_consensus.cc:1240] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
W20260709 13:07:21.408046 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.400458  4565 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.409940  5025 raft_consensus.cc:697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:21.410446  4804 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:07:21.411005  4804 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 27 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:21.411319  4565 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:21.406337 29146 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 22.
I20260709 13:07:21.412034  4804 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.412410  5013 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.385869 29064 rpcz_store.cc:276] 0709 13:07:19.382819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:19.383136 (+   317us) service_pool.cc:224] Handling call
0709 13:07:19.588885 (+205749us) log.cc:844] Serialized 506 byte log entry
0709 13:07:21.355764 (+1766879us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19407,"raft.queue_time_us":49805,"raft.run_cpu_time_us":1959,"raft.run_wall_time_us":10873,"spinlock_wait_cycles":6016,"thread_start_us":365762,"threads_started":2,"wal-append.queue_time_us":286193,"wal-append.run_cpu_time_us":1746,"wal-append.run_wall_time_us":1001756}
I20260709 13:07:21.382331  4535 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21.
I20260709 13:07:21.395623  4377 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.355887  4705 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:21.399757  4907 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.426406 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:21.437183  5054 log.cc:927] Time spent T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:07:21.437412 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet b55b583a63b64ca6832b73f813985e49: Aborted: Op aborted by new leader
I20260709 13:07:21.438722 29188 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:21.441112  4565 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:21.444216 29141 raft_consensus.cc:3060] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:21.452265  4661 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:21.450691 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:21.457042 29039 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:21.459057 29910 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:21.464254  4705 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.478343 29141 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:07:21.479380 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:21.462664  4864 raft_consensus.cc:2804] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:21.411021  5025 consensus_queue.cc:237] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.480567 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.532186 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:21.480702  4377 raft_consensus.cc:2804] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 13:07:21.538657 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.546310 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.526223  4970 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:07:21.524569  4864 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:07:21.553719 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.575909 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:21.522079  5048 consensus_queue.cc:1059] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:07:21.550541  4855 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:21.549081  4888 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:07:21.584820 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.590229 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.556087  4970 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.572635  4864 raft_consensus.cc:3060] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:21.599573 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 154180us
I20260709 13:07:21.587448  4796 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.602073 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:21.602622  4796 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.605237  4796 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:21.594882  4970 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:21.592571  5004 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:07:21.608600 29185 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:41631 timed out after 1.635s (SENT) [suppressed 19 similar messages]
I20260709 13:07:21.597885  4661 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:21.631951  5012 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:21.632604  5012 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.638132  4970 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.659083  4661 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:21.660022  4930 raft_consensus.cc:3055] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:07:21.660523  4930 raft_consensus.cc:740] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
W20260709 13:07:21.661121  5053 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.327s	user 0.002s	sys 0.000s
I20260709 13:07:21.650553  5004 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.661543  4930 consensus_queue.cc:260] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.662937  4930 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:21.671964 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:21.679952 29216 raft_consensus.cc:2410] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:07:21.681167 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.687575 29140 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:21.694018 29069 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:21.693949  5069 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.700845  5069 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:21.704402 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 106644us
I20260709 13:07:21.659749  4661 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.710347 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:21.716581  4909 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:07:21.719584  4930 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:21.720295  4930 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.722316 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:21.728098  4377 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:21.728314 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:21.728695  4377 raft_consensus.cc:3060] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:07:21.728921 29185 leader_election.cc:400] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 23. Current term is 24.
I20260709 13:07:21.696965  5004 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:21.722844  4930 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:21.722949 29214 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:21.711045 29906 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
I20260709 13:07:21.738675 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.736861 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:21.764842 29145 raft_consensus.cc:2468] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 20.
I20260709 13:07:21.769738  4930 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:21.771293 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:21.772953 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:21.687939  4970 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:21.775106 29145 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:21.780469 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:21.782418 29222 raft_consensus.cc:1275] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:07:21.793210 29216 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:07:21.774204  4930 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.794413  4930 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:21.796299  5069 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.797744 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:21.804571  4888 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.808264  4855 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:21.808910  5069 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.811450  5069 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.809381  5080 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.813318  5080 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.816004  4843 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:21.768563  5004 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.817334  5004 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.822857 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.837985 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.863479  4855 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.866019  4855 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:21.866361  5065 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.308s	user 0.000s	sys 0.002s
I20260709 13:07:21.866612  4855 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:21.868901  5069 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:21.869295  5079 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.141s	user 0.002s	sys 0.000s
I20260709 13:07:21.805321  4888 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.871419  4888 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:21.853838 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 73461.5us
I20260709 13:07:21.872681 29144 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:07:21.874039  4948 consensus_queue.cc:1048] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:07:21.874428 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.874927 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:21.877144  5077 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.218s	user 0.002s	sys 0.000s
I20260709 13:07:21.880148 29142 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:07:21.881222  4661 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:21.881623 29145 raft_consensus.cc:2468] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:21.883220  4948 consensus_queue.cc:1048] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:21.882452 29142 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [18.12-19.13]
I20260709 13:07:21.884874 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.882620  5012 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.883420  4705 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.867332  4855 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.886300 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:21.900039 29142 raft_consensus.cc:2468] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:21.891474 29186 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:21.896912 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 66844.1us
I20260709 13:07:21.900599  4909 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:21.911507  4909 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:21.912200  5082 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.255s	user 0.003s	sys 0.000s
I20260709 13:07:21.913692  4909 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:21.913856  4804 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:07:21.913645 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.917294 29145 raft_consensus.cc:2393] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 24: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:21.919468 29145 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:07:21.921135  5083 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 13:07:21.923343  5091 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:07:21.925830 29147 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:07:21.915262  4930 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:21.926354  5064 log.cc:927] Time spent T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.265s	user 0.002s	sys 0.000s
I20260709 13:07:21.927753  4930 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:21.928048 29147 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 13:07:21.928967 29143 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:21.929946 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.930552 29143 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:21.930747 29147 raft_consensus.cc:1275] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:07:21.932082  4350 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:21.932814  4350 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.932993  5069 consensus_queue.cc:1059] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:21.935017  5069 raft_consensus.cc:3055] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:07:21.935516  5069 raft_consensus.cc:740] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:21.932825  4661 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:07:21.940946 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:21.942534  5069 consensus_queue.cc:260] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.945227  5069 raft_consensus.cc:3060] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:21.949149 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 57813.3us
I20260709 13:07:21.937007 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:07:21.938217  5084 log.cc:927] Time spent T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:07:21.938862 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:21.936280 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.884887  4907 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:21.958784 29037 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.952666 29188 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:21.961304 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:21.889405  4635 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:07:21.929005  5059 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.349s	user 0.000s	sys 0.002s
I20260709 13:07:21.963583 29039 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:21.935716  4350 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:21.974721 29188 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:21.964716  4965 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:21.956043  5086 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:21.902333  4843 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:21.973603  4830 raft_consensus.cc:2804] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:07:21.998670 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 44622.5us
I20260709 13:07:21.978928  4888 raft_consensus.cc:2804] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:22.015131 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:22.016350 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:22.016673  4830 raft_consensus.cc:697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:22.017627 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:22.018397 29221 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 21: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:22.017644  4830 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:21.968987  4661 consensus_queue.cc:1059] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 13:07:22.020615 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.022394 29222 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:21.995817  4965 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.996469  5086 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.036598 29902 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 27 index: 22. (index mismatch)
W20260709 13:07:22.004451 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:21.988646  4565 raft_consensus.cc:2804] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:21.987589  5004 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:22.020431  5060 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.365s	user 0.005s	sys 0.000s
I20260709 13:07:22.003193  4888 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.998518  4535 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:07:22.076768 29216 raft_consensus.cc:1240] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 20. Current term is 21. Ops: []
W20260709 13:07:22.108304 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 198459us
I20260709 13:07:22.111445 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:22.039889  4965 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:07:22.126036 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 127475us
I20260709 13:07:22.041165  5080 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:22.039288  5086 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:07:22.130081 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 144632us
I20260709 13:07:22.070574  4565 raft_consensus.cc:697] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:22.114387  4888 raft_consensus.cc:3060] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:22.040243  4909 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.085s
W20260709 13:07:22.142848 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.138001 29039 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:07:22.143587 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:22.144979 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 94104.9us
I20260709 13:07:22.129794  5085 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.111630  5092 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.092s
I20260709 13:07:22.158546 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:22.159194 29902 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 25
W20260709 13:07:22.154157 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:22.149233  4864 consensus_meta.cc:287] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.005s
W20260709 13:07:22.140816  5025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.023s	sys 0.060s
I20260709 13:07:22.161198 29140 raft_consensus.cc:1275] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:07:22.164433 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:22.165632 29147 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:22.140074  5004 consensus_queue.cc:1059] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 13:07:22.166473  5025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.024s	sys 0.060s
I20260709 13:07:22.199010 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:22.142062  5093 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:07:22.204805 29214 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:22.157521  4377 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:22.218621  5095 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:07:22.220142  4377 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:22.220963  5061 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.571s	user 0.003s	sys 0.001s
I20260709 13:07:22.222476 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:22.222471 29068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3400) took 1059 ms. Trace:
W20260709 13:07:22.228657  5070 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.568s	user 0.000s	sys 0.003s
W20260709 13:07:22.230374  5075 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.602s	user 0.000s	sys 0.005s
I20260709 13:07:22.231179 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:22.231712  5089 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:07:22.228437 29068 rpcz_store.cc:276] 0709 13:07:21.162889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.164764 (+  1875us) service_pool.cc:224] Handling call
0709 13:07:21.165148 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:21.165545 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:21.165572 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:21.166036 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:07:21.476726 (+310690us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:21.476783 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.476870 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:22.144450 (+667580us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:22.222064 (+ 77614us) raft_consensus.cc:1685] finished
0709 13:07:22.222085 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.222292 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":343327488,"thread_start_us":310235,"threads_started":1,"wal-append.queue_time_us":423570}
W20260709 13:07:22.212764 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 231374us
I20260709 13:07:22.213129 29065 raft_consensus.cc:1275] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:07:22.214947  4843 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:22.215103 29073 raft_consensus.cc:1275] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:07:22.172783  4864 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.163950  5085 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:22.265516 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.268541 29069 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 26.
I20260709 13:07:22.213609  4855 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:22.266741 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.192826  4796 consensus_queue.cc:1048] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:07:22.266173 29902 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:22.317298 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.318655 29069 raft_consensus.cc:2410] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 20 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 13:07:22.284696  5074 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.564s	user 0.003s	sys 0.000s
I20260709 13:07:22.315263 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:22.315874  5062 log.cc:927] Time spent T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.697s	user 0.002s	sys 0.002s
I20260709 13:07:22.308944  4843 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:22.362638 29113 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:22.360443 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:22.360522 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:22.313161  5004 raft_consensus.cc:3055] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:07:22.359509 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 54471.7us
I20260709 13:07:22.361310 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:22.366310 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.369518 29214 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 24: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:22.370847 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:22.371815 29216 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 25: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:22.378437 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.240271  4565 consensus_queue.cc:237] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.379383 29071 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:22.373734  4907 raft_consensus.cc:2804] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:22.362519  4843 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.368227  5004 raft_consensus.cc:740] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:22.397671 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 377249us
W20260709 13:07:22.400810 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 362782us
I20260709 13:07:22.401319 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:22.405033 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:22.411301 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 259234us
I20260709 13:07:22.390496  5103 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.379384  5080 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:07:22.422789 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:22.425665 29069 raft_consensus.cc:3060] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:07:22.427403 29185 leader_election.cc:400] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Vote denied by peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631) with higher term. Message: Invalid argument: T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 20. Current term is 21.
I20260709 13:07:22.417695  4907 raft_consensus.cc:697] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:22.405623  4843 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:07:22.452574 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 63865us
I20260709 13:07:22.424535  4974 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.462600 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:22.424350  4377 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:22.465646 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 228358us
W20260709 13:07:22.466375  5101 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:07:22.448873  5103 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.466815 29139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3404) took 1271 ms. Trace:
I20260709 13:07:22.469251  5106 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.464465 29067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47974 (request call id 3397) took 1302 ms. Trace:
I20260709 13:07:22.426370  4761 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:22.484727 29186 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:22.451987  5110 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.496035  5106 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:22.499047  5106 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:22.485699 29067 rpcz_store.cc:276] 0709 13:07:21.161746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.164059 (+  2313us) service_pool.cc:224] Handling call
0709 13:07:21.164392 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:21.166724 (+  2332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:21.166752 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:21.167217 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:07:21.407694 (+240477us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:07:21.407752 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.407837 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:22.126162 (+718325us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:22.429115 (+302953us) raft_consensus.cc:1685] finished
0709 13:07:22.429138 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.464367 (+ 35229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1914,"raft.queue_time_us":2079,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":409858304,"thread_start_us":240016,"threads_started":1,"wal-append.queue_time_us":371729}
I20260709 13:07:22.510135 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:22.511408 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:22.579603 29069 raft_consensus.cc:2468] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 23.
I20260709 13:07:22.581612 29146 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 19. (term mismatch)
I20260709 13:07:22.582130 29146 pending_rounds.cc:85] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Aborting all ops after (but not including) 18
I20260709 13:07:22.582559 29146 pending_rounds.cc:107] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Aborting uncommitted NO_OP operation due to leader change: 26.19
I20260709 13:07:22.582881 29146 raft_consensus.cc:2889] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:22.450173  5109 consensus_queue.cc:1048] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.087s
I20260709 13:07:22.573239 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
W20260709 13:07:22.622956 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 138299us
W20260709 13:07:22.587936  5101 log.cc:927] Time spent T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:07:22.626652 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:22.627635 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:22.569798  4864 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:22.484776 29139 rpcz_store.cc:276] 0709 13:07:21.195235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.207092 (+ 11857us) service_pool.cc:224] Handling call
0709 13:07:21.207524 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:21.207962 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:07:21.309691 (+101729us) log.cc:844] Serialized 14 byte log entry
0709 13:07:21.714101 (+404410us) log.cc:844] Serialized 14 byte log entry
0709 13:07:21.714287 (+   186us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:21.714778 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:07:21.714902 (+   124us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:07:21.714933 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.714996 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:22.284031 (+569035us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:22.466423 (+182392us) raft_consensus.cc:1685] finished
0709 13:07:22.466458 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.466605 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":161776,"prepare.queue_time_us":189939,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":81,"raft.queue_time_us":20959,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":16658,"spinlock_wait_cycles":324085632,"thread_start_us":307451,"threads_started":2,"wal-append.queue_time_us":350634}
I20260709 13:07:22.513954  4965 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.488556  5012 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.129s
I20260709 13:07:22.476615  4974 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:22.515513  4761 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.619122  4635 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.475996  5004 consensus_queue.cc:260] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.643478  5085 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:22.525923  5110 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.665047 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.638523  5040 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.668561 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.648787  5115 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.669816 29069 raft_consensus.cc:2393] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 24: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:22.675613 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:22.679973  5076 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.816s	user 0.008s	sys 0.005s
W20260709 13:07:22.681051  5114 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:07:22.681454 29148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42054 (request call id 3398) took 1488 ms. Trace:
I20260709 13:07:22.632781  4970 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 13:07:22.682366 29219 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:22.661244  4796 raft_consensus.cc:2804] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:07:22.683336 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 172021us
I20260709 13:07:22.682224 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:22.668955  5099 log.cc:927] Time spent T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 13:07:22.681838 29148 rpcz_store.cc:276] 0709 13:07:21.193275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.204538 (+ 11263us) service_pool.cc:224] Handling call
0709 13:07:21.226867 (+ 22329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:21.251079 (+ 24212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:21.251112 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:21.297411 (+ 46299us) log.cc:844] Serialized 26 byte log entry
0709 13:07:21.528026 (+230615us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:07:21.528156 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:21.528309 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:07:22.314686 (+786377us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:22.681160 (+366474us) raft_consensus.cc:1685] finished
0709 13:07:22.681182 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:07:22.681396 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":735,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":18760,"spinlock_wait_cycles":462910208,"thread_start_us":172214,"threads_started":1,"wal-append.queue_time_us":319640}
I20260709 13:07:22.650853  4635 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.691039  5123 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:22.691565  5116 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:07:22.689143 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:22.692703  4635 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:22.693269  5022 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:22.693432  4864 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:22.631215  4907 consensus_queue.cc:237] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.696877 29071 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:07:22.697062  5096 log.cc:927] Time spent T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:07:22.698653  5134 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:22.689756  5081 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.648s	user 0.006s	sys 0.000s
W20260709 13:07:22.692613  5119 log.cc:927] Time spent T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:07:22.677903  5004 raft_consensus.cc:3060] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:07:22.700932 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 183921us
I20260709 13:07:22.709013 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.690434  5127 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:07:22.716388 29111 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:22.716977 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:22.721765 29910 raft_consensus.cc:2393] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 25: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:22.686146  5115 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:22.723520  4974 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:22.724602  5115 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:22.725469  5069 consensus_meta.cc:287] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.780s	user 0.008s	sys 0.004s
I20260709 13:07:22.726904  4843 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.728268  5131 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:22.728864 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.729511 29073 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 27.
I20260709 13:07:22.729070  5131 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:22.731490  4888 consensus_meta.cc:287] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.576s	user 0.005s	sys 0.002s
I20260709 13:07:22.732115 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:22.732237 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 50125.1us
I20260709 13:07:22.733546 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:22.734565 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:22.735324 29902 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
W20260709 13:07:22.713147 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 83405.9us
I20260709 13:07:22.743445  4843 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:22.744264  4705 consensus_queue.cc:1048] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:07:22.745108  4705 consensus_queue.cc:1243] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:07:22.755888  5103 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:22.688601  5040 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.759423  4965 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:22.760030  5118 consensus_queue.cc:1048] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:07:22.761590 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 77605us
I20260709 13:07:22.761590 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:22.700004  4796 raft_consensus.cc:697] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:22.793934 29906 raft_consensus.cc:2393] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 27: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:22.777190 29111 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:22.792550 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:22.782845 29037 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:22.799768 29071 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 27 index: 22. (index mismatch)
I20260709 13:07:22.801285 29222 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:07:22.801443 29037 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:22.802932 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:22.762928  5012 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:07:22.803586 29139 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
W20260709 13:07:22.806087 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 72628.9us
I20260709 13:07:22.805575 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:22.808101 29038 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:22.808043 29037 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:22.816630 29037 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:22.737100  5134 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.824291  5120 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:22.825124  5102 log.cc:927] Time spent T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: Append to log took a long time: real 0.363s	user 0.000s	sys 0.002s
I20260709 13:07:22.824918  5120 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.825656  5125 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:22.825876  5134 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:22.741406  5131 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:22.792670  5069 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:22.823457 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 46376us
I20260709 13:07:22.829092 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:22.733800  4888 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.833544 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:22.830691 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 55850.2us
I20260709 13:07:22.841324 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:22.809738  4843 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:22.851688 29111 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:22.853745 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:22.856660 29111 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:22.846009  5131 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.839490  4965 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:22.839881  4855 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:22.866564 29071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.867352 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.875847 29069 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:07:22.869449  5111 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:22.869896  5108 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:22.872618  4705 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:22.878011 29111 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:22.857139  5085 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:22.861519  4661 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:22.886300 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.888833 29073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:22.889458 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:22.892697 29066 raft_consensus.cc:1275] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:07:22.839690  5134 consensus_queue.cc:1048] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:07:22.900292 29071 raft_consensus.cc:3060] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:07:22.900251 29064 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 23: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:22.905009 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.905128 29070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:22.875100  4855 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:07:22.915736  4855 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:22.906436  4796 consensus_queue.cc:237] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:22.895952  5131 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:22.923417 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:22.924063 29066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.925657 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:22.926836 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1"
I20260709 13:07:22.895056  5085 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:22.927485  5085 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:22.928725  5144 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:22.929524  5144 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:07:22.912067 27934 tablet_server.cc:179] TabletServer@127.27.71.129:0 shutting down...
I20260709 13:07:22.912375 29070 raft_consensus.cc:2393] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 21: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
I20260709 13:07:22.913480 29072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.893436  4965 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:22.927631 29069 raft_consensus.cc:2393] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 23: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:22.890635  5093 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:22.908560  5124 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.053s
I20260709 13:07:22.922827  5131 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:22.908360 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 52617.3us
I20260709 13:07:22.938787 29071 raft_consensus.cc:1275] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:07:22.943552  4705 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:22.943763 29068 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:07:22.946535 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.947052 29067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.864964  4864 consensus_queue.cc:1059] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:07:22.944559 29069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a936878f59c840a6bd176a1d2d7347b1" is_pre_election: true
I20260709 13:07:22.947178 29064 raft_consensus.cc:2468] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:07:22.947332 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:22.944342 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 61933.6us
I20260709 13:07:22.947705 29067 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 27.
I20260709 13:07:22.965901 29071 raft_consensus.cc:1240] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 25. Current term is 26. Ops: []
I20260709 13:07:22.945269  5120 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:22.950829  4705 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:07:22.977067 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 99168.7us
I20260709 13:07:22.980036 29111 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:22.981534 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:22.975068  4350 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:22.945628  5093 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:22.986163 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:22.975828  5110 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:22.977499  4705 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:22.997442 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:22.983769  4350 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:22.983836  5134 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:23.024916 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:23.004077  5131 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:22.996510  5145 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:07:23.035162  5147 log.cc:927] Time spent T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:07:23.041328  5151 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:23.044845 29111 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:23.045778  5058 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:23.048089 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:23.048943  5058 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:07:23.049365  5058 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.049731  5058 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:23.009143  4888 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.026542  4350 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:23.008172  5143 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:23.055066 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 88234.3us
I20260709 13:07:23.068344 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.081274 29140 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:23.069288 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
W20260709 13:07:23.030814  5086 consensus_meta.cc:287] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Time spent flushing consensus metadata: real 0.875s	user 0.006s	sys 0.013s
I20260709 13:07:23.026032  4907 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:23.100206 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.100453 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:23.103313 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.105294 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.073534  5013 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.040s
I20260709 13:07:23.132579 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:23.100219  5086 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:23.130785  4965 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:23.160020  5086 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:23.164708 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:23.166257 29908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:07:23.168188  5130 log.cc:927] Time spent T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.465s	user 0.004s	sys 0.000s
I20260709 13:07:23.160197 29218 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 13:07:23.181348  5093 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.181654  4350 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:23.183118 29904 raft_consensus.cc:2393] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 25: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:23.186590  4661 raft_consensus.cc:3055] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:23.187004  4661 raft_consensus.cc:740] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:23.126595  4855 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:23.187867  4661 consensus_queue.cc:260] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.189751  4855 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:23.189669 29066 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:07:23.189139  4661 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:23.197177  4705 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.202311  4350 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:23.204891  5085 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.205544  4705 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:23.208231 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:23.212316 29188 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.212409  5040 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.207033  5085 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.207960  5058 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:23.212916 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:23.213174  5131 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:23.221637 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.209583 29065 raft_consensus.cc:1275] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:07:23.218700 29910 raft_consensus.cc:2368] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25.
I20260709 13:07:23.217971  5086 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.222343 29145 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
I20260709 13:07:23.224326 29037 leader_election.cc:400] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25.
I20260709 13:07:23.224670 29037 leader_election.cc:403] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:07:23.225358  5118 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.227612  4970 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:23.228116 29147 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:07:23.228307  4970 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.229512  4761 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.222759  5120 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.230535  4970 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:23.227753  5118 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:23.219699  4830 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.240355 29145 raft_consensus.cc:1240] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Rejecting Update request from peer 6ad9faa1c14444b29c9db8e71f46e2d6 for earlier term 26. Current term is 27. Ops: []
I20260709 13:07:23.230893  5120 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.244843 29147 raft_consensus.cc:1275] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:07:23.249120 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.226217  5092 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:23.226374  5086 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.295019  5118 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:23.253104  4830 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.303274 29219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3433) took 1289 ms. Trace:
I20260709 13:07:23.307096 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:23.303833 29219 rpcz_store.cc:276] 0709 13:07:22.013960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.014236 (+   276us) service_pool.cc:224] Handling call
0709 13:07:22.014597 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:22.682814 (+668217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.302944 (+620130us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54003561d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:23.303172 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485642624}
I20260709 13:07:23.314023  5013 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:23.314213 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:23.315117 29214 raft_consensus.cc:2410] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:07:23.317855 29147 raft_consensus.cc:1240] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Rejecting Update request from peer 7e78ffbcc98242068a874a5d9e341477 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 13:07:23.314700  5013 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:23.325170  5013 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:23.325485  4830 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:23.327708  5120 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:23.329660  5086 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:23.319761 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.330892  4855 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.331429  4855 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.333580  4855 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.333668  5105 raft_consensus.cc:3055] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:23.334095  5105 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:23.334872  5105 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.290249  5151 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:23.337186 29218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42970 (request call id 3560) took 1002 ms. Trace:
I20260709 13:07:23.337610 29218 rpcz_store.cc:276] 0709 13:07:22.335289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:22.335434 (+   145us) service_pool.cc:224] Handling call
0709 13:07:22.359982 (+ 24548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:23.170776 (+810794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:23.336942 (+166166us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5400339ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:23.337128 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839678848}
I20260709 13:07:23.337152 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:23.339648  5093 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:23.340210  5093 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:23.342501  5093 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:23.344189  5105 raft_consensus.cc:3060] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:23.344292 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:23.346066  5093 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:23.346570  5093 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.346972  5093 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:23.349761 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:23.350903 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:23.351504 29141 raft_consensus.cc:3060] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:23.355413 29039 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:23.373675 29143 raft_consensus.cc:1240] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:23.374413 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:23.377599 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:23.377705  5134 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:23.378319  5134 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.379379  4974 consensus_queue.cc:1059] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:07:23.380157 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:23.380749  4974 raft_consensus.cc:3055] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:07:23.381136  4974 raft_consensus.cc:740] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:23.382066 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:23.382012  4974 consensus_queue.cc:260] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:23.383316  4974 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:23.383724 29188 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:23.378862 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:23.404547 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.405792 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.393414 29188 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:23.407081 29141 raft_consensus.cc:2468] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
W20260709 13:07:23.410808 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 167359us
I20260709 13:07:23.371778  5080 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:07:23.407027 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet cddea8e9cff6495ba269c65b0edf8b6c: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.415802 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.420464 29148 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:23.414659 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:23.405743  4796 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:23.431578 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.434942 29875 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:23.437750 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:23.432523 29142 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:23.413755  5058 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:23.417603  5022 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:23.442996 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.428824  4782 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:23.433590  4796 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.454365 29148 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 27.
I20260709 13:07:23.442323  5127 raft_consensus.cc:2804] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:07:23.466786 29871 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.451471  4782 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.456918  4796 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:23.487123  4635 consensus_queue.cc:1048] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:07:23.488620  4919 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:07:23.503015 29910 raft_consensus.cc:1275] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:07:23.508001 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:23.478471  4782 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:23.515630 29185 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:23.523510 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 91801us
I20260709 13:07:23.513662  4635 consensus_queue.cc:1059] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 13:07:23.512300  4919 consensus_queue.cc:1048] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:07:23.521613  4965 consensus_queue.cc:1048] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:23.525975 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:23.529870 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:23.480527  5127 raft_consensus.cc:697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:07:23.535012 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.541597 29871 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:23.545449 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:23.527138  5092 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) with higher term. Message: Invalid argument: T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25.
I20260709 13:07:23.543061  5080 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:23.548959 29188 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:23.529201  4855 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.551671 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:23.552333 29217 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:23.550724 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:23.540642  4565 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:07:23.550997 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:23.555851 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:23.514767  4930 consensus_queue.cc:1048] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.037s
I20260709 13:07:23.545622  5149 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:23.571691 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:23.549912  5134 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.576357 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:23.574282 29142 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:23.557021  4635 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:07:23.581108 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:07:23.583380 29186 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.550460  4350 consensus_queue.cc:1048] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:07:23.572955  5115 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:23.569520  4565 raft_consensus.cc:697] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:23.576196  5122 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:23.575613  5105 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.554548  5080 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.557278  4855 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.600317 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:23.600510 29221 raft_consensus.cc:1240] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:07:23.601348 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:23.601951 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:23.634742  5167 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:07:23.590356  5127 consensus_queue.cc:237] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:23.635841  5169 log.cc:927] Time spent T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:07:23.637952  5080 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:23.603861 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 48058.5us
I20260709 13:07:23.590152  5108 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.635774 29213 raft_consensus.cc:1240] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:07:23.603564 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:23.596242  5115 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.638597  5080 consensus_queue.cc:1059] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:07:23.608170  5105 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.645491  4855 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:23.646952  5162 log.cc:927] Time spent T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.412s	user 0.005s	sys 0.000s
W20260709 13:07:23.652670  5166 log.cc:927] Time spent T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:07:23.654568 29019 auto_rebalancer.cc:629] Failed to schedule move for tablet 77b0625d42ef413aae4ec112d9ba450b: Illegal state: Replica 6ad9faa1c14444b29c9db8e71f46e2d6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:23.655722  5093 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.641266 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:23.628281 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:23.666313 29111 leader_election.cc:336] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:23.670717 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 74930.7us
I20260709 13:07:23.671638 29871 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:23.673245 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 92541.4us
W20260709 13:07:23.670218 29111 consensus_peers.cc:597] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:23.674450 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:23.628980  4377 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:23.641031  5120 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:23.685904  4796 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.688460 29903 raft_consensus.cc:2410] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for term 28 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 13:07:23.689244  5022 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:07:23.667793  5108 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.689663  5022 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.690032  5022 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:23.691349  5108 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.677938  4948 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:23.692387  4948 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.692763  4948 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:23.693571  5093 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.696071  4377 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.698371  4377 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.698835  4565 consensus_queue.cc:237] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:23.665391  5115 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:23.704332 29214 raft_consensus.cc:1240] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:07:23.710354  5106 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:07:23.711218  5137 consensus_queue.cc:1059] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:23.712455  5137 consensus_queue.cc:1243] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Peer a936878f59c840a6bd176a1d2d7347b1 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:07:23.718947  5138 raft_consensus.cc:3055] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:07:23.719426  5138 raft_consensus.cc:740] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:23.720043  5022 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:23.676821 29111 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.722536 29113 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:23.723560 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 46785.7us
I20260709 13:07:23.726629 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:23.707794 29185 leader_election.cc:336] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.699730  5105 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.732051 29871 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6
I20260709 13:07:23.733942 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:23.729285 29111 leader_election.cc:336] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.740334 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:23.743708 29111 consensus_peers.cc:597] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:23.746348 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:23.750869 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:23.744576 29111 consensus_peers.cc:597] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:23.753321 29111 consensus_peers.cc:597] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:23.753432 29185 leader_election.cc:336] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.747637  5134 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:23.754350 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:23.755887 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:23.724386  4968 consensus_queue.cc:1048] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 13:07:23.738341  5012 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:07:23.757359 29111 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:23.759320 29111 leader_election.cc:336] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.723176  5120 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.762084  5108 raft_consensus.cc:3060] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:23.762099  4350 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:07:23.768025  5125 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:07:23.760691 29111 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.749481  5151 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:23.770783  5173 log.cc:927] Time spent T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:07:23.771075  5022 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:23.749819  5144 consensus_queue.cc:1059] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:07:23.773113 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:23.754513  5025 raft_consensus.cc:2764] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:07:23.754851  5105 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:23.737213  4855 consensus_queue.cc:1059] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260709 13:07:23.775336 29111 leader_election.cc:336] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:23.780479 29185 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:23.781848 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 106598us
I20260709 13:07:23.775358  4350 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:23.788658 29185 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:23.782763 29111 leader_election.cc:336] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:23.798483 29111 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:23.790354  5125 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:23.789439  4350 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:07:23.799978 29111 consensus_peers.cc:597] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:23.781304  4974 consensus_queue.cc:1048] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:07:23.805851  5120 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:23.807410  4782 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.804661  5105 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 13:07:23.850389 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.851130 29139 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:23.852025 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.852782 29139 raft_consensus.cc:2393] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 23: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:23.853606 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.843398 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.853546 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:23.865278 29112 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:23.868323 29220 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 26. Current term is 27. Ops: []
I20260709 13:07:23.868762 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:23.869395  4782 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:23.869968 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:23.870532 29143 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:23.873873  4350 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:23.876457  4948 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.878830  4948 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:23.893441  5105 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:23.894057  5105 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.894452  5105 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:23.897397  5182 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.898087  5182 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:23.901437 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:23.902351 29148 raft_consensus.cc:2393] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 27: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:23.907217 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:23.907979 29139 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:07:23.915685 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 134174us
W20260709 13:07:23.922015 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 149025us
I20260709 13:07:23.923251 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:23.909545  5144 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:07:23.936164 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:23.939625  4350 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 28 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:23.934214  5125 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.943611  4377 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:07:23.922122  5115 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:23.998942  4888 raft_consensus.cc:3055] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:07:23.999225  5188 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:23.999450  4888 raft_consensus.cc:740] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:24.001065  5179 log.cc:927] Time spent T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:07:24.004609 29143 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 26.
I20260709 13:07:24.006193 29039 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:24.021144 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:24.006517  5104 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.014s	sys 0.017s
W20260709 13:07:24.025013  5104 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.015s	sys 0.017s
I20260709 13:07:24.025420  4888 consensus_queue.cc:260] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.021251  5182 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:23.980906  5125 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.031950  5187 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:24.082000  5138 consensus_queue.cc:260] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:24.089975 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 152635us
I20260709 13:07:24.087637  5138 raft_consensus.cc:3060] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:07:24.094154 28964 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 167) took 44359 ms (client timeout 15000 ms). Trace:
W20260709 13:07:24.100759 29111 consensus_peers.cc:597] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:24.101833 29111 consensus_peers.cc:597] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:24.074602  5187 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 26 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:24.120503 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 184448us
I20260709 13:07:24.122639 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:24.125077 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.094713 28964 rpcz_store.cc:269] 0709 13:06:39.747589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:39.766898 (+ 19309us) service_pool.cc:224] Handling call
0709 13:07:24.088698 (+44321800us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:21.793690 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:21.794488 (+   798us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:21.794544 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:07:21.794565 (+    21us) tablet.cc:662] Decoding operations
0709 13:07:21.795522 (+   957us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:21.795866 (+   344us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:21.795908 (+    42us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:07:21.796732 (+   824us) tablet.cc:701] Row locks acquired
0709 13:07:21.796777 (+    45us) write_op.cc:260] PREPARE: finished
0709 13:07:21.796987 (+   210us) write_op.cc:270] Start()
0709 13:07:21.797148 (+   161us) write_op.cc:276] Timestamp: P: 1783602441796917 usec, L: 0
0709 13:07:21.797190 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:07:21.948370 (+151180us) log.cc:844] Serialized 2835 byte log entry
0709 13:07:22.698965 (+750595us) spinlock_profiling.cc:243] Waited 7.86 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:23.053203 (+354238us) op_driver.cc:464] REPLICATION: finished
0709 13:07:23.088116 (+ 34913us) write_op.cc:301] APPLY: starting
0709 13:07:23.088264 (+   148us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:23.115642 (+ 27378us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:23.115686 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:23.184791 (+ 69105us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:23.185104 (+   313us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:07:23.185174 (+    70us) write_op.cc:312] APPLY: finished
0709 13:07:23.185588 (+   414us) log.cc:844] Serialized 73 byte log entry
0709 13:07:23.185789 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:23.185903 (+   114us) write_op.cc:454] Released schema lock
0709 13:07:23.185964 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:23.186382 (+   418us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:23.186410 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":223,"mutex_wait_us":210540,"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":34524,"apply.run_cpu_time_us":13837,"apply.run_wall_time_us":98273,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33387,"num_ops":7,"prepare.queue_time_us":453712,"prepare.run_cpu_time_us":25064,"prepare.run_wall_time_us":387545,"raft.queue_time_us":157564,"raft.run_cpu_time_us":12204,"raft.run_wall_time_us":98309,"replication_time_us":1212358,"spinlock_wait_cycles":18070784,"thread_start_us":577360,"threads_started":4,"wal-append.queue_time_us":444916}]]}
I20260709 13:07:24.136512  5198 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:24.137312 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:24.138856  5198 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:24.114162  5182 raft_consensus.cc:3055] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:24.146643  5198 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:24.150743  5182 raft_consensus.cc:740] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:24.160120  5183 log.cc:927] Time spent T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
W20260709 13:07:24.175606 29871 leader_election.cc:336] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:24.178689 29139 raft_consensus.cc:2468] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 28.
W20260709 13:07:24.188113  5189 log.cc:927] Time spent T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:07:24.211199 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 28 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:24.178366  5115 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:24.183171  5108 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:24.224094  5197 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:07:24.224861  5182 consensus_queue.cc:260] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.226150  5182 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:24.228530 29903 raft_consensus.cc:1240] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 27. Current term is 28. Ops: []
I20260709 13:07:24.179095  5198 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.231493  5198 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:24.233741  5151 raft_consensus.cc:3055] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:07:24.234328  5151 raft_consensus.cc:740] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:24.242151  5151 consensus_queue.cc:260] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:24.239130  5193 log.cc:927] Time spent T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:07:24.249778 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 469302us
I20260709 13:07:24.253566 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:24.259899 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:24.237109  4855 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:24.261052 29038 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:24.262392 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:24.269856  5118 raft_consensus.cc:3055] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:07:24.263797  5188 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:24.272480  5118 raft_consensus.cc:740] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:24.273411  5118 consensus_queue.cc:260] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.273900  5125 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:24.274650  5118 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:24.263881 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:24.276335  5125 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:07:24.276917 29141 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 26: Already voted for candidate a936878f59c840a6bd176a1d2d7347b1 in this term.
W20260709 13:07:24.261571 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 393370us
I20260709 13:07:24.181041  5187 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.280579 29217 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:07:24.282151 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.269126 29188 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:24.277559  4855 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.243906  5144 consensus_queue.cc:1059] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.010s
I20260709 13:07:24.286696  5144 raft_consensus.cc:3055] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:07:24.287155  5144 raft_consensus.cc:740] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:24.282956 29220 raft_consensus.cc:2368] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 27. Current term is 28.
I20260709 13:07:24.287560 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.288131  5144 consensus_queue.cc:260] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 27.22, Last appended by leader: 22, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.288105  4830 raft_consensus.cc:2804] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:07:24.289412  4830 raft_consensus.cc:697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:24.259130  5151 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:07:24.260131  5010 consensus_queue.cc:1175] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7e78ffbcc98242068a874a5d9e341477" responder_term: 26 status { last_received { term: 26 index: 15 } last_committed_idx: 15 last_received_current_leader { term: 26 index: 15 } }
I20260709 13:07:24.290297  4830 consensus_queue.cc:237] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.293285 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:24.294054 29218 raft_consensus.cc:2393] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 26: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:24.263355 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:24.296620  5198 consensus_queue.cc:1059] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:24.249784  5022 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:07:24.269718  5093 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:24.297725  4377 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.297793  5093 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.298380  4377 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.295593 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.300668 29217 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:24.301532  4377 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:24.301764 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.302721 29220 raft_consensus.cc:2368] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24.
I20260709 13:07:24.304625 29220 raft_consensus.cc:1240] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 13:07:24.307116 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.308947 29220 raft_consensus.cc:2410] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:07:24.278167 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:24.311988 29141 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:24.314702 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.315338 29218 raft_consensus.cc:1240] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:07:24.319077 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.317171 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.309454  5093 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:24.329061 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:24.297210  4855 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:24.346056 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.353897 29216 raft_consensus.cc:2368] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
I20260709 13:07:24.297345  5198 consensus_queue.cc:1243] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 25.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:07:24.355795 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:24.363906 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.367163 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 53933.5us
I20260709 13:07:24.356995 29218 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 24.
I20260709 13:07:24.369138 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:24.369674 29141 raft_consensus.cc:2468] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 24.
I20260709 13:07:24.374926 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:24.308431  5105 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.366308 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.382793 29911 raft_consensus.cc:1275] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:07:24.386031 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.389142 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:24.389825 29904 raft_consensus.cc:2468] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:24.394838 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:24.355667  5144 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:24.403245 29037 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:24.405536 29139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:24.406208 29139 raft_consensus.cc:2468] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 23.
I20260709 13:07:24.401875 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:24.416711 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:24.412797 29188 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:24.412966 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:24.425792 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:24.408020 29185 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:24.428396 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:24.388463  4565 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:24.430758 29216 raft_consensus.cc:2393] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 24: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:24.408560  4635 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:24.435388 29218 raft_consensus.cc:1275] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:07:24.425343  5093 raft_consensus.cc:2804] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:24.420117  5198 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:24.449663  5198 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.437471  5022 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:24.444954  5093 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.454783  5198 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:24.452548  5125 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.433387  5108 raft_consensus.cc:2804] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:24.434111  4855 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:24.443627  5120 raft_consensus.cc:2804] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:07:24.474058 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:24.464977  5093 raft_consensus.cc:3060] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:24.438648  4635 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.464644  5188 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:24.437935  4565 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.469785  5108 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:24.501205 29216 raft_consensus.cc:1240] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Rejecting Update request from peer 730186a8c70f43f2a57d80163bbc37d2 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 13:07:24.471948  5125 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.502568 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:24.511524  4796 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:24.497085  5120 raft_consensus.cc:697] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:24.517423 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:24.505214  5108 raft_consensus.cc:3060] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:24.518157 29147 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:24.519297 29039 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:24.521865 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 83596.9us
I20260709 13:07:24.502457  5188 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.531198 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:24.532379 29214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42964 (request call id 3437) took 1024 ms. Trace:
I20260709 13:07:24.532917 29214 rpcz_store.cc:276] 0709 13:07:23.507574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:23.507703 (+   129us) service_pool.cc:224] Handling call
0709 13:07:23.514697 (+  6994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:07:23.704796 (+190099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:07:24.532156 (+827360us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54001338d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:07:24.532321 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435616128}
I20260709 13:07:24.527622  5120 consensus_queue.cc:237] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.526048  5198 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:24.550990 29147 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:24.552837  5205 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.553870  4565 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:24.553694  5205 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.563777 29219 raft_consensus.cc:2468] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 27.
I20260709 13:07:24.526286  5178 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.566865  5105 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:24.569202  4635 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:24.582589 29113 leader_election.cc:400] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
I20260709 13:07:24.583155 29113 leader_election.cc:403] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:07:24.573570  5188 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:24.576282 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:24.576282 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:24.586835 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:24.572719 29215 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:07:24.565110  5125 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:24.592739 29113 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:24.598701 29113 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:24.602713 29113 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:24.605909 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:24.576033  5178 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.593611  5192 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 25. Current term is 26.
I20260709 13:07:24.613615  5151 consensus_queue.cc:1048] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:07:24.625766 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.638525 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.604377  5010 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:24.609822  5012 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:24.607527  5134 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:24.664742  5208 consensus_queue.cc:1048] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 13:07:24.668545  5205 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:24.671151  5178 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:24.650938  5188 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.672695  5188 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.675220 29145 raft_consensus.cc:1275] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:07:24.643246  5136 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.607784  5058 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:24.678465 29039 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:24.687616 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:24.687976 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:24.687291 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:24.689711 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:24.658736  5010 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:24.695837 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:24.696902 29144 raft_consensus.cc:2410] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 28 because replica has last-logged OpId of term: 27 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 21.
W20260709 13:07:24.698714 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 101891us
I20260709 13:07:24.703647 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:24.684386  5188 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:24.704541 29146 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:24.688055  5136 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.692378  5182 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:24.730679  5182 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:24.732043  5182 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.698424  5010 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:24.751808 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:24.757054 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:24.757359  5188 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:24.758126  5125 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:24.759392 29039 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
W20260709 13:07:24.761677 28956 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 79cdfc6a725145348b70034422ef477b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7e78ffbcc98242068a874a5d9e341477 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:24.774281 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:24.787829 29111 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:24.789196 29111 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:24.803165 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:24.766419  5188 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:24.822140  5188 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:24.822750  5188 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:07:24.864785 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 256470us
W20260709 13:07:24.860325  5187 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.007s	sys 0.021s
I20260709 13:07:24.868129 29113 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:24.863384  5115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.007s	sys 0.009s
I20260709 13:07:24.864115  4377 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.863054  5192 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
W20260709 13:07:24.885597  5115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.010s	sys 0.013s
I20260709 13:07:24.893874  5093 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.892346  4888 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:24.904253  5134 raft_consensus.cc:2804] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 13:07:24.881312  5187 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.007s	sys 0.021s
I20260709 13:07:24.864372  5108 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.910581  5093 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:24.915521  5177 raft_consensus.cc:3055] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:07:24.916196 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
W20260709 13:07:24.926657  5137 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.009s	sys 0.000s
I20260709 13:07:24.895378  4377 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:24.924042  5110 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.008s	sys 0.000s
I20260709 13:07:24.919059  5177 raft_consensus.cc:740] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:24.948873  5177 consensus_queue.cc:260] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.951875  5177 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:07:24.942171  5110 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.009s	sys 0.000s
I20260709 13:07:24.910912  5192 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.955003  5192 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:24.934684  5137 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.010s	sys 0.000s
I20260709 13:07:24.933053  5187 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:24.957180  5178 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:24.985291  5125 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:24.943742  5188 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.987846  5012 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:24.989902  5178 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:24.988579  5010 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:24.988821  5187 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.000317  5134 raft_consensus.cc:697] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:25.014389 29147 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:07:25.011886  5108 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:25.022979  4377 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:25.024267  5178 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:25.025738 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:25.026692 29910 raft_consensus.cc:2410] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 28 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 13:07:25.019596  5182 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.037122  5182 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.038105  5010 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:25.039512  5134 consensus_queue.cc:237] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.047080  5188 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:25.048251 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.050284  5136 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:25.052076 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.052438 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:25.052775 29215 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 20.
I20260709 13:07:25.053117 29221 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:25.055518 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:25.061672 29185 leader_election.cc:336] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:25.063872 29185 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.063988 29221 raft_consensus.cc:2468] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:25.064555 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:25.070684 29113 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:25.074610 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:25.075271 29215 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:25.075531 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.076103 29221 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:25.082144 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.087155 29216 raft_consensus.cc:1240] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 26. Current term is 27. Ops: [26.13-26.13]
I20260709 13:07:25.090050 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:25.090687 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:25.084228  5182 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:25.080514  5181 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:25.100198 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:25.112136 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.081144  5058 raft_consensus.cc:2804] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:07:25.144670 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 66795.5us
W20260709 13:07:25.147655 29185 leader_election.cc:336] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:25.151880 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.152499 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.153754  5188 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.153736  5223 raft_consensus.cc:493] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:25.156843  5188 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.157716  5223 raft_consensus.cc:515] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:25.159336  5188 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:25.160266 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.164428 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.171681 29215 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:25.163552  5058 raft_consensus.cc:697] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:25.174217 29038 leader_election.cc:400] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 27. Current term is 28.
I20260709 13:07:25.174640 29038 leader_election.cc:403] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:25.160032  4888 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:25.165069  5223 leader_election.cc:290] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:25.176370 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:25.178535 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:25.179373 29218 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:25.172374  4377 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:25.181346  4377 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:25.175319 29185 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.195537 29185 leader_election.cc:336] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:25.197434 29185 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.198437 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:25.175356  5188 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:25.206840  5188 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:25.210345  5187 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:25.211263  4377 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:25.183967  5093 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:25.199189  5109 consensus_queue.cc:1048] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 13:07:25.208539  5110 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:25.183706  4888 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:25.210141 29112 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:25.178206  5125 raft_consensus.cc:3060] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Advancing to term 28
I20260709 13:07:25.211627  5058 consensus_queue.cc:237] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.221043 29141 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:07:25.232240 29185 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.233985 29185 consensus_peers.cc:597] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.235167  5223 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:25.237356 29185 leader_election.cc:336] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.238196 29141 raft_consensus.cc:1275] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
W20260709 13:07:25.239428 29185 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.240149 29185 consensus_peers.cc:597] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.240715 29185 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.240288 28963 catalog_manager.cc:5697] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 12 to 19, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 19 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:25.241667 29185 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.242134 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:25.243741 29185 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.234210  4888 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:25.245121 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:25.246605 29185 consensus_peers.cc:597] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.247249 29185 leader_election.cc:336] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.247848 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:25.248800 29141 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 28.
I20260709 13:07:25.251055 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:25.252679 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:25.247859 29185 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:25.254788 29911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:25.255599 29185 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.255903  5022 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:25.256585  5181 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:07:25.257090 29185 leader_election.cc:336] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.257638 29185 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:25.258855 29185 leader_election.cc:336] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.259300 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:25.260562 29185 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.261175 29185 leader_election.cc:336] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.261605 29185 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:25.262825 29185 leader_election.cc:336] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.230265  4377 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:25.263971 29185 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.264952 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:25.265293 29185 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.175889  5108 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 13:07:25.215384  5188 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:25.268085 29185 consensus_peers.cc:597] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.264338 28963 catalog_manager.cc:5697] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 15 to 20, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 20 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:25.268786 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:25.273864 29904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5acfd220cbb54cd181c08050de276f65" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:25.265548  5022 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:25.273622 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:25.263361  5104 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:25.265586  5221 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:07:25.284462  5226 log.cc:927] Time spent T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:07:25.288142 28963 catalog_manager.cc:5697] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 17 to 19, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 19 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:25.289242  5214 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.315s	user 0.002s	sys 0.004s
I20260709 13:07:25.290138 28963 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:25.291579 28963 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:25.283949 29142 raft_consensus.cc:2393] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 28: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
W20260709 13:07:25.284339  5225 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:07:25.299557  5228 log.cc:927] Time spent T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:07:25.330191  5022 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:07:25.330458 29185 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.330871  5022 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:25.331727 29186 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb329a9e56064f1480f8f42e09ff6679. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.333091  5022 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:25.333400 29112 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:25.334714  5144 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:25.269675  4377 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:25.340297  4377 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:25.312266 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c54509bf16d642138312814fefa3f6dc" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:25.312911  5125 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 27. Current term is 28.
I20260709 13:07:25.322028 29218 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 26.
I20260709 13:07:25.282861 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:25.342114  4635 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:25.341815 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:25.346258 28963 catalog_manager.cc:5697] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 21 to 22, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 22 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:25.348384 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 167669us
I20260709 13:07:25.349718 29038 leader_election.cc:400] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24.
I20260709 13:07:25.349843 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:25.350185 29038 leader_election.cc:403] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:25.351109  4377 raft_consensus.cc:3060] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:07:25.353863 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:25.348786 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:25.390419 29039 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:25.391511 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:25.392112 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:25.391571 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:25.392619 29145 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:25.390954 28963 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 19 to 20, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 20 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:25.395758 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:25.396044 29142 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:07:25.397779 29141 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:07:25.400233 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:25.393525  5188 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:25.390187  5108 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:25.411397 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:25.408496 29038 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:25.408582 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:25.412920 28963 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:25.394731  4804 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:07:25.418740 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:25.401803  5093 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:25.420015 29145 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 26.
I20260709 13:07:25.421708 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:25.418952 28963 catalog_manager.cc:5697] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 13 to 17, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 17 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:25.426332 29038 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:25.440402 28963 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 19 to 23, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 23 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:25.420063  5196 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:25.435096 29142 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:25.409958  5178 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:25.448305 29112 leader_election.cc:304] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:25.442641 28963 catalog_manager.cc:5697] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 15 to 22, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 22 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:25.460461 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.462028 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.463001 29215 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 20.
I20260709 13:07:25.418196  4804 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.466722 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.467932 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:25.448881  5080 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:25.470077 29039 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:25.406281  4888 consensus_queue.cc:1059] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:07:25.480436 29214 raft_consensus.cc:1240] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Rejecting Update request from peer a936878f59c840a6bd176a1d2d7347b1 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 13:07:25.435984  5188 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:07:25.476353 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:25.475689 29871 leader_election.cc:336] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:25.488178 29187 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.488468 29871 consensus_peers.cc:597] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.488582 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.491546  4377 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for earlier term 23. Current term is 24.
I20260709 13:07:25.446425  5223 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:25.516621  5223 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.519340  5223 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:25.464569  5151 raft_consensus.cc:2749] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:07:25.522467 29185 consensus_peers.cc:597] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.525660 29185 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.529287 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 59297.9us
I20260709 13:07:25.530066 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:25.531250 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:25.531399  5229 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:07:25.532086  5177 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:25.493494 29213 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 13:07:25.489318 29871 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.533151 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 43881.2us
W20260709 13:07:25.533742 29871 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.533856 29222 raft_consensus.cc:1275] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 13:07:25.534327 29871 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.534844 29871 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.535182 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:25.535950  5218 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:07:25.493295  4888 consensus_queue.cc:1243] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:07:25.537254 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.479086  4804 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:07:25.541368  5151 consensus_queue.cc:1048] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:07:25.499169 28963 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 12 to 16, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 16 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:25.545954 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 118500us
I20260709 13:07:25.546419 28963 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:25.549510 29145 raft_consensus.cc:1275] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:07:25.485172  5125 consensus_queue.cc:1059] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 13:07:25.550669  5125 raft_consensus.cc:3055] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:07:25.551054  5125 raft_consensus.cc:740] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:25.551983  5125 consensus_queue.cc:260] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.553608  5125 raft_consensus.cc:3060] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:25.532265  5229 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:07:25.561900 29871 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.570847 29039 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:25.564750 29871 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.584328 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:25.581380 29185 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.585320 29185 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:25.586169 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:25.586889  4377 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:25.585999 28963 catalog_manager.cc:5697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 10 to 17, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 17 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:25.587316 29038 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:25.593326 29185 consensus_peers.cc:597] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.594614 29871 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.594185 29185 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:25.597549 29185 leader_election.cc:336] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.598748 29185 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:25.562696  5178 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
W20260709 13:07:25.596930 29871 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.605240 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:25.581019  5151 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.591878  4888 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:25.629658  5151 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:25.630208  4888 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:25.588398  5093 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 13:07:25.633276 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:25.634380  4377 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:25.608951  5178 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:25.637427  5178 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:25.612407  5241 log.cc:927] Time spent T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 13:07:25.608057 29871 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.640307 28963 catalog_manager.cc:5697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 13 to 15, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 15 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:25.613785  4804 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:25.596630  4635 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:07:25.596390  5108 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:25.588601  5182 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:25.602371 29185 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.646711 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.603822  5181 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:25.656572 29213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.658979 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:25.660910 29213 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 27.
I20260709 13:07:25.642069  5151 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:25.668123 29113 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:25.600026  5188 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
I20260709 13:07:25.647504 28963 catalog_manager.cc:5697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:25.657766  5144 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:25.664718  5125 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.681658  5125 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.682348 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:25.681062  4804 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:25.682962 29145 raft_consensus.cc:3060] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:07:25.687793  5144 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.690316 29145 raft_consensus.cc:2468] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 29.
I20260709 13:07:25.691529 29875 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
W20260709 13:07:25.694051 29111 leader_election.cc:336] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:25.697793 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 95459.9us
W20260709 13:07:25.698812 29185 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.699254 29185 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:25.702598 29185 leader_election.cc:336] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:25.709147 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 219056us
I20260709 13:07:25.710000 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:25.706095 29111 leader_election.cc:336] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.702757  4804 raft_consensus.cc:2804] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:07:25.701144  5187 raft_consensus.cc:3055] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:07:25.716813  5187 raft_consensus.cc:740] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:25.716918  5115 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:07:25.718051  5187 consensus_queue.cc:260] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.720088  4804 raft_consensus.cc:697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 29 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:25.721365  4804 consensus_queue.cc:237] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:25.725881  5104 consensus_queue.cc:1048] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:07:25.714498 29111 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:25.671979  5080 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 26 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:25.709374  4635 raft_consensus.cc:2749] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:07:25.675724  5151 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 13:07:25.713999 29185 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.723704  5181 consensus_queue.cc:1048] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:25.733165 28963 catalog_manager.cc:5697] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 13 to 15, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 15 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:25.706483  5022 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 13:07:25.737787 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 149195us
I20260709 13:07:25.700139  5125 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:25.745430 29039 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:25.745630 29111 consensus_peers.cc:597] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.752310 28963 catalog_manager.cc:5697] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:25.752913 29039 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:25.748502 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 105489us
W20260709 13:07:25.740095 29185 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.765550 28963 catalog_manager.cc:5697] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 16 to 20, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 20 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:07:25.772933 29036 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:07:25.767243 29185 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.775990 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:25.738539  5151 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.789335 28963 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:25.750216  5205 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:25.757596  5188 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:25.772913  5108 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:25.788932  5151 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:25.793952  5125 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:25.821610 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 54415.8us
W20260709 13:07:25.824328 29185 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.786671 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:25.825135 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:25.803236  5178 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.833110 29039 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:25.796847  5243 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:25.803529  5124 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:25.807989  5108 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:25.808190  5239 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:25.850968 28963 catalog_manager.cc:5697] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 17 to 22, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 22 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:25.848300  5196 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:25.853260  5108 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:25.823489  5177 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.855360 28963 catalog_manager.cc:5697] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 17 to 22, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 22 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:25.860059 29185 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:25.860853 29185 consensus_peers.cc:597] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.827834  5242 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:25.863792 29185 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.865212  5242 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.867009 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:25.866159 29185 consensus_peers.cc:597] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:25.871673 29037 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:25.870203 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 81788.1us
I20260709 13:07:25.831770  5125 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.807205  5080 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.858518  5144 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:25.863237  5104 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:25.849642  5178 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:25.857272  5124 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:25.908512  4635 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.884863  5188 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:25.911365  5125 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:25.864627  5177 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.885984  5182 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:25.850519  5243 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:25.956110 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:25.953150 29216 raft_consensus.cc:1275] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:07:25.959821 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:25.965430 29111 leader_election.cc:336] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:25.977914 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:25.946656  5196 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.020495  5249 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:26.021150  5249 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.021588  5243 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:26.022302  5178 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:25.947665  4635 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:25.947418  5188 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:26.027439 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.036801 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.038960 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.045725  5229 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:26.024188  5248 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.048345  5248 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.049814  4635 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:26.050485 29219 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 26.
I20260709 13:07:26.053637 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:26.056612  5124 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.057898  5178 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:26.058123  5058 consensus_queue.cc:1048] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:07:26.060659  5245 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:07:26.061902  5187 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:26.050568  5248 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:26.063560 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.064999 29113 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:26.073498 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.078474 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:26.078765 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:26.086344 29037 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:26.092442 29222 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:26.074396 29140 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
W20260709 13:07:26.105060 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 65239.7us
I20260709 13:07:26.109843 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.110579 29140 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:26.111495 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:26.116802 29185 leader_election.cc:336] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.117491 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:26.018599  5196 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:26.109982  5257 log.cc:927] Time spent T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:07:26.103834  5248 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:26.109423  4888 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.115408 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:26.116793 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:26.142752 29037 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:26.117151  5243 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:26.159718  4377 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.160310  4377 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.156024  4888 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:26.147279  5229 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.168278 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.153121  4965 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:26.171015 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 29 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:26.172310 29142 raft_consensus.cc:2393] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 20: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:26.173797 29906 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 28.
I20260709 13:07:26.157891 29140 raft_consensus.cc:2955] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Committing config change with OpId 24.14: config changed from index -1 to 14, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:07:26.159372  5182 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:26.184444  5177 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.163118  5080 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.200947  5080 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.155637  5178 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:26.218478  4377 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:26.225317 29875 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:26.228443  5188 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.229064  5188 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:26.160077  5252 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:26.237210 29112 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:26.226065  5178 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.275002  4888 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.281536  5080 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:26.287763  5265 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 13:07:26.288452 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 52451.4us
I20260709 13:07:26.291350  5270 ts_tablet_manager.cc:933] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.238992  5252 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.229564  5188 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.271282  5258 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:07:26.208017  5229 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:26.311652  5252 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:26.316006  5266 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:26.316676  5266 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:26.303208  5265 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.301291  5178 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:26.326898  5265 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:07:26.360464  5252 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:26.360471  5265 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.366775  5265 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:26.303316  5151 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.378600  5151 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 election: Requested vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:26.379606 29145 raft_consensus.cc:2368] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 27. Current term is 28.
I20260709 13:07:26.403326 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 29 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:26.404016 29910 raft_consensus.cc:3060] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:07:26.404330 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:26.405761 29906 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Advancing to term 28
I20260709 13:07:26.408303  5266 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.407822  5221 raft_consensus.cc:2955] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Committing config change with OpId 24.14: config changed from index -1 to 14, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:07:26.409121  5187 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:26.409945  5187 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.412649  5188 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:26.419533  5108 consensus_meta.cc:287] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: Time spent flushing consensus metadata: real 0.555s	user 0.005s	sys 0.005s
I20260709 13:07:26.424845  5080 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.427178  5080 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:26.429255  5080 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.363808  5252 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:26.350405  5196 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.425915  5188 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.432385  5188 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:26.433512 29906 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 28.
I20260709 13:07:26.434618  5188 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.435635 29112 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:26.445520  5229 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.450049  5151 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:07:26.450663  5151 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:26.451845  5151 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.435034 29139 raft_consensus.cc:1275] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 13:07:26.458437 29910 raft_consensus.cc:2468] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 29.
I20260709 13:07:26.452642  5187 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:26.459498  5251 consensus_queue.cc:1048] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.002s
I20260709 13:07:26.465782 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.467577 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:26.476992 28956 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 79cdfc6a725145348b70034422ef477b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7e78ffbcc98242068a874a5d9e341477 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:26.477072  5178 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:26.467790  5108 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.480747 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.482077 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.482827 29146 raft_consensus.cc:2410] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 for term 27 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 13:07:26.487950  5258 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:26.491169  5108 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:26.493269  5242 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.480870 29147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.493813 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.499554 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.500217 29142 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 26.
I20260709 13:07:26.500829 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:26.501549 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.502089 29218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.502177 29214 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:26.505211 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.505357 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.507472 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:26.508111 29111 consensus_peers.cc:597] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:26.508644 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:26.509099 29142 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:26.509136 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.509888 29221 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 27: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:26.510768 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.511945 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:26.516691 29112 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:26.518074 29215 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:07:26.520521 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:26.521329 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.521632 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:26.522187 29875 leader_election.cc:400] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 27. Current term is 28.
I20260709 13:07:26.560374 29875 leader_election.cc:403] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:07:26.521899 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.561398 29140 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:26.560123 29217 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 27.
I20260709 13:07:26.561383 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.549412 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:26.563776 29220 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:26.540818 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:26.480088  5178 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.565887 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:26.523547 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:26.520700  5108 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:26.574239 29902 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:26.575448 29111 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.577332 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:26.577554 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 54096.3us
I20260709 13:07:26.517928  5248 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:26.579875 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:26.580590 29215 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:26.567323  5109 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337) with higher term. Message: Invalid argument: T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 27. Current term is 28.
I20260709 13:07:26.584997  4888 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:07:26.585474  4888 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:26.585906  4888 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:26.559772  5205 raft_consensus.cc:2804] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 13:07:26.594623 29871 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:26.587173  5108 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:26.596144 29871 leader_election.cc:336] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:26.608880 29871 leader_election.cc:336] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:26.610374 29871 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 149 similar messages]
I20260709 13:07:26.480420  5270 tablet_copy_client.cc:323] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.131:45851
I20260709 13:07:26.560338  5196 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:26.613826 29140 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:26.617986 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.618409 29215 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 28.
I20260709 13:07:26.623091 29214 raft_consensus.cc:2393] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 27: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:26.629026 29902 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
I20260709 13:07:26.614332  5248 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:26.615015  5205 raft_consensus.cc:697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 29 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:26.641309 29142 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:26.643340 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:26.642418 29039 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:26.654924 29038 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:26.656531  5187 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:26.657948 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:26.658939  5187 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:26.660641  5243 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:26.662166 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:26.663147 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:26.663650 29220 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:26.663987 29214 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:26.665916 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:26.622432  5196 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.635890  5249 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.666471 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.647800  5242 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:26.671757 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.674475 29217 raft_consensus.cc:2368] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 27. Current term is 28.
I20260709 13:07:26.673789  5125 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:07:26.673890  5277 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:26.669036  5178 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:26.679792 29232 tablet_copy_service.cc:140] P 6ad9faa1c14444b29c9db8e71f46e2d6: Received BeginTabletCopySession request for tablet cb329a9e56064f1480f8f42e09ff6679 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:51820)
I20260709 13:07:26.680488 29232 tablet_copy_service.cc:161] P 6ad9faa1c14444b29c9db8e71f46e2d6: Beginning new tablet copy session on tablet cb329a9e56064f1480f8f42e09ff6679 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:51820: session id = 7e78ffbcc98242068a874a5d9e341477-cb329a9e56064f1480f8f42e09ff6679
I20260709 13:07:26.661230  5243 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:26.681740  5243 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:26.688669  5279 log.cc:927] Time spent T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:07:26.661242  5108 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:26.684209  5205 consensus_queue.cc:237] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 27.22, Last appended by leader: 22, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.696543 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.674789 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:26.716859 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:26.723949 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:26.724745 29874 leader_election.cc:400] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 for earlier term 27. Current term is 28.
W20260709 13:07:26.726016 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 104609us
I20260709 13:07:26.732136 29214 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 21.
W20260709 13:07:26.733229 29185 leader_election.cc:336] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.730790  5277 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:26.725929  5108 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:26.668242  5221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:55430 (request call id 2) took 2083 ms. Trace:
I20260709 13:07:26.729518  5238 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:07:26.767624 29038 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:26.751686  5178 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:07:26.777664 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 109300us
I20260709 13:07:26.772084  5108 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:26.803658  5108 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:26.827951 29185 leader_election.cc:336] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.828778 29185 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:26.837219  4782 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:07:26.822497  5196 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:26.854831 29111 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.855281 29111 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:26.847719  5281 log.cc:927] Time spent T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.192s	user 0.003s	sys 0.000s
W20260709 13:07:26.860324 29111 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:26.862923 29111 consensus_peers.cc:597] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:26.860832  5282 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:26.757560  5221 rpcz_store.cc:276] 0709 13:07:24.569799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.570627 (+   828us) service_pool.cc:224] Handling call
0709 13:07:24.666824 (+ 96197us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4c0006a340. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7834dd9a20 00007f7834deaa29 00007f7834db0644 00007f7834dd7f2d 00007f7834dd7b93 00007f7836a661af 00007f7836a65f3e 00007f7836aeb888 00007f7836b0875c 00007f7836b0976a 00007f7836b0891b
0709 13:07:24.693742 (+ 26918us) log.cc:844] Serialized 506 byte log entry
0709 13:07:25.288328 (+594586us) spinlock_profiling.cc:243] Waited 9.48 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f78366a886e 00007f78366a7bb6 00007f78366a77d8 00007f78366b1221 00007f78366b11d9 00007f78366b1169
0709 13:07:26.652737 (+1364409us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8641,"raft.run_cpu_time_us":2278,"raft.run_wall_time_us":2812,"spinlock_wait_cycles":51800576,"thread_start_us":194546,"threads_started":1,"wal-append.queue_time_us":246542,"wal-append.run_cpu_time_us":15451,"wal-append.run_wall_time_us":1348956}
I20260709 13:07:26.836587  5248 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:26.874948  4888 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:26.877774  4888 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 28 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:26.878755  5278 log.cc:927] Time spent T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.225s	user 0.006s	sys 0.000s
I20260709 13:07:26.874157  5108 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.880407  5242 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:26.881274  5242 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:26.883239  5242 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:26.884555  5282 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:07:26.887420 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 265840us
W20260709 13:07:26.899446 29871 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.901155 29037 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:26.903956 29037 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:26.908412 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:26.910379 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:26.910636 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:26.915479 29871 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:26.915644  5178 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:26.915357  5242 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:26.922828 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:26.923458 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.923460 29140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:26.924371 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:26.924988 29142 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:26.927248 29037 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:26.931558 29148 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:07:26.911404  5282 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:26.938274  5282 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:26.936496  5277 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:26.942390  5280 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.258s	user 0.000s	sys 0.002s
I20260709 13:07:26.934253  5242 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:07:26.947989 27934 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 29098)
I20260709 13:07:26.944787 29232 tablet_copy_source_session.cc:215] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:26.954636  5282 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.961920 29142 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 28.
W20260709 13:07:26.948833 29111 consensus_peers.cc:597] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:26.937851 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:26.964569  5242 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:26.964650  5248 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:26.968123  5242 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:26.971941 29039 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:26.979454  5178 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:07:26.980065  5178 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 28 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
W20260709 13:07:26.980386  5284 log.cc:927] Time spent T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:07:26.980994  5178 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:26.983758 29111 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.986783 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:26.987592 29217 raft_consensus.cc:2393] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 24: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:26.989310 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:26.990134 29220 raft_consensus.cc:2368] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 26. Current term is 27.
W20260709 13:07:26.991360 29111 leader_election.cc:336] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:26.996626 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:26.997203 29214 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:27.004689  5270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb329a9e56064f1480f8f42e09ff6679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:27.010408 29214 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 28.
I20260709 13:07:27.023515 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:27.027024 29113 leader_election.cc:400] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 26. Current term is 27.
I20260709 13:07:27.027349 29113 leader_election.cc:403] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 13:07:27.030097  5144 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:27.048043  5151 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:27.022645 29214 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
W20260709 13:07:27.071466 29186 consensus_peers.cc:597] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041): Couldn't send request to peer 7e78ffbcc98242068a874a5d9e341477. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0c33ea0ff53432c8be5e364242b23b3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:27.071768 29142 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:07:27.074703  5181 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:27.075498  5108 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.080463  5108 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.075654  5277 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.082758  5108 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.083455  5277 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.086256  5277 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.118040  5144 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:27.124694 29217 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:07:27.127540  5247 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:27.129818 29910 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:07:27.144192 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.145148 29146 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:27.145262 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.154693  5277 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:27.155107 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 115528us
I20260709 13:07:27.144065  5247 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
W20260709 13:07:27.157908 29111 consensus_peers.cc:597] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:27.156904 29039 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:27.164294 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.169359 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.169863 29220 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 24.
I20260709 13:07:27.177497 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:27.155273  5277 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.178762  5080 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:27.168401  5178 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:27.180861 29910 raft_consensus.cc:1275] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 27 index: 22. Preceding OpId from leader: term: 29 index: 23. (index mismatch)
I20260709 13:07:27.173009  5151 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851) with higher term. Message: Invalid argument: T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 for earlier term 26. Current term is 27.
I20260709 13:07:27.190279  5277 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.182029  5178 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:27.191105  5178 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:27.191148 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.185432  5196 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.183111  5270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:54878 (request call id 1371) took 1154 ms. Trace:
I20260709 13:07:27.196679 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.202363 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:27.194666  5196 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.185189  5247 consensus_queue.cc:1048] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.015s
I20260709 13:07:27.203007  5277 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:07:27.211695 29185 consensus_peers.cc:597] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:27.214381  5299 log.cc:927] Time spent T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
I20260709 13:07:27.234499  5196 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.238500 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.239048 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.246256  5196 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.246616 29217 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:07:27.246930  5196 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.260088 29038 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:27.261191  5196 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.261868  5242 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:27.263309  5242 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:07:27.266383 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.266830 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.195729  5270 rpcz_store.cc:276] 0709 13:07:26.031478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:26.031639 (+   161us) service_pool.cc:224] Handling call
0709 13:07:26.312946 (+281307us) ts_tablet_manager.cc:934] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
0709 13:07:27.003269 (+690323us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:07:27.109413 (+106144us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:27.109554 (+   141us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:07:27.127878 (+ 18324us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:07:27.172054 (+ 44176us) tablet_replica.cc:206] Creating consensus instance
0709 13:07:27.179386 (+  7332us) ts_tablet_manager.cc:989] Replica registered.
0709 13:07:27.179468 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":56735,"tablet-copy.queue_time_us":227961,"thread_start_us":104579,"threads_started":2}
I20260709 13:07:27.272590  5270 tablet_copy_client.cc:806] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:27.273548  5270 tablet_copy_client.cc:670] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:27.277633 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:27.279778  5242 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:07:27.276979 29111 consensus_peers.cc:597] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:27.281406  5300 log.cc:927] Time spent T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:07:27.286651 29111 consensus_peers.cc:597] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:27.281713  5270 tablet_copy_client.cc:538] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:27.302793 29146 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:07:27.304255  5242 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:07:27.307888  5181 raft_consensus.cc:2955] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:07:27.313009 29146 raft_consensus.cc:2955] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:07:27.281170  5178 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.338047  5178 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.338989 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:27.339568 29217 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:27.341586 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:27.342236 29142 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:27.348582 29217 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:27.349910 29038 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1; no voters: 
I20260709 13:07:27.351099 29142 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a936878f59c840a6bd176a1d2d7347b1 in term 25.
I20260709 13:07:27.365782  5178 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:27.376766  5178 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Becoming Leader. State: Replica: a936878f59c840a6bd176a1d2d7347b1, State: Running, Role: LEADER
I20260709 13:07:27.392648  5178 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:27.398617 29111 consensus_peers.cc:597] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:27.399284 29146 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:07:27.383841  5270 tablet_bootstrap.cc:492] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
I20260709 13:07:27.442716 29220 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 28 index: 14. (term mismatch)
I20260709 13:07:27.443140 29220 pending_rounds.cc:85] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 13
I20260709 13:07:27.443358 29220 pending_rounds.cc:107] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 25.14
I20260709 13:07:27.443586 29220 raft_consensus.cc:2889] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:27.419143  5277 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:07:27.466606  5108 raft_consensus.cc:493] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.458559  5242 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.458367  4888 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:27.488611  5178 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.456090  5196 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:07:27.505293  5196 consensus_queue.cc:1243] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:07:27.520661  5309 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.528281  5309 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:27.530988  5309 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:27.532156 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:27.536131  5277 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.536746  5277 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.543022  5277 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.478575  5108 raft_consensus.cc:515] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.549467  5108 leader_election.cc:290] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.496335  4888 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:27.496659  5242 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:27.553922  5242 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:27.555500 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:27.571723  5108 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.572431  5108 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.574797  5108 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:27.578977 29111 consensus_peers.cc:597] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:27.583953  4888 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:27.598846 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:27.618505 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a57ab99b58a448de88023e7166f78ce2" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.619099  4888 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:27.619803  4888 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.620690 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.622252  4888 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.623588 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.629428 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.630021 29145 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 13:07:27.630317 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.633965 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.635234 29039 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:27.639045  5242 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:07:27.639878  5242 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:07:27.644357  5315 ts_tablet_manager.cc:933] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477: Initiating tablet copy from peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.507025  5178 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.655661 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:27.673120 29910 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:07:27.677484  5306 log.cc:927] Time spent T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: Append to log took a long time: real 0.164s	user 0.000s	sys 0.017s
W20260709 13:07:27.677302  5305 log.cc:927] Time spent T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.170s	user 0.004s	sys 0.010s
W20260709 13:07:27.693339 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 200095us
I20260709 13:07:27.671181  5277 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:27.697080 29222 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:07:27.758705 29220 pending_rounds.cc:85] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 12
I20260709 13:07:27.681545  5080 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:27.766026 29220 pending_rounds.cc:107] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 13:07:27.766719 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7517b9e93904801886467474f000b2d" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.767761 29222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082c896dd04e451198eba8392e70fba7" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.769183 29220 raft_consensus.cc:2889] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:27.775290 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:27.766676  5285 log.cc:927] Time spent T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: Append to log took a long time: real 0.314s	user 0.008s	sys 0.004s
W20260709 13:07:27.776711 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 83505.4us
I20260709 13:07:27.777432 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.778218 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:27.781978 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 87630.1us
I20260709 13:07:27.793480  5196 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.794047  5196 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.806617 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.809455 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.781095  5307 raft_consensus.cc:493] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.783133  5277 raft_consensus.cc:493] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.774700  5080 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:27.780838  4888 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:07:27.843600  5080 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:27.847759 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:27.825711  5277 raft_consensus.cc:515] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.863018  5277 leader_election.cc:290] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.830212  5178 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.867997 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.877660  5178 raft_consensus.cc:493] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:27.878373  5178 raft_consensus.cc:515] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:27.880784  5196 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:27.881644  5178 leader_election.cc:290] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:27.845801  5308 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.888019 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:27.825527  5307 raft_consensus.cc:515] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.897651 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:27.916368  5315 tablet_copy_client.cc:323] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Beginning tablet copy session from remote peer at address 127.27.71.131:45851
I20260709 13:07:27.949369 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44d88fbf78394409bcacfadb40f377d6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:27.950080 27934 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 29098)
I20260709 13:07:27.951354 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.953030 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a52106dbee45398d36a633f281eef6" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.951370 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.959293 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b27f1b9ed9964227ac908c2897d935e9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:27.964356  5178 raft_consensus.cc:493] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:27.916927  5308 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:27.980365 29232 tablet_copy_service.cc:140] P 6ad9faa1c14444b29c9db8e71f46e2d6: Received BeginTabletCopySession request for tablet c0c33ea0ff53432c8be5e364242b23b3 from peer 7e78ffbcc98242068a874a5d9e341477 ({username='slave'} at 127.0.0.1:51820)
I20260709 13:07:27.981173 29232 tablet_copy_service.cc:161] P 6ad9faa1c14444b29c9db8e71f46e2d6: Beginning new tablet copy session on tablet c0c33ea0ff53432c8be5e364242b23b3 from peer 7e78ffbcc98242068a874a5d9e341477 at {username='slave'} at 127.0.0.1:51820: session id = 7e78ffbcc98242068a874a5d9e341477-c0c33ea0ff53432c8be5e364242b23b3
I20260709 13:07:27.993136  5316 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:07:28.006167 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 68736us
I20260709 13:07:27.964977  5178 raft_consensus.cc:515] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:28.009443  5178 leader_election.cc:290] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:28.010092  5307 leader_election.cc:290] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:28.016572  5308 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:28.017007 28962 catalog_manager.cc:5697] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 17 to 19, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 19 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:28.020709 29039 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:28.022862  5277 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:28.019227 28962 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 19 to 21, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 21 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:28.032068 28962 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 20 to 21, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 21 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:28.039908 28962 catalog_manager.cc:5697] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 19 to 21, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 21 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:28.041205 28962 catalog_manager.cc:5697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:28.044118 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 86323.6us
I20260709 13:07:28.048206 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:28.034626 29232 tablet_copy_source_session.cc:215] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:28.067373 28962 catalog_manager.cc:5697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:28.070521 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 88709.2us
I20260709 13:07:28.071504 29038 leader_election.cc:304] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:28.073697 28963 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 267) took 37434 ms (client timeout 15000 ms). Trace:
I20260709 13:07:28.083037 29038 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.088567 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.091889  5309 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:28.078428  5080 raft_consensus.cc:2749] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:28.112182 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:28.074290 28963 rpcz_store.cc:269] 0709 13:06:50.634577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:03.358795 (+12724218us) service_pool.cc:224] Handling call
0709 13:07:03.614528 (+255733us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b54001c0610. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e90b2 00007f783c962a89 00007f783c8fada6 00007f783c8cc088 00007f783c9ca37e 00007f783ae8b2a4 00007f783ae8b232 00007f783ae8b161 00007f783ae8b0dc
0709 13:07:28.069256 (+24454728us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:26.589757 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:26.590332 (+   575us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:26.590364 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:07:26.590384 (+    20us) tablet.cc:662] Decoding operations
0709 13:07:26.655147 (+ 64763us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:26.655208 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:26.655232 (+    24us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:07:26.656893 (+  1661us) tablet.cc:701] Row locks acquired
0709 13:07:26.656915 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:07:26.657070 (+   155us) write_op.cc:270] Start()
0709 13:07:26.657187 (+   117us) write_op.cc:276] Timestamp: P: 1783602446657021 usec, L: 0
0709 13:07:26.657209 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:07:26.657948 (+   739us) log.cc:844] Serialized 8636 byte log entry
0709 13:07:26.893055 (+235107us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f7833a030ac 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b
0709 13:07:27.206533 (+313478us) op_driver.cc:464] REPLICATION: finished
0709 13:07:27.284734 (+ 78201us) write_op.cc:301] APPLY: starting
0709 13:07:27.284833 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:27.313167 (+ 28334us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:27.313192 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:27.389135 (+ 75943us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:27.418097 (+ 28962us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:07:27.418155 (+    58us) write_op.cc:312] APPLY: finished
0709 13:07:27.418664 (+   509us) log.cc:844] Serialized 196 byte log entry
0709 13:07:27.418844 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:27.419096 (+   252us) write_op.cc:454] Released schema lock
0709 13:07:27.420565 (+  1469us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:27.423167 (+  2602us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:27.423213 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":50,"spinlock_wait_cycles":28426624,"child_traces":[["op",{"apply.queue_time_us":77912,"apply.run_cpu_time_us":31119,"apply.run_wall_time_us":136470,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96087,"num_ops":22,"prepare.queue_time_us":422029,"prepare.run_cpu_time_us":23385,"prepare.run_wall_time_us":393873,"raft.queue_time_us":171077,"raft.run_cpu_time_us":54200,"raft.run_wall_time_us":82056,"replication_time_us":543294,"spinlock_wait_cycles":37173632,"thread_start_us":862701,"threads_started":4,"wal-append.queue_time_us":361994}]]}
I20260709 13:07:28.116410 29144 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:07:28.109014  5307 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:28.114194 28962 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:28.127490 28962 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:28.135936  5315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c33ea0ff53432c8be5e364242b23b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:28.141431 29038 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.141782  5331 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:28.142554  5178 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:07:28.143004 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.144075  5309 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:28.147040 29038 leader_election.cc:304] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.150084 29038 leader_election.cc:304] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:28.150731  5308 raft_consensus.cc:2749] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:28.152053  5178 raft_consensus.cc:2749] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:28.151078  5331 raft_consensus.cc:493] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:28.152393 29038 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.156416  5331 raft_consensus.cc:515] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:28.149179 28962 catalog_manager.cc:5697] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 15 to 17, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 17 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:28.174634  5331 leader_election.cc:290] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:28.175189  5308 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:28.176222 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.176617 28962 catalog_manager.cc:5697] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:28.174964 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.184332 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.183121 28962 catalog_manager.cc:5697] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 17 to 18, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 18 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:28.193063 28962 catalog_manager.cc:5697] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:28.199760 29214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "348ae3050d97486fa402a5da2bbdce46" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:28.201225 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee1e380ba6084a0e9d9978fa9983edc1" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:28.206940 29217 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Refusing update from remote peer a936878f59c840a6bd176a1d2d7347b1: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:07:28.211839 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "984bea8952114e3f9c0a045e7f557e37" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:28.213436 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:28.213995 28956 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 79cdfc6a725145348b70034422ef477b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 7e78ffbcc98242068a874a5d9e341477 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:28.204272  5331 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:07:28.197746  5307 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.231098 29037 leader_election.cc:304] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:28.238380 29038 leader_election.cc:304] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:28.227731  5307 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:28.250519 29038 leader_election.cc:304] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.261773 29038 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.242772  5080 raft_consensus.cc:2749] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:07:28.253234  5331 raft_consensus.cc:2749] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:28.264384  5196 raft_consensus.cc:2749] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:28.272250  5309 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:07:28.272802  5108 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:28.288878  5309 consensus_queue.cc:1243] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [LEADER]: Peer 6ad9faa1c14444b29c9db8e71f46e2d6 log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:07:28.296880 29216 pending_rounds.cc:85] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting all ops after (but not including) 13
I20260709 13:07:28.297241 29216 pending_rounds.cc:107] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 13:07:28.296995  5308 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.297556 29216 raft_consensus.cc:2889] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:07:28.297823  5308 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:28.301017  5308 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:28.300933 29215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:28.302672 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:28.307098  5315 tablet_copy_client.cc:806] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:28.319625 29037 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:28.321929 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:28.310038  5315 tablet_copy_client.cc:670] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:28.325685  5308 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:28.325958  5307 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:28.373152 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.381291 29039 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:28.378533  5311 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.385404  5311 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:28.387547  5311 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:28.390604  5270 tablet_bootstrap.cc:492] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=1 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:28.409162  5270 tablet_bootstrap.cc:492] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
W20260709 13:07:28.399904 29871 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:28.395625  4888 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:07:28.422446  5270 ts_tablet_manager.cc:1403] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 1.039s	user 0.065s	sys 0.019s
W20260709 13:07:28.425562 29185 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:28.435539 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.437304 29145 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 27.
I20260709 13:07:28.435142  5315 tablet_copy_client.cc:538] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:28.460328  5270 raft_consensus.cc:359] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [term 24 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:07:28.480394 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:28.484804  5311 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:07:28.485191  5311 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:28.485311 29185 leader_election.cc:336] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:28.485595  5311 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:28.481703  5323 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.482805  4888 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.487805  5323 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:28.488174  4888 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:28.486577  5270 raft_consensus.cc:740] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [term 24 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:07:28.490521  4888 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:28.493084 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:28.499593 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.504940 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:28.505504 29871 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:28.507732 29185 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:28.509332  5242 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:28.491657  5270 consensus_queue.cc:260] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:07:28.524817 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.537117  5270 ts_tablet_manager.cc:1434] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.114s	user 0.008s	sys 0.002s
I20260709 13:07:28.538357  5323 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:28.540218 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:28.544332 29232 tablet_copy_service.cc:342] P 6ad9faa1c14444b29c9db8e71f46e2d6: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-cb329a9e56064f1480f8f42e09ff6679 received from {username='slave'} at 127.0.0.1:51820
I20260709 13:07:28.544831 29232 tablet_copy_service.cc:434] P 6ad9faa1c14444b29c9db8e71f46e2d6: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-cb329a9e56064f1480f8f42e09ff6679 on tablet cb329a9e56064f1480f8f42e09ff6679 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:28.548249 29038 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.552279  4888 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:28.574364  5311 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:28.584807 29871 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:28.588012  5315 tablet_bootstrap.cc:492] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap starting.
W20260709 13:07:28.590600 29185 consensus_peers.cc:597] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:28.591320 29185 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:28.603868 29185 leader_election.cc:336] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:28.606060 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:28.606845 29145 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:28.615425 29145 raft_consensus.cc:2468] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 28.
I20260709 13:07:28.616778 29188 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:28.617789  5310 raft_consensus.cc:2804] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:07:28.618387  5310 raft_consensus.cc:697] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:28.619303  5310 consensus_queue.cc:237] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:28.621742  5242 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:28.622455  5242 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:28.607473  5311 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:28.635665 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:28.636528 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1cace028c94f79adfce2be604fb35a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.635845  5242 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:28.642037 29039 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.655696  5242 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:28.670936 29185 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:28.685240 29111 consensus_peers.cc:597] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:28.758728  5323 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:28.759518  5323 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:28.761942  5323 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:28.768117 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:28.770298 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.776692 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:28.780189  5323 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:28.785687 29185 consensus_peers.cc:597] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:28.806841 28962 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 168) took 27870 ms (client timeout 15000 ms). Trace:
W20260709 13:07:28.813045 29185 consensus_peers.cc:597] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:28.819981  5323 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.820573  5323 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:28.822887  5323 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:28.825860 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.828107 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12e21412fcd24f95a8e3635fc5bbe138" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:28.829924 29038 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.830875  5242 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:28.831429  5317 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.832091  5317 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:28.834383  5317 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:28.836973 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.839571 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:28.842157 29038 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.843636  5323 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:28.846637  5323 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.848733  5323 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:28.872813 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:28.873030  5323 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:28.876132 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:28.807472 28962 rpcz_store.cc:269] 0709 13:07:00.936407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:11.858286 (+10921879us) service_pool.cc:224] Handling call
0709 13:07:28.806774 (+16948488us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:28.458215 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:28.458788 (+   573us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:28.458816 (+    28us) write_op.cc:435] Acquired schema lock
0709 13:07:28.458833 (+    17us) tablet.cc:662] Decoding operations
0709 13:07:28.459904 (+  1071us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:28.459978 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:28.460004 (+    26us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:07:28.482269 (+ 22265us) tablet.cc:701] Row locks acquired
0709 13:07:28.482290 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:07:28.482439 (+   149us) write_op.cc:270] Start()
0709 13:07:28.482555 (+   116us) write_op.cc:276] Timestamp: P: 1783602448482392 usec, L: 0
0709 13:07:28.482585 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:07:28.483250 (+   665us) log.cc:844] Serialized 4813 byte log entry
0709 13:07:28.632733 (+149483us) op_driver.cc:464] REPLICATION: finished
0709 13:07:28.703760 (+ 71027us) write_op.cc:301] APPLY: starting
0709 13:07:28.703859 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:28.712567 (+  8708us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:28.712690 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:28.733450 (+ 20760us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:28.733733 (+   283us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:07:28.733786 (+    53us) write_op.cc:312] APPLY: finished
0709 13:07:28.734102 (+   316us) log.cc:844] Serialized 113 byte log entry
0709 13:07:28.748088 (+ 13986us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:28.748234 (+   146us) write_op.cc:454] Released schema lock
0709 13:07:28.748287 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:28.748697 (+   410us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:28.748726 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":235566,"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":70539,"apply.run_cpu_time_us":16577,"apply.run_wall_time_us":44887,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54287,"num_ops":12,"prepare.queue_time_us":85870,"prepare.run_cpu_time_us":18454,"prepare.run_wall_time_us":51363,"raft.queue_time_us":36533,"raft.run_cpu_time_us":51466,"raft.run_wall_time_us":69889,"replication_time_us":149980,"spinlock_wait_cycles":534784,"thread_start_us":114309,"threads_started":4,"wal-append.queue_time_us":65862}]]}
W20260709 13:07:28.889657 29871 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:28.890188 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:28.891542  5323 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:28.890508 28965 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 21 to 24, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 24 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:28.901685 29185 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:28.896595 28965 catalog_manager.cc:5697] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 20 to 22, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 22 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:28.909034 28965 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:28.916451 28965 catalog_manager.cc:5697] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 17 to 19, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 19 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:28.943363 28965 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:28.945696 28965 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:28.951078 27934 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 29098)
I20260709 13:07:28.977003 28965 catalog_manager.cc:5697] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:28.981987 28965 catalog_manager.cc:5697] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:28.984086 28965 catalog_manager.cc:5697] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 10 to 25. New cstate: current_term: 25 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:28.985334 28965 catalog_manager.cc:5697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 20 to 24, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 24 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:28.985097  5242 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:28.987039  5242 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:28.987566 28965 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 19 to 20, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 20 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:28.989464  5242 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:28.989328 28965 catalog_manager.cc:5697] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 15 to 17, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 17 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:28.990934 28965 catalog_manager.cc:5697] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 15 to 17, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 17 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:28.994220 28965 catalog_manager.cc:5697] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:28.995771 28965 catalog_manager.cc:5697] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 22 to 24, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 24 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:28.997135 28965 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 18 to 24, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 24 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:28.998719 28965 catalog_manager.cc:5697] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:29.001921 28965 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 23 to 24, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 24 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:29.003641 28965 catalog_manager.cc:5697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 18 to 20, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 20 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:29.005184 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:29.006084 28965 catalog_manager.cc:5697] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 22 to 23, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 23 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:29.010249 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfda5dcdd94083afe1620564f9d376" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:29.019179 29038 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:29.017710 28965 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 16 to 17, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 17 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:29.085721 29038 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 57944.5us
I20260709 13:07:29.085997  5357 raft_consensus.cc:493] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.086582  5357 raft_consensus.cc:515] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:29.089709  5357 leader_election.cc:290] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:29.090368  5357 raft_consensus.cc:493] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:29.008849  5317 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:29.106650 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:29.030478  4888 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:29.108994 29145 raft_consensus.cc:2393] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a936878f59c840a6bd176a1d2d7347b1 in current term 28: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
W20260709 13:07:29.109818 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 98334.4us
I20260709 13:07:29.109414  5317 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:29.141148  4888 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.156962 29910 raft_consensus.cc:1217] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [term 24 LEARNER]: Deduplicated request from leader. Original: 24.13->[24.14-24.14]   Dedup: 24.14->[]
I20260709 13:07:29.091030  5357 raft_consensus.cc:515] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:29.175290 29111 consensus_peers.cc:597] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:29.196488 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9334a7c40c744cbbff4dec0358f721b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:29.201992 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 44893.3us
I20260709 13:07:29.209431 29038 leader_election.cc:304] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
W20260709 13:07:29.220302 29185 consensus_peers.cc:597] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:29.230677  5356 raft_consensus.cc:3060] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:29.234371  5357 leader_election.cc:290] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:29.247185  5363 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:29.247809  5363 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:29.170010  4888 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:29.277213  5363 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:29.328950 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 126112us
I20260709 13:07:29.331051 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:29.341579 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:29.344332 29871 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.345180 29871 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.326737 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 126336us
I20260709 13:07:29.348984 29145 raft_consensus.cc:1275] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:07:29.329561  5363 raft_consensus.cc:493] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.350795  5363 raft_consensus.cc:515] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:29.352990  5363 leader_election.cc:290] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:29.345853 29871 consensus_peers.cc:597] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:29.324106  5317 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:29.379887 29902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:29.370447  4888 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:29.372644  5364 consensus_queue.cc:1048] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
W20260709 13:07:29.424073 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 177379us
W20260709 13:07:29.434146 29871 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.442651 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 71581.6us
W20260709 13:07:29.494036 29111 consensus_peers.cc:597] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:29.494521 29111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 78786us
W20260709 13:07:29.508286 29185 consensus_peers.cc:597] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:29.510926 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:29.510972 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cabbc6d2424a53a76857f415831c36" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:29.513737 29145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:29.532939  5315 tablet_bootstrap.cc:492] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:29.534224  5315 tablet_bootstrap.cc:492] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477: Bootstrap complete.
I20260709 13:07:29.535254  5315 ts_tablet_manager.cc:1403] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477: Time spent bootstrapping tablet: real 0.998s	user 0.060s	sys 0.045s
I20260709 13:07:29.505924  5356 raft_consensus.cc:2749] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:29.539340 29039 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:29.546489 29185 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.556427 29871 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.571226 29111 consensus_peers.cc:597] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:29.558221  5323 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:07:29.577142 29871 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.619091 29185 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.619984 29185 consensus_peers.cc:597] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.620865 29185 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:29.622545  5315 raft_consensus.cc:359] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:07:29.624213  5315 raft_consensus.cc:740] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Initialized, Role: LEARNER
I20260709 13:07:29.629197  5315 consensus_queue.cc:260] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } }
I20260709 13:07:29.631276  5377 raft_consensus.cc:1064] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:07:29.639859  5315 ts_tablet_manager.cc:1434] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477: Time spent starting tablet: real 0.101s	user 0.004s	sys 0.003s
I20260709 13:07:29.642659  5377 consensus_queue.cc:237] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 12, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:29.631979 29185 consensus_peers.cc:597] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.655232 29185 consensus_peers.cc:597] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:29.669127 29232 tablet_copy_service.cc:342] P 6ad9faa1c14444b29c9db8e71f46e2d6: Request end of tablet copy session 7e78ffbcc98242068a874a5d9e341477-c0c33ea0ff53432c8be5e364242b23b3 received from {username='slave'} at 127.0.0.1:51820
I20260709 13:07:29.669632 29232 tablet_copy_service.cc:434] P 6ad9faa1c14444b29c9db8e71f46e2d6: ending tablet copy session 7e78ffbcc98242068a874a5d9e341477-c0c33ea0ff53432c8be5e364242b23b3 on tablet c0c33ea0ff53432c8be5e364242b23b3 with peer 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:29.669492 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5d7d7e7e55b46f89b835614737ee91b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:29.671362 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "023c90d48e094012bc60b1e600fffa14" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:29.672546 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b55b583a63b64ca6832b73f813985e49" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:29.679174 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
W20260709 13:07:29.703423 29185 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:29.706616 29111 consensus_peers.cc:597] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:29.714887 29910 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [term 24 LEARNER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:07:29.715431 29146 raft_consensus.cc:1275] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:07:29.716450  5378 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:29.717389  5311 consensus_queue.cc:1048] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:07:29.724169 29185 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:29.752278  5384 log.cc:927] Time spent T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:07:29.757683 29038 leader_election.cc:304] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:29.758950  5357 raft_consensus.cc:2749] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:29.766116 29038 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:29.767331  5357 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:29.768862 29038 leader_election.cc:304] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:29.771056  5357 raft_consensus.cc:2749] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:29.772565 29038 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:29.773345  5357 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:29.838261  5311 raft_consensus.cc:2955] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Committing config change with OpId 24.15: config changed from index 14 to 15, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:29.847118 28962 catalog_manager.cc:5697] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 17 to 22, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 22 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:29.847478 29910 raft_consensus.cc:2955] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Committing config change with OpId 24.15: config changed from index 14 to 15, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:29.856887 28965 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 278) took 18705 ms (client timeout 15000 ms). Trace:
I20260709 13:07:29.852938 28962 catalog_manager.cc:5697] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 22 to 24, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 24 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:29.864451  5357 raft_consensus.cc:493] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:29.864954 29146 raft_consensus.cc:2955] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Committing config change with OpId 24.15: config changed from index 14 to 15, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:29.865957  5357 raft_consensus.cc:515] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:29.868173  5357 leader_election.cc:290] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:29.871840 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:29.866959 28962 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 19, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 19 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.874236 28962 catalog_manager.cc:5697] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
W20260709 13:07:29.875434 28965 rpcz_store.cc:269] 0709 13:07:11.151202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:21.124702 (+9973500us) service_pool.cc:224] Handling call
0709 13:07:29.856779 (+8732077us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:29.357597 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:29.358155 (+   558us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:29.358189 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:07:29.358207 (+    18us) tablet.cc:662] Decoding operations
0709 13:07:29.374673 (+ 16466us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:29.376695 (+  2022us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:29.379684 (+  2989us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:07:29.452749 (+ 73065us) tablet.cc:701] Row locks acquired
0709 13:07:29.452772 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:07:29.478495 (+ 25723us) write_op.cc:270] Start()
0709 13:07:29.488950 (+ 10455us) write_op.cc:276] Timestamp: P: 1783602449477577 usec, L: 0
0709 13:07:29.493445 (+  4495us) op_driver.cc:348] REPLICATION: starting
0709 13:07:29.529555 (+ 36110us) log.cc:844] Serialized 8320 byte log entry
0709 13:07:29.608827 (+ 79272us) op_driver.cc:464] REPLICATION: finished
0709 13:07:29.611396 (+  2569us) write_op.cc:301] APPLY: starting
0709 13:07:29.614644 (+  3248us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:29.675955 (+ 61311us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:29.676053 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:29.755414 (+ 79361us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:29.755701 (+   287us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:07:29.755745 (+    44us) write_op.cc:312] APPLY: finished
0709 13:07:29.756151 (+   406us) log.cc:844] Serialized 188 byte log entry
0709 13:07:29.756351 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:29.756548 (+   197us) write_op.cc:454] Released schema lock
0709 13:07:29.756609 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:29.772560 (+ 15951us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:29.772597 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":2280,"apply.run_cpu_time_us":26066,"apply.run_wall_time_us":161124,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91907,"num_ops":21,"prepare.queue_time_us":148655,"prepare.run_cpu_time_us":7020,"prepare.run_wall_time_us":172295,"replication_time_us":115154,"thread_start_us":121559,"threads_started":2}]]}
I20260709 13:07:29.879601 28962 catalog_manager.cc:5697] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 22 to 24, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 24 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.881683 28962 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 23 to 27, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 27 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.893666 28962 catalog_manager.cc:5697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 18 to 21, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 21 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.895169 28962 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 21 to 23, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 23 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.896767 28962 catalog_manager.cc:5697] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 19 to 26. New cstate: current_term: 26 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.898315 28962 catalog_manager.cc:5697] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 22 to 25. New cstate: current_term: 25 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:29.899653 28962 catalog_manager.cc:5697] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 14 to 22, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 22 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.901137 28962 catalog_manager.cc:5697] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 17 to 23, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 23 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:29.903175 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c24ba590bc46427dbac6cd4a4da67ee8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:29.914455 29038 leader_election.cc:304] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:29.915338  5357 raft_consensus.cc:2749] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:07:29.902367 28962 catalog_manager.cc:5697] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 21 to 24, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 24 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:29.925275 28962 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 20 to 23. New cstate: current_term: 23 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.927170 28962 catalog_manager.cc:5697] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 18 to 21, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 21 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:29.930639 28962 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 17 to 19, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 19 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.932358 28962 catalog_manager.cc:5697] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 17 to 24, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 24 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:29.933959 28962 catalog_manager.cc:5697] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 22, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 22 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:29.951589 27934 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 29098)
I20260709 13:07:29.935559 28962 catalog_manager.cc:5697] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 18 to 20, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 20 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:29.981022 28962 catalog_manager.cc:5697] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.983711 28962 catalog_manager.cc:5697] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.985594 28962 catalog_manager.cc:5697] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 reported cstate change: term changed from 24 to 26, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 26 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:29.986434 29910 raft_consensus.cc:1217] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.13->[20.14-20.14]   Dedup: 20.14->[]
I20260709 13:07:30.039076  5357 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.039736  5357 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:30.043476 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:30.047290  5357 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:30.049263 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b0625d42ef413aae4ec112d9ba450b" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:30.077070 29038 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:30.077994  5357 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:30.116185  5356 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.116786  5356 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:30.119278  5356 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:30.121503 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:30.133471 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:30.136919 29039 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:30.138144  5356 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:30.138525 28963 catalog_manager.cc:5697] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.142155 28963 catalog_manager.cc:5697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.143605 28963 catalog_manager.cc:5697] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 23 to 24, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 24 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.145740 28963 catalog_manager.cc:5697] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 17 to 20, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 20 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:30.144338 29111 consensus_peers.cc:597] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:30.168082 29111 consensus_peers.cc:597] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:30.147354 28963 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.177063 28963 catalog_manager.cc:5697] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 17 to 19, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 19 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.183284 28963 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 20 to 23, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 23 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.184636 28963 catalog_manager.cc:5697] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 13 to 24. New cstate: current_term: 24 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.185813 28963 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 24 to 25, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 25 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.187793 28962 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58570 (request call id 245) took 6664 ms. Trace:
W20260709 13:07:30.188719 29871 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:30.188839 28963 catalog_manager.cc:5697] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 23 to 25, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 25 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.190574 28963 catalog_manager.cc:5697] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.188221 28962 rpcz_store.cc:276] 0709 13:07:23.526832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.892267 (+5365435us) service_pool.cc:224] Handling call
0709 13:07:30.187704 (+1295437us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:29.994324 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:29.994843 (+   519us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:29.994886 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:07:29.994903 (+    17us) tablet.cc:662] Decoding operations
0709 13:07:29.996560 (+  1657us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:29.996619 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:29.996647 (+    28us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:07:29.998269 (+  1622us) tablet.cc:701] Row locks acquired
0709 13:07:29.998288 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:07:29.998430 (+   142us) write_op.cc:270] Start()
0709 13:07:29.998543 (+   113us) write_op.cc:276] Timestamp: P: 1783602449998386 usec, L: 0
0709 13:07:29.998564 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:07:29.999190 (+   626us) log.cc:844] Serialized 8704 byte log entry
0709 13:07:30.002408 (+  3218us) op_driver.cc:464] REPLICATION: finished
0709 13:07:30.002867 (+   459us) write_op.cc:301] APPLY: starting
0709 13:07:30.002946 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:30.071232 (+ 68286us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:30.071341 (+   109us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:30.099580 (+ 28239us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:30.099966 (+   386us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:07:30.100021 (+    55us) write_op.cc:312] APPLY: finished
0709 13:07:30.100484 (+   463us) log.cc:844] Serialized 196 byte log entry
0709 13:07:30.105998 (+  5514us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:30.106238 (+   240us) write_op.cc:454] Released schema lock
0709 13:07:30.106308 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:30.124229 (+ 17921us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:30.124267 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10331,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":242,"apply.run_cpu_time_us":28911,"apply.run_wall_time_us":121323,"cfile_cache_hit":88,"cfile_cache_hit_bytes":91960,"num_ops":22,"prepare.queue_time_us":403,"prepare.run_cpu_time_us":5167,"prepare.run_wall_time_us":7767,"replication_time_us":3704}]]}
I20260709 13:07:30.192703 28963 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 24 to 25, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 25 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.197530 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:30.200152 27934 tablet_replica.cc:333] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.201164  5356 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.201826  5356 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:30.201292 27934 raft_consensus.cc:2243] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.204303 27934 raft_consensus.cc:2272] T d7517b9e93904801886467474f000b2d P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.207362 27934 tablet_replica.cc:333] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.208148  5356 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:30.209182 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:30.209571 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:30.212182 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:30.212500 27934 raft_consensus.cc:2243] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.214057 27934 raft_consensus.cc:2272] T 9a29f89bc1cb471d94fee8d14f56314c P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.212999  5356 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:30.231040 27934 tablet_replica.cc:333] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.234148 27934 raft_consensus.cc:2243] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.235074 27934 raft_consensus.cc:2272] T a57ab99b58a448de88023e7166f78ce2 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.237941 27934 tablet_replica.cc:333] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.248489 27934 raft_consensus.cc:2243] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.249358 27934 raft_consensus.cc:2272] T b9574de46fcd490e84208e41907200b0 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.253015  5357 raft_consensus.cc:493] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:30.253633 27934 tablet_replica.cc:333] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.253574  5357 raft_consensus.cc:515] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:30.254665 27934 raft_consensus.cc:2243] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.255249 27934 raft_consensus.cc:2272] T 023c90d48e094012bc60b1e600fffa14 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.259361 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:30.263528 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e250871b4ad74e079a538cdb04065b00" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:30.272912 27934 tablet_replica.cc:333] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.273936 27934 raft_consensus.cc:2243] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.274787 27934 pending_rounds.cc:70] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Trying to abort 1 pending ops.
I20260709 13:07:30.275092 27934 pending_rounds.cc:77] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1: Aborting op as it isn't in flight: id { term: 22 index: 13 } timestamp: 7305635595744256000 op_type: NO_OP noop_request { }
I20260709 13:07:30.275552 27934 raft_consensus.cc:2889] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:30.275890 27934 raft_consensus.cc:2272] T ee1e380ba6084a0e9d9978fa9983edc1 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.279049 27934 tablet_replica.cc:333] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.279268 29037 leader_election.cc:304] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:30.280241  5356 raft_consensus.cc:2749] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:30.279887 27934 raft_consensus.cc:2243] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.281591 27934 raft_consensus.cc:2272] T b55b583a63b64ca6832b73f813985e49 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.262718  5357 leader_election.cc:290] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:30.295992 27934 tablet_replica.cc:333] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.297619 27934 raft_consensus.cc:2243] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.298501 27934 raft_consensus.cc:2272] T 984bea8952114e3f9c0a045e7f557e37 P a936878f59c840a6bd176a1d2d7347b1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.301661 27934 tablet_replica.cc:333] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.302582 27934 raft_consensus.cc:2243] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.303634 27934 raft_consensus.cc:2272] T ea48540c1af24bf3b55edec5c52ec133 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:30.310076 29111 consensus_peers.cc:597] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:30.326536 27934 tablet_replica.cc:333] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.327709 27934 raft_consensus.cc:2243] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.328609 27934 raft_consensus.cc:2272] T d5d7d7e7e55b46f89b835614737ee91b P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.341360 27934 tablet_replica.cc:333] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.342433 27934 raft_consensus.cc:2243] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.336560  5357 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:30.357168  5357 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:30.366531 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:30.371986 29111 consensus_peers.cc:597] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:30.376276  5356 raft_consensus.cc:493] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.376920  5356 raft_consensus.cc:515] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:30.379172  5356 leader_election.cc:290] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:30.385316 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:30.386493 27934 raft_consensus.cc:2272] T 9e3e05d4f95940f5a4cac560d26d0ec8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.390923 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac412099b9f64d4e8bcceebe1c7ffe87" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:30.392204  5356 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:30.393471 29039 leader_election.cc:304] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:30.396924  5356 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:30.400978 29185 consensus_peers.cc:597] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:30.416216 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e96bceabcbf747e0acec327bc1abc729" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:30.430907 27934 tablet_replica.cc:333] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.432003 27934 raft_consensus.cc:2243] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.420112  5357 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:30.434419  5389 raft_consensus.cc:2749] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:30.447968 27934 raft_consensus.cc:2272] T 3842b36ffd134cd28c657475df48e741 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.439092 29037 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
W20260709 13:07:30.505180 29037 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 66156.4us
I20260709 13:07:30.506480  5390 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:30.480898 29039 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 87532.5us
I20260709 13:07:30.512199 27934 tablet_replica.cc:333] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.450155  5356 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:30.448956 28963 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 169) took 13628 ms (client timeout 15000 ms). Trace:
I20260709 13:07:30.512919 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:30.456624 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:30.482317 28965 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 19 to 25, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 25 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.515578 28965 catalog_manager.cc:5697] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:30.513067 28963 rpcz_store.cc:269] 0709 13:07:16.820823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:28.148997 (+11328174us) service_pool.cc:224] Handling call
0709 13:07:30.448840 (+2299843us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:30.214845 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:30.215447 (+   602us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:30.215481 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:07:30.215498 (+    17us) tablet.cc:662] Decoding operations
0709 13:07:30.216588 (+  1090us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:30.216646 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:30.216673 (+    27us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:07:30.217707 (+  1034us) tablet.cc:701] Row locks acquired
0709 13:07:30.217723 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:07:30.217875 (+   152us) write_op.cc:270] Start()
0709 13:07:30.217987 (+   112us) write_op.cc:276] Timestamp: P: 1783602450217818 usec, L: 0
0709 13:07:30.218016 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:07:30.218665 (+   649us) log.cc:844] Serialized 4746 byte log entry
0709 13:07:30.232163 (+ 13498us) op_driver.cc:464] REPLICATION: finished
0709 13:07:30.234683 (+  2520us) write_op.cc:301] APPLY: starting
0709 13:07:30.234773 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:30.289289 (+ 54516us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:30.289391 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:30.376301 (+ 86910us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:30.376581 (+   280us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:07:30.376630 (+    49us) write_op.cc:312] APPLY: finished
0709 13:07:30.390952 (+ 14322us) log.cc:844] Serialized 113 byte log entry
0709 13:07:30.396904 (+  5952us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:30.397063 (+   159us) write_op.cc:454] Released schema lock
0709 13:07:30.397117 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:30.405457 (+  8340us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:30.405712 (+   255us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3473,"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":2296,"apply.run_cpu_time_us":16674,"apply.run_wall_time_us":168091,"cfile_cache_hit":48,"cfile_cache_hit_bytes":50160,"num_ops":12,"prepare.queue_time_us":106,"prepare.run_cpu_time_us":4179,"prepare.run_wall_time_us":6710,"replication_time_us":13989,"spinlock_wait_cycles":47488}]]}
I20260709 13:07:30.517181 28965 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 24 to 28. New cstate: current_term: 28 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:30.519035 28965 catalog_manager.cc:5697] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 15 to 22, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 22 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.466684 29910 consensus_queue.cc:237] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:30.520654 28965 catalog_manager.cc:5697] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.472440  5311 raft_consensus.cc:1064] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: attempting to promote NON_VOTER 7e78ffbcc98242068a874a5d9e341477 to VOTER
I20260709 13:07:30.522015 28965 catalog_manager.cc:5697] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 19 to 20, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 20 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.523478 28965 catalog_manager.cc:5697] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 26 to 29, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 29 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:30.525373 28965 catalog_manager.cc:5697] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 21 to 23, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 23 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.526852 28965 catalog_manager.cc:5697] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 22 to 27, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 27 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.525753  5311 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:30.513986 27934 raft_consensus.cc:2243] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.532871 27934 pending_rounds.cc:70] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Trying to abort 1 pending ops.
I20260709 13:07:30.536712 27934 pending_rounds.cc:77] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1: Aborting op as it isn't in flight: id { term: 28 index: 16 } timestamp: 7305635589127495680 op_type: NO_OP noop_request { }
I20260709 13:07:30.537189 27934 raft_consensus.cc:2889] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:30.540140 27934 raft_consensus.cc:2272] T c24ba590bc46427dbac6cd4a4da67ee8 P a936878f59c840a6bd176a1d2d7347b1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.540331 28965 catalog_manager.cc:5697] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 22 to 23, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 23 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.542311 28965 catalog_manager.cc:5697] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 24 to 26, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 26 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.543723 28965 catalog_manager.cc:5697] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 24 to 25, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 25 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:30.546890 27934 tablet_replica.cc:333] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.547451 28965 catalog_manager.cc:5697] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 22 to 28, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 28 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.568475 29038 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:30.569433 28965 catalog_manager.cc:5697] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 17 to 19, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 19 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.571681 28965 catalog_manager.cc:5697] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 19 to 25. New cstate: current_term: 25 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.572702 27934 raft_consensus.cc:2243] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:07:30.573606 28955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 91471.8us
I20260709 13:07:30.573324 28965 catalog_manager.cc:5697] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 18 to 24, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 24 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.572206  5356 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.576567  5356 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:30.579866  5356 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:30.579879 28955 catalog_manager.cc:4755] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): ChangeConfig:REMOVE_PEER RPC failed for tablet 79cdfc6a725145348b70034422ef477b: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:30.581195 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:30.583993 28965 catalog_manager.cc:5697] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.585346 28965 catalog_manager.cc:5697] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 16 to 19, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 19 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.586586 28965 catalog_manager.cc:5697] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.587713 28965 catalog_manager.cc:5697] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 25 to 29. New cstate: current_term: 29 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.589465 28965 catalog_manager.cc:5697] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.590600 28965 catalog_manager.cc:5697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 17 to 26, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 26 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.591904 28965 catalog_manager.cc:5697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 20 to 23, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129). New cstate: current_term: 23 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.581435 27934 raft_consensus.cc:2272] T 79cdfc6a725145348b70034422ef477b P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.593530 29144 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:07:30.593605 28965 catalog_manager.cc:5697] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 20 to 21, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 21 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.595346  5393 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:30.595110 28965 catalog_manager.cc:5697] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.595804 27934 tablet_replica.cc:333] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.596891 27934 raft_consensus.cc:2243] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.596446 28965 catalog_manager.cc:5697] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index -1 to 14, term changed from 23 to 24, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131), NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 24 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:07:30.598285 28965 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 19 to 20, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 20 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:30.599457  5356 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:30.606945 27934 raft_consensus.cc:2272] T 7f1cace028c94f79adfce2be604fb35a P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.627663 29220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fae9bf99a434b83a73451cee1c56e6a" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:30.633762 27934 tablet_replica.cc:333] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.635030 27934 raft_consensus.cc:2243] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.648005 27934 pending_rounds.cc:70] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Trying to abort 1 pending ops.
I20260709 13:07:30.648396 27934 pending_rounds.cc:77] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1: Aborting op as it isn't in flight: id { term: 24 index: 15 } timestamp: 7305635581225443328 op_type: NO_OP noop_request { }
I20260709 13:07:30.649008 27934 raft_consensus.cc:2889] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:30.649403 27934 raft_consensus.cc:2272] T e9334a7c40c744cbbff4dec0358f721b P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:30.665230 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 94604us
I20260709 13:07:30.670456 29144 raft_consensus.cc:1275] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:07:30.680919 29038 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:30.684689  5391 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:30.684880 29910 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEARNER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:07:30.697708 29871 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:30.704320 27934 tablet_replica.cc:333] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.703063  5378 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:07:30.746724 27934 raft_consensus.cc:2243] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.756100  5390 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.756747  5390 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:30.759202  5390 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:30.760057 27934 raft_consensus.cc:2272] T b27f1b9ed9964227ac908c2897d935e9 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.761713  5356 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.762223  5356 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:30.763178 27934 tablet_replica.cc:333] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.765919 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:30.767105 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:30.768342 27934 raft_consensus.cc:2243] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.768728 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:30.769321 27934 raft_consensus.cc:2272] T 43a52106dbee45398d36a633f281eef6 P a936878f59c840a6bd176a1d2d7347b1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.770462  5356 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:30.772399 27934 tablet_replica.cc:333] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.773306 27934 raft_consensus.cc:2243] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.773944 27934 raft_consensus.cc:2272] T 082c896dd04e451198eba8392e70fba7 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.776050 29039 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:30.778607  5356 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:30.783432 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:30.788312 27934 tablet_replica.cc:333] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
W20260709 13:07:30.788650 29871 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:30.788893  5356 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:30.789294 27934 raft_consensus.cc:2243] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.789974 27934 raft_consensus.cc:2272] T 12e21412fcd24f95a8e3635fc5bbe138 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.789585  5356 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:30.792404  5356 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:30.792853 27934 tablet_replica.cc:333] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.793593 29144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:30.796458 29038 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:30.797745  5391 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:07:30.732293  5398 consensus_queue.cc:1048] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 13:07:30.793727 27934 raft_consensus.cc:2243] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.804497 27934 raft_consensus.cc:2272] T 88cabbc6d2424a53a76857f415831c36 P a936878f59c840a6bd176a1d2d7347b1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.806756 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:30.807405 27934 tablet_replica.cc:333] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.808354 27934 raft_consensus.cc:2243] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:30.809033 27934 raft_consensus.cc:2272] T 348ae3050d97486fa402a5da2bbdce46 P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:30.809487 29185 consensus_peers.cc:597] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:30.809909 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 107793us
I20260709 13:07:30.810083 29038 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:30.813184  5391 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:30.853729 29185 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:30.870832  5393 raft_consensus.cc:2955] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.873052 29910 raft_consensus.cc:2955] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.879288  5255 raft_consensus.cc:2955] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Committing config change with OpId 21.14: config changed from index 10 to 14, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.886991 29148 raft_consensus.cc:2955] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 10 to 14, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.900945 29148 raft_consensus.cc:2955] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:30.940547 28956 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b9574de46fcd490e84208e41907200b0 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:30.944217 27934 tablet_replica.cc:333] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:30.946931 27934 raft_consensus.cc:2243] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:30.962584 27934 raft_consensus.cc:2272] T 813e480732fd4061837e784d4d768b1f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:30.993969 27934 tablet_replica.cc:333] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
W20260709 13:07:31.033221 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 152511us
I20260709 13:07:31.019735  5390 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.038311 27934 raft_consensus.cc:2243] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.041267 27934 pending_rounds.cc:70] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Trying to abort 1 pending ops.
I20260709 13:07:31.041664 27934 pending_rounds.cc:77] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1: Aborting op as it isn't in flight: id { term: 24 index: 13 } timestamp: 7305635586396848128 op_type: NO_OP noop_request { }
I20260709 13:07:31.042423 27934 raft_consensus.cc:2889] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:31.043147 27934 raft_consensus.cc:2272] T cb329a9e56064f1480f8f42e09ff6679 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.047051 27934 tablet_replica.cc:333] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:31.057437  5390 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:31.092981 27934 raft_consensus.cc:2243] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.094820 27934 pending_rounds.cc:70] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Trying to abort 1 pending ops.
I20260709 13:07:31.095275 27934 pending_rounds.cc:77] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1: Aborting op as it isn't in flight: id { term: 19 index: 11 } timestamp: 7305635576507154432 op_type: NO_OP noop_request { }
I20260709 13:07:31.095824 27934 raft_consensus.cc:2889] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:31.105157 27934 raft_consensus.cc:2272] T e8195256ce88462cab8bb734ddbb65ad P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.116590  5390 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:31.117007 27934 tablet_replica.cc:333] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:31.120112 29146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:31.130192 27934 raft_consensus.cc:2243] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.142812 27934 raft_consensus.cc:2272] T c59eab5922dc4c07a3a86e915e3cab7f P a936878f59c840a6bd176a1d2d7347b1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.146667 28965 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 279) took 4066 ms. Trace:
I20260709 13:07:31.150426 27934 tablet_replica.cc:333] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:31.147245 28965 rpcz_store.cc:276] 0709 13:07:27.080015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:29.904864 (+2824849us) service_pool.cc:224] Handling call
0709 13:07:31.146345 (+1241481us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:30.614455 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:30.615021 (+   566us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:30.615058 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:07:30.615070 (+    12us) tablet.cc:662] Decoding operations
0709 13:07:30.616915 (+  1845us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:30.616967 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:30.616997 (+    30us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:07:30.662437 (+ 45440us) tablet.cc:701] Row locks acquired
0709 13:07:30.662461 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:07:30.662623 (+   162us) write_op.cc:270] Start()
0709 13:07:30.662767 (+   144us) write_op.cc:276] Timestamp: P: 1783602450662574 usec, L: 0
0709 13:07:30.662789 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:07:30.664460 (+  1671us) log.cc:844] Serialized 10658 byte log entry
0709 13:07:30.674201 (+  9741us) op_driver.cc:464] REPLICATION: finished
0709 13:07:30.685015 (+ 10814us) write_op.cc:301] APPLY: starting
0709 13:07:30.685105 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:30.777388 (+ 92283us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:30.777498 (+   110us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:30.815974 (+ 38476us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:30.831143 (+ 15169us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=54,mrs_lookups=0
0709 13:07:30.831215 (+    72us) write_op.cc:312] APPLY: finished
0709 13:07:30.831935 (+   720us) log.cc:844] Serialized 236 byte log entry
0709 13:07:30.832128 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:30.832359 (+   231us) write_op.cc:454] Released schema lock
0709 13:07:30.832429 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:30.832758 (+   329us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:30.832787 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13597,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":10550,"apply.run_cpu_time_us":33299,"apply.run_wall_time_us":147729,"cfile_cache_hit":108,"cfile_cache_hit_bytes":112860,"num_ops":27,"prepare.queue_time_us":122,"prepare.run_cpu_time_us":6943,"prepare.run_wall_time_us":50303,"replication_time_us":11259}]]}
I20260709 13:07:31.151827 27934 raft_consensus.cc:2243] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:07:31.161043 29111 consensus_peers.cc:597] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:31.161885 27934 raft_consensus.cc:2272] T cc31aa85851e41609957a765b2d40774 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.188174 27934 tablet_replica.cc:333] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
W20260709 13:07:31.193303 29871 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:31.193861 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 58832.4us
I20260709 13:07:31.197288  5390 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.198076  5390 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:31.202096  5390 leader_election.cc:290] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:31.206480 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:31.208179 29185 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:31.208722 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48036.4us
I20260709 13:07:31.209002 27934 raft_consensus.cc:2243] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:31.210626 27934 raft_consensus.cc:2272] T cddea8e9cff6495ba269c65b0edf8b6c P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.222474 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:31.233042  5390 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.233836  5390 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:31.240411 29037 leader_election.cc:304] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:31.241456  5389 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:07:31.244933 28970 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 266) took 62425 ms (client timeout 15000 ms). Trace:
W20260709 13:07:31.245607 28970 rpcz_store.cc:269] 0709 13:06:28.819513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:28.847005 (+ 27492us) service_pool.cc:224] Handling call
0709 13:07:31.244656 (+62397651us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121108,"reported_tablets":50,"spinlock_wait_cycles":20480}
I20260709 13:07:31.247180 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:31.323581 28963 catalog_manager.cc:5697] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:31.326393 28963 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:31.374073 28963 catalog_manager.cc:5697] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 21 to 23, leader changed from 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 23 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:31.380832 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 90126.1us
W20260709 13:07:31.383061 29871 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 83262.7us
W20260709 13:07:31.386343 29112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 128447us
I20260709 13:07:31.390280  5151 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.390878  5151 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:31.393409  5151 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:31.399719 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 124115us
I20260709 13:07:31.401396  5356 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:31.402369  5356 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:31.410480 28963 catalog_manager.cc:5697] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 23 to 25, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 25 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:31.426422 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:31.433200 27934 tablet_replica.cc:333] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:31.415513 28963 catalog_manager.cc:5697] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 14 to 15, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 24 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:31.434403 27934 raft_consensus.cc:2243] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.435408 27934 raft_consensus.cc:2272] T e250871b4ad74e079a538cdb04065b00 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.434913 28963 catalog_manager.cc:5697] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 19 to 23, leader changed from 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 23 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:31.437161 28963 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index -1 to 14, NON_VOTER 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) added. New cstate: current_term: 20 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: NON_VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: true } } }
I20260709 13:07:31.441408  5390 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:31.463321 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:31.467665  5356 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:31.505991 29038 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:07:31.506847  5356 raft_consensus.cc:493] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.507543  5356 raft_consensus.cc:515] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:31.511055 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 261371us
I20260709 13:07:31.512387 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:31.517253 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:31.507505 29038 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:31.522756  5356 leader_election.cc:290] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:31.525915 29219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c02e27bf25948329604215b11034f44" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:31.471699 27934 tablet_replica.cc:333] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:31.529119 27934 raft_consensus.cc:2243] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.531829  5391 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:31.518124 29216 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 26.
I20260709 13:07:31.534449  5414 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.535079  5414 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:31.540499  5414 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:31.549055 27934 raft_consensus.cc:2272] T 77b0625d42ef413aae4ec112d9ba450b P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.549824  5414 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.550577  5414 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:31.553604  5414 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:31.553766 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:31.554538 29142 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
I20260709 13:07:31.564530 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:31.567888 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c777774468e477ca1ab0352dff56c9c" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:31.573905 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:31.585800  5412 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.586699  5255 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.586611  5412 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:31.587532  5255 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:31.590288 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:31.578122 27934 tablet_replica.cc:333] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:31.592820 27934 raft_consensus.cc:2243] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.593286  5378 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:31.595572  5255 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:31.596055  5378 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.596508  5378 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:31.596666 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:31.597383 29216 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 23.
I20260709 13:07:31.597723 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:31.598387 29903 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 23.
I20260709 13:07:31.598479 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:31.599491  5416 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:07:31.599606  5391 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:31.599994  5416 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.600497  5416 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:31.600292  5391 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:31.601118  5412 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:31.599828 29186 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:31.602357  5412 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:07:31.593374 29871 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:31.602766  5412 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.603175  5412 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:07:31.586730  5410 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:31.594699 29039 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:31.605526  5390 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:31.600591  5414 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.604120  5410 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.607434  5410 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:07:31.594908 29185 consensus_peers.cc:597] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:31.607472  5414 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:31.610827  5414 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:07:31.602773  5391 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:31.612437 29185 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 318 similar messages]
I20260709 13:07:31.616611 27934 raft_consensus.cc:2272] T 1fdfda5dcdd94083afe1620564f9d376 P a936878f59c840a6bd176a1d2d7347b1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.617746 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:31.630920 29142 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
I20260709 13:07:31.635476 27934 tablet_replica.cc:333] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:31.655031 27934 raft_consensus.cc:2243] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.636236 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:31.625131 29038 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:31.637701 29039 leader_election.cc:304] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:31.640956 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:31.641238  5412 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:31.661991  5391 raft_consensus.cc:2749] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:31.652352 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:31.662873  5412 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:31.663513  5412 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:31.663875  5391 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:31.665411 29903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477"
I20260709 13:07:31.665990 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:31.663884  5412 raft_consensus.cc:493] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.667982  5412 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:07:31.653790 29111 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.627151  5416 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:31.671195 29903 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 24: Already voted for candidate 7e78ffbcc98242068a874a5d9e341477 in this term.
I20260709 13:07:31.677071 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:31.677930 29216 raft_consensus.cc:2393] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e78ffbcc98242068a874a5d9e341477 in current term 24: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:31.629686  5378 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:31.685570 27934 raft_consensus.cc:2272] T b09437a3ae8d46379ccdbb1829b14f27 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.700695  5378 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:31.647433  5410 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:31.686019  5416 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:31.690399  5412 raft_consensus.cc:515] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:31.721402 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:31.722271 29142 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in current term 27: Already voted for candidate 730186a8c70f43f2a57d80163bbc37d2 in this term.
I20260709 13:07:31.725095  5412 leader_election.cc:290] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:31.728235 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:31.729152 29216 raft_consensus.cc:2393] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 730186a8c70f43f2a57d80163bbc37d2 in current term 27: Already voted for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in this term.
I20260709 13:07:31.730062 27934 tablet_replica.cc:333] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:31.731160 27934 raft_consensus.cc:2243] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.734067  5414 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.734889  5414 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:31.740401  5414 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:31.741546 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:31.728842  5410 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:31.782213 28965 catalog_manager.cc:5697] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 10 to 14, VOTER 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131) evicted. New cstate: current_term: 21 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:31.784490 28965 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:31.786295 28965 catalog_manager.cc:5697] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:31.789670  5410 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.797863 29142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a29f89bc1cb471d94fee8d14f56314c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:31.798564 29142 raft_consensus.cc:3060] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:31.798243  5410 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:31.802403 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:31.803015 29221 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
I20260709 13:07:31.804797  5410 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:31.792012 27934 raft_consensus.cc:2272] T 2c02e27bf25948329604215b11034f44 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.805830 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:31.807477 29142 raft_consensus.cc:2468] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 27.
I20260709 13:07:31.809607 27934 tablet_replica.cc:333] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:31.810920 27934 raft_consensus.cc:2243] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.829970 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:31.849047 29148 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 21.
I20260709 13:07:31.850359 27934 pending_rounds.cc:70] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Trying to abort 2 pending ops.
I20260709 13:07:31.850967 27934 pending_rounds.cc:77] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Aborting op as it isn't in flight: id { term: 19 index: 13 } timestamp: 7305635590360580096 op_type: NO_OP noop_request { }
I20260709 13:07:31.851440 27934 raft_consensus.cc:2889] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:31.851814 27934 pending_rounds.cc:77] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: Aborting op as it isn't in flight: id { term: 18 index: 12 } timestamp: 7305635553587679232 op_type: NO_OP noop_request { }
I20260709 13:07:31.852255 27934 raft_consensus.cc:2889] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:31.852662 27934 raft_consensus.cc:2272] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.842630 28965 catalog_manager.cc:5697] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:31.817310 28963 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 170) took 1074 ms. Trace:
I20260709 13:07:31.889410 29188 leader_election.cc:304] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:07:31.884071 29185 consensus_peers.cc:597] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:31.893159 29111 consensus_peers.cc:597] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:31.898417 29038 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:31.886719 28965 catalog_manager.cc:5697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "a936878f59c840a6bd176a1d2d7347b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:31.855880 27934 tablet_replica.cc:333] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
W20260709 13:07:31.900408 29185 consensus_peers.cc:597] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:31.899427 29111 consensus_peers.cc:597] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:31.899678 28965 catalog_manager.cc:5697] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 19 to 28. New cstate: current_term: 28 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:31.876492  5378 raft_consensus.cc:493] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:31.912577  5378 raft_consensus.cc:515] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } attrs { replace: true } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:31.904838 29038 raft_consensus.cc:2686] T c0c33ea0ff53432c8be5e364242b23b3 P a936878f59c840a6bd176a1d2d7347b1: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:07:31.903111  5412 raft_consensus.cc:2804] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 13:07:31.928051 29111 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.928736  5412 raft_consensus.cc:697] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:31.928902 29111 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:31.930065  5412 consensus_queue.cc:237] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:31.915721  5378 leader_election.cc:290] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:31.947826  5407 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.948534  5407 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:31.949612 27934 raft_consensus.cc:2243] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.950582 27934 raft_consensus.cc:2272] T 9ca58eb3f3b147bc941a1c21818d370c P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:31.950856  5407 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:31.889022 28963 rpcz_store.cc:276] 0709 13:07:30.742500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:30.743739 (+  1239us) service_pool.cc:224] Handling call
0709 13:07:31.817241 (+1073502us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:07:31.544257 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:07:31.544965 (+   708us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:07:31.545013 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:07:31.545038 (+    25us) tablet.cc:662] Decoding operations
0709 13:07:31.551141 (+  6103us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:07:31.554001 (+  2860us) write_op.cc:641] Partition lock acquired for write op
0709 13:07:31.554038 (+    37us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:07:31.554779 (+   741us) tablet.cc:701] Row locks acquired
0709 13:07:31.554826 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:07:31.578770 (+ 23944us) write_op.cc:270] Start()
0709 13:07:31.578959 (+   189us) write_op.cc:276] Timestamp: P: 1783602451578656 usec, L: 0
0709 13:07:31.578998 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:07:31.579763 (+   765us) log.cc:844] Serialized 3024 byte log entry
0709 13:07:31.644916 (+ 65153us) op_driver.cc:464] REPLICATION: finished
0709 13:07:31.660880 (+ 15964us) write_op.cc:301] APPLY: starting
0709 13:07:31.660992 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 13:07:31.706446 (+ 45454us) tablet.cc:1370] finished BulkCheckPresence
0709 13:07:31.706578 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:07:31.734117 (+ 27539us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:07:31.734443 (+   326us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:07:31.734501 (+    58us) write_op.cc:312] APPLY: finished
0709 13:07:31.734882 (+   381us) log.cc:844] Serialized 73 byte log entry
0709 13:07:31.757567 (+ 22685us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:31.757688 (+   121us) write_op.cc:454] Released schema lock
0709 13:07:31.757767 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 13:07:31.758178 (+   411us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:07:31.758207 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":27,"spinlock_wait_cycles":10880,"child_traces":[["op",{"apply.queue_time_us":15642,"apply.run_cpu_time_us":11987,"apply.run_wall_time_us":97290,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33387,"num_ops":7,"prepare.queue_time_us":18599,"prepare.run_cpu_time_us":4622,"prepare.run_wall_time_us":90840,"replication_time_us":65750,"spinlock_wait_cycles":66304,"thread_start_us":72860,"threads_started":2}]]}
I20260709 13:07:31.953567 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:31.954450 29221 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 21.
W20260709 13:07:31.957476 29111 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.957975 29111 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:31.960343  5407 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:31.942584  5410 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 13:07:31.962198 29185 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:31.963281 29185 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:31.964208 29871 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:31.964919 29185 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:31.965895 29185 consensus_peers.cc:597] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:31.966503 29185 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:31.970217 27934 tablet_replica.cc:333] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:31.971398 27934 raft_consensus.cc:2243] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:31.972129 27934 raft_consensus.cc:2272] T 3c777774468e477ca1ab0352dff56c9c P a936878f59c840a6bd176a1d2d7347b1 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:31.972241 29871 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:31.973276 29871 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:31.973837 29871 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.974232 29871 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:31.926438 28965 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: config changed from index 14 to 15, 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } attrs { replace: true } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:31.977187 29185 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:31.977707 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:31.978806  5378 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:31.956014 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:07:31.982580 29871 consensus_peers.cc:597] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:31.983114  5255 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:07:31.986261 29871 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:31.989895  5410 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:31.990423  5410 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:31.990854  5410 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:07:31.996304 29111 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:32.017720 29111 consensus_peers.cc:597] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:32.024215  5422 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.024987  5422 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:31.977543 27934 tablet_replica.cc:333] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.026986 27934 raft_consensus.cc:2243] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.028038 27934 pending_rounds.cc:70] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Trying to abort 1 pending ops.
I20260709 13:07:32.028323 27934 pending_rounds.cc:77] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1: Aborting op as it isn't in flight: id { term: 20 index: 12 } timestamp: 7305635591326584832 op_type: NO_OP noop_request { }
I20260709 13:07:32.029958 27934 raft_consensus.cc:2889] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:32.030391 27934 raft_consensus.cc:2272] T ac412099b9f64d4e8bcceebe1c7ffe87 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.030676 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:32.031415 29148 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
W20260709 13:07:32.033846 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 129829us
I20260709 13:07:32.035579 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:32.032573  5410 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:32.040472 29871 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.033046  5422 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:32.042157 27934 tablet_replica.cc:333] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.037768 29221 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:32.044876 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:32.057472 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9574de46fcd490e84208e41907200b0" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:32.046160 29185 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:32.074880 29111 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.065196 29185 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:32.076925  5406 raft_consensus.cc:2749] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:07:32.078111 29185 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.078559 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:32.080703  5390 raft_consensus.cc:493] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:32.081073  5406 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:07:32.081313  5390 raft_consensus.cc:515] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:32.082288  5410 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:32.082846 29185 leader_election.cc:336] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:32.083719 29185 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.085484 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:32.083901  5390 leader_election.cc:290] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:32.090071 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ee8b4c1bf649b9b89ad6243f3890a8" candidate_uuid: "a936878f59c840a6bd176a1d2d7347b1" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:32.091939 29874 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:32.093619 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3e05d4f95940f5a4cac560d26d0ec8" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:32.094180 29217 raft_consensus.cc:3060] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:32.096453 29038 leader_election.cc:304] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a936878f59c840a6bd176a1d2d7347b1; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:32.102149  5408 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:07:32.102643  5408 raft_consensus.cc:493] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.103012  5408 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:32.079879 27934 raft_consensus.cc:2243] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.109941 27934 raft_consensus.cc:2272] T d8373166c3664170b540f729a4e3c93f P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.110842  5390 raft_consensus.cc:2749] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:32.084117 29185 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:32.116873 29217 raft_consensus.cc:2468] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 22.
I20260709 13:07:32.115310  5408 raft_consensus.cc:515] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:32.132311 29871 leader_election.cc:336] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.134907 29113 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:32.135412  5412 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.135694  5408 leader_election.cc:290] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:32.135990  5412 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:32.137529 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:32.138124 29148 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:32.141392 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79cdfc6a725145348b70034422ef477b" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:32.142005 29221 raft_consensus.cc:3060] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:07:32.138128  5412 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:32.146039  5407 raft_consensus.cc:2804] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:07:32.147717  5407 raft_consensus.cc:697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:32.148917  5407 consensus_queue.cc:237] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:32.150903 29111 consensus_peers.cc:597] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:32.156422 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 72726us
W20260709 13:07:32.157151 29185 consensus_peers.cc:597] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:32.169692 29148 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:32.155773 29221 raft_consensus.cc:2468] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 22.
I20260709 13:07:32.181725  5421 raft_consensus.cc:2749] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:32.185160 29874 leader_election.cc:304] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:32.186297  5408 raft_consensus.cc:2804] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:07:32.187048 29185 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:32.192893 29185 leader_election.cc:336] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.198786 29185 leader_election.cc:304] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:32.197872 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:32.192229  5408 raft_consensus.cc:697] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:32.189199 27934 tablet_replica.cc:333] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.200955  5408 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:32.207986  5406 raft_consensus.cc:2749] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:07:32.208735 27934 raft_consensus.cc:2243] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.215461 27934 raft_consensus.cc:2272] T e96bceabcbf747e0acec327bc1abc729 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.221338 27934 tablet_replica.cc:333] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.234730  5407 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.252341  5407 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:32.254659  5407 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:32.252811 27934 raft_consensus.cc:2243] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 28 LEADER]: Raft consensus shutting down.
W20260709 13:07:32.236845 29871 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:32.257941 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:32.258625 29221 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 25.
I20260709 13:07:32.259768 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:32.261539  5410 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:32.261922  5410 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.262316  5410 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:07:32.265211 29111 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.278599 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet b9574de46fcd490e84208e41907200b0 with delete_type TABLET_DATA_TOMBSTONED (TS 6ad9faa1c14444b29c9db8e71f46e2d6 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:55420
I20260709 13:07:32.281396  5410 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:32.262202 27934 raft_consensus.cc:2272] T 40b726ca47da439b9be7ad6d531144fe P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.291635 27934 tablet_replica.cc:333] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.301988  5412 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.302857 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813e480732fd4061837e784d4d768b1f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:32.302702  5412 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:32.303471 29221 raft_consensus.cc:3060] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:32.304136  5410 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:32.306635  5412 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:32.306747 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:32.313720 29221 raft_consensus.cc:2468] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 26.
I20260709 13:07:32.314831 29113 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:32.319867  5410 raft_consensus.cc:2804] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:32.322865 27934 raft_consensus.cc:2243] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:32.325001 29111 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.326988  5410 raft_consensus.cc:697] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:32.330885  5410 consensus_queue.cc:237] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:32.342547 27934 raft_consensus.cc:2272] T d2ee8b4c1bf649b9b89ad6243f3890a8 P a936878f59c840a6bd176a1d2d7347b1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.346122 27934 tablet_replica.cc:333] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.346982 27934 raft_consensus.cc:2243] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.347595 27934 pending_rounds.cc:70] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Trying to abort 1 pending ops.
I20260709 13:07:32.347844 27934 pending_rounds.cc:77] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1: Aborting op as it isn't in flight: id { term: 22 index: 13 } timestamp: 7305635589166764032 op_type: NO_OP noop_request { }
I20260709 13:07:32.348246 27934 raft_consensus.cc:2889] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:32.348660 27934 raft_consensus.cc:2272] T 2e40d8d1f334428db02ee54c8467d058 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.354483 28963 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 21 to 22, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 22 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:32.358879  5426 tablet_replica.cc:333] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:32.358745 28965 catalog_manager.cc:5697] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 21 to 22, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 22 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.360212  5426 raft_consensus.cc:2243] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.360947  5426 raft_consensus.cc:2272] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:32.384143 29185 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.384774 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:32.385867  5412 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:07:32.387207 29185 consensus_peers.cc:597] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:32.388075 29185 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:32.397149 29185 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:32.398371 29185 consensus_peers.cc:597] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:32.402526  5426 ts_tablet_manager.cc:1916] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:07:32.402755 29185 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:32.417979 29185 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:32.418850 29185 leader_election.cc:336] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.419487 29185 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:32.420089 27934 tablet_replica.cc:333] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.420565  5409 raft_consensus.cc:2749] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:32.421065 27934 raft_consensus.cc:2243] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.421772 27934 pending_rounds.cc:70] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Trying to abort 1 pending ops.
W20260709 13:07:32.425170 29185 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:32.432278 29185 consensus_peers.cc:597] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:32.459422 29871 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:32.421999 27934 pending_rounds.cc:77] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1: Aborting op as it isn't in flight: id { term: 21 index: 12 } timestamp: 7305635541005352960 op_type: NO_OP noop_request { }
I20260709 13:07:32.469301 27934 raft_consensus.cc:2889] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:32.469762 27934 raft_consensus.cc:2272] T 44d88fbf78394409bcacfadb40f377d6 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.473397 27934 tablet_replica.cc:333] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.474251 27934 raft_consensus.cc:2243] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.475188 27934 raft_consensus.cc:2272] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:32.493456 29185 consensus_peers.cc:597] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.512622  5426 ts_tablet_manager.cc:1929] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 21.13
I20260709 13:07:32.513262  5426 log.cc:1199] T b9574de46fcd490e84208e41907200b0 P 6ad9faa1c14444b29c9db8e71f46e2d6: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/wals/b9574de46fcd490e84208e41907200b0
I20260709 13:07:32.515435 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet b9574de46fcd490e84208e41907200b0 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:07:32.531241 29143 raft_consensus.cc:1275] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:07:32.533205  5378 consensus_queue.cc:1048] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:07:32.536706 27934 tablet_replica.cc:333] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.537721 27934 raft_consensus.cc:2243] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.538411 27934 raft_consensus.cc:2272] T e68315e143764938be60ec716705c15f P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.549379 28962 catalog_manager.cc:5697] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 25 to 26, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 26 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:32.573118  5416 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.573756  5416 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:32.578167 29217 raft_consensus.cc:1275] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:07:32.580152  5407 consensus_queue.cc:1048] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:07:32.582924 29871 leader_election.cc:336] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.584254  5416 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:32.587370 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:32.591012 29221 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 20.
I20260709 13:07:32.589753  5407 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.592058  5407 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:32.594354  5407 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:32.597294 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:32.599450 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:32.600828  5416 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:07:32.601290  5416 raft_consensus.cc:493] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:07:32.601596 29111 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.601770  5416 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:07:32.608685 27934 tablet_replica.cc:333] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.609223 29113 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:32.609622 27934 raft_consensus.cc:2243] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.610414 27934 raft_consensus.cc:2272] T cdb0c0ac78fc4de7a353a995edd44346 P a936878f59c840a6bd176a1d2d7347b1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.616124  5407 raft_consensus.cc:2749] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:32.644363 27934 tablet_replica.cc:333] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.645291 27934 raft_consensus.cc:2243] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.646023 27934 pending_rounds.cc:70] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Trying to abort 1 pending ops.
I20260709 13:07:32.646294 27934 pending_rounds.cc:77] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1: Aborting op as it isn't in flight: id { term: 23 index: 14 } timestamp: 7305635586182778880 op_type: NO_OP noop_request { }
I20260709 13:07:32.646677 27934 raft_consensus.cc:2889] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:32.647014 27934 raft_consensus.cc:2272] T d506d5cb9c9943c49f64dbb8fa09e5c9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.648406  5410 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.648977  5410 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:32.656790 29111 consensus_peers.cc:597] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:32.661062 29871 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:32.674544  5410 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:32.677934 28964 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 268) took 15988 ms (client timeout 15000 ms). Trace:
I20260709 13:07:32.680226 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet b9574de46fcd490e84208e41907200b0 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 10 (current committed config index is 14)) from {username='slave'} at 127.0.0.1:55420
I20260709 13:07:32.687475 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:32.691546 29143 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:07:32.671720 29111 leader_election.cc:336] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.705516 29113 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:32.701272 28964 rpcz_store.cc:269] 0709 13:07:16.689216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:24.181594 (+7492378us) service_pool.cc:224] Handling call
0709 13:07:32.677714 (+8496120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23505,"reported_tablets":50}
I20260709 13:07:32.708268  5408 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:07:32.705025 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet b9574de46fcd490e84208e41907200b0 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:07:32.700354 27934 tablet_replica.cc:333] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.721560 28962 catalog_manager.cc:5697] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 26 to 27, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 27 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } }
I20260709 13:07:32.749416 27934 raft_consensus.cc:2243] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.712294  5410 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:32.777357 27934 raft_consensus.cc:2272] T a385068bbb144e6fa0e8e9974398b1b9 P a936878f59c840a6bd176a1d2d7347b1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.782945  5439 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.783689  5439 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:32.790112 29111 consensus_peers.cc:597] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:32.790978 29111 consensus_peers.cc:597] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:32.784416 28967 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 166) took 72888 ms (client timeout 15000 ms). Trace:
I20260709 13:07:32.765476  5421 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
W20260709 13:07:32.795575 28967 rpcz_store.cc:269] 0709 13:06:19.895730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.896021 (+   291us) service_pool.cc:224] Handling call
0709 13:06:19.949497 (+ 53476us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b54001c0610. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e90b2 00007f783c962a89 00007f783c8fada6 00007f783c8cc088 00007f783c9ca37e 00007f783ae8b2a4 00007f783ae8b232 00007f783ae8b161 00007f783ae8b0dc
0709 13:06:20.030073 (+ 80576us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b5400018954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a77090 00007f7836b15f3a 00007f783c8cc1cc 00007f783c9ca37e 00007f783ae8b2a4 00007f783ae8b232 00007f783ae8b161 00007f783ae8b0dc 00007f783ae8a392 00007f7834e4848c 00007f7834e478c6
0709 13:07:32.784116 (+72754043us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5554874,"reported_tablets":30,"spinlock_wait_cycles":99098752}
W20260709 13:07:32.826362 29871 consensus_peers.cc:597] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.794863  5416 raft_consensus.cc:515] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:32.853482 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 51157.6us
W20260709 13:07:32.860893 28966 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 263) took 134275 ms (client timeout 15000 ms). Trace:
W20260709 13:07:32.861379 28966 rpcz_store.cc:269] 0709 13:05:18.592442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.616452 (+ 24010us) service_pool.cc:224] Handling call
0709 13:07:32.860639 (+134244187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":547765,"reported_tablets":49}
I20260709 13:07:32.864647 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:32.865490 29143 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:32.873342  5416 leader_election.cc:290] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:32.812367  5421 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:32.876034  5421 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:32.879547 27934 tablet_replica.cc:333] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1: stopping tablet replica
I20260709 13:07:32.880352 27934 raft_consensus.cc:2243] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.881028 27934 raft_consensus.cc:2272] T 2fae9bf99a434b83a73451cee1c56e6a P a936878f59c840a6bd176a1d2d7347b1 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:32.892409 29871 leader_election.cc:336] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.919374 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet b9574de46fcd490e84208e41907200b0 with delete_type TABLET_DATA_TOMBSTONED (Replica has no consensus available (current committed config index is 14)) from {username='slave'} at 127.0.0.1:55420
I20260709 13:07:32.921411 29188 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:32.931809  5420 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:32.932312  5420 raft_consensus.cc:493] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.932723  5420 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:32.945010 29148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:32.945771 29148 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 27.
W20260709 13:07:32.948093 29111 consensus_peers.cc:597] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:32.949414  5420 raft_consensus.cc:515] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:32.954453  5420 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:32.952248 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:32.956414  5437 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:07:32.956871  5437 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:32.960103  5437 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:07:32.958613 29185 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:32.968387  5439 leader_election.cc:290] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:32.980255 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 169664us
I20260709 13:07:32.987589 29221 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:07:32.989553 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc31aa85851e41609957a765b2d40774" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:32.990181 29221 raft_consensus.cc:3060] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:07:32.997417 29111 consensus_peers.cc:597] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:33.010232  5433 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:07:33.021047 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 65180.4us
I20260709 13:07:33.025911 29221 raft_consensus.cc:2468] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 21.
I20260709 13:07:32.948544 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet b9574de46fcd490e84208e41907200b0 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
W20260709 13:07:33.026968 28957 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 78442.5us
I20260709 13:07:33.029114  5437 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:33.030342 29874 leader_election.cc:304] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:33.040326  5408 raft_consensus.cc:2804] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:07:33.040979  5408 raft_consensus.cc:697] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:33.042089  5408 consensus_queue.cc:237] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:33.052928 29221 raft_consensus.cc:1275] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
W20260709 13:07:33.064662 29185 leader_election.cc:336] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.042562  5437 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:33.075407 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cddea8e9cff6495ba269c65b0edf8b6c" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:33.076119 29143 raft_consensus.cc:3060] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:33.066715  5441 consensus_queue.cc:1048] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:07:33.082154 28966 catalog_manager.cc:5697] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 20 to 21, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 21 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:33.086192 29143 raft_consensus.cc:2468] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
I20260709 13:07:33.093817 29141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:33.094499 29141 raft_consensus.cc:3060] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:07:33.121973 29185 leader_election.cc:336] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.123857 29141 raft_consensus.cc:2468] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 28.
W20260709 13:07:33.135318 29111 consensus_peers.cc:597] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:33.136839 28968 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 165) took 94829 ms (client timeout 15000 ms). Trace:
W20260709 13:07:33.137275 28968 rpcz_store.cc:269] 0709 13:05:58.307355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:58.307631 (+   276us) service_pool.cc:224] Handling call
0709 13:07:33.121088 (+94813457us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33566,"reported_tablets":29}
I20260709 13:07:33.166189 29188 leader_election.cc:304] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:33.178576  5412 raft_consensus.cc:2804] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:33.179395  5412 raft_consensus.cc:697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:33.180968 29188 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:33.181187  5412 consensus_queue.cc:237] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:33.181883  5409 raft_consensus.cc:2804] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:07:33.183234  5409 raft_consensus.cc:697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:33.184398  5409 consensus_queue.cc:237] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:33.188819 29185 leader_election.cc:336] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:33.333422 29871 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:33.339826 28969 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58574 (request call id 276) took 62368 ms (client timeout 15000 ms). Trace:
W20260709 13:07:33.341708 28971 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52826 (request call id 164) took 116821 ms (client timeout 15000 ms). Trace:
W20260709 13:07:33.348299 28971 rpcz_store.cc:269] 0709 13:05:36.528258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.532279 (+  4021us) service_pool.cc:224] Handling call
0709 13:07:33.341519 (+116809240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20509,"reported_tablets":29}
W20260709 13:07:33.340370 28969 rpcz_store.cc:269] 0709 13:06:30.971779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:30.972128 (+   349us) service_pool.cc:224] Handling call
0709 13:06:31.641860 (+669732us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400018954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a77090 00007f7836b15f3a 00007f783c8cc1cc 00007f783c9ca37e 00007f783ae8b2a4 00007f783ae8b232 00007f783ae8b161 00007f783ae8b0dc 00007f783ae8a392 00007f7834e4848c 00007f7834e478c6
0709 13:07:33.339613 (+61697753us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33293,"reported_tablets":50,"spinlock_wait_cycles":444378496}
I20260709 13:07:33.367720  5409 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:33.368321  5409 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:33.373383 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
W20260709 13:07:33.376291 29185 consensus_peers.cc:597] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:33.379376  5409 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:33.400893 29204 tablet_service.cc:1558] Processing DeleteTablet for tablet b9574de46fcd490e84208e41907200b0 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 10 (current committed config index is 14)) from {username='slave'} at 127.0.0.1:55420
I20260709 13:07:33.404772  5441 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:33.404984  5432 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:33.405447  5432 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:33.411471 28957 catalog_manager.cc:5028] TS 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): tablet b9574de46fcd490e84208e41907200b0 (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
I20260709 13:07:33.415329 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:33.416067 29217 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 28.
I20260709 13:07:33.417321 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
W20260709 13:07:33.413758 29185 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.418315 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:33.418944  5436 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:07:33.419112  5409 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:07:33.419471  5436 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:33.419976  5436 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:07:33.420867 29185 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:33.405406  5441 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:33.424850  5441 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:33.425870 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:33.407718  5432 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:33.433900 29111 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:33.440356 29111 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.440866 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:33.443709  5432 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:33.446372 28970 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58588 (request call id 269) took 1357 ms. Trace:
I20260709 13:07:33.446844 28970 rpcz_store.cc:276] 0709 13:07:32.089261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:32.089389 (+   128us) service_pool.cc:224] Handling call
0709 13:07:33.446308 (+1356919us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10749,"reported_tablets":50,"spinlock_wait_cycles":14336}
I20260709 13:07:33.442698  5436 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:33.451259 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40b726ca47da439b9be7ad6d531144fe" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:33.452000 29216 raft_consensus.cc:3060] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:07:33.451534  5436 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:33.453006  5437 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:33.453517  5437 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:33.447810 29217 consensus_queue.cc:237] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:33.455611  5437 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:33.461247 29185 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:33.462498 29111 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.462574 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2" is_pre_election: true
I20260709 13:07:33.465612 29143 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 25.
I20260709 13:07:33.467487 29188 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:33.468665  5437 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:07:33.462729 29216 raft_consensus.cc:2468] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 29.
I20260709 13:07:33.479557 29113 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:33.474956 28970 catalog_manager.cc:5697] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 27 to 28, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 28 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:33.475188  5437 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a936878f59c840a6bd176a1d2d7347b1)
I20260709 13:07:33.481374  5437 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:33.482735  5436 raft_consensus.cc:2804] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:07:33.485273  5436 raft_consensus.cc:697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 29 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:33.483678 28970 catalog_manager.cc:5697] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 25 to 26, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 26 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:33.486192  5436 consensus_queue.cc:237] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:33.510270 29902 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:07:33.512693  5409 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:07:33.518469 29871 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:33.526149  5437 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:33.528532  5437 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:33.530632  5408 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.531424 29143 raft_consensus.cc:1275] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:07:33.531184  5408 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:33.533547 29185 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.536245 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:33.536970 29217 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 24.
I20260709 13:07:33.536594 28970 catalog_manager.cc:5697] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 28 to 29, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 29 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:33.538424 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:33.539285  5433 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:07:33.539707  5433 raft_consensus.cc:493] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.540127  5433 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:33.540386 29143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "730186a8c70f43f2a57d80163bbc37d2"
I20260709 13:07:33.541016 29143 raft_consensus.cc:3060] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:07:33.544346  5409 raft_consensus.cc:2955] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:33.548563  5420 consensus_queue.cc:1048] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:07:33.551009 29111 consensus_peers.cc:597] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:33.560248 29871 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.544247  5408 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:33.567548 29902 raft_consensus.cc:2955] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:33.582010 29871 consensus_peers.cc:597] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:33.585737 29143 raft_consensus.cc:2468] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9faa1c14444b29c9db8e71f46e2d6 in term 26.
I20260709 13:07:33.587124 29188 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:33.588428  5439 raft_consensus.cc:2804] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:07:33.588989  5439 raft_consensus.cc:697] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Becoming Leader. State: Replica: 6ad9faa1c14444b29c9db8e71f46e2d6, State: Running, Role: LEADER
I20260709 13:07:33.590101  5439 consensus_queue.cc:237] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:33.596485 29141 raft_consensus.cc:2955] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:33.567795  5433 raft_consensus.cc:515] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:33.605894 28957 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c0c33ea0ff53432c8be5e364242b23b3 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:33.611557 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3842b36ffd134cd28c657475df48e741" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:33.612262 29216 raft_consensus.cc:3060] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:07:33.625939 29216 raft_consensus.cc:2468] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 25.
I20260709 13:07:33.627247 29874 leader_election.cc:304] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:33.627048 28970 catalog_manager.cc:5697] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 15 to 16, VOTER a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) evicted. New cstate: current_term: 20 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:33.628222  5408 raft_consensus.cc:2804] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:07:33.636988  5408 raft_consensus.cc:697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:33.638075  5408 consensus_queue.cc:237] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:33.654502  5433 leader_election.cc:290] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:33.663405 29185 consensus_peers.cc:597] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:33.657711 29871 leader_election.cc:336] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:33.667515 29216 raft_consensus.cc:1275] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:07:33.671231 29185 consensus_peers.cc:597] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:33.677002  5440 consensus_queue.cc:1048] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:07:33.681593 28971 catalog_manager.cc:5697] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 reported cstate change: term changed from 25 to 26, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131). New cstate: current_term: 26 leader_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:33.710276 28955 catalog_manager.cc:4755] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c0c33ea0ff53432c8be5e364242b23b3: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:07:33.748786 29143 raft_consensus.cc:1275] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:07:33.750039 28971 catalog_manager.cc:5697] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 23 to 25, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 25 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:33.756211  5420 consensus_queue.cc:1048] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:07:33.769951 29141 raft_consensus.cc:1275] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:07:33.771622  5420 consensus_queue.cc:1048] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:07:33.925490 29185 connection.cc:573] client connection to 127.27.71.129:41631 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:07:33.926055 29185 consensus_peers.cc:597] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:33.933944 29216 raft_consensus.cc:1275] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 28 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
W20260709 13:07:33.935621 29111 connection.cc:769] client connection to 127.27.71.129:41631 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 13:07:33.940744  5436 consensus_queue.cc:1048] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:07:33.944926 29871 connection.cc:573] client connection to 127.27.71.129:41631 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 13:07:33.950858  5436 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.957073  5436 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:33.959239  5436 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:33.961233 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:33.962095 29217 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 27.
I20260709 13:07:33.964727 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:33.965732  5436 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:07:33.969406  5436 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.969897  5436 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:34.010182  5436 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:34.018246 29185 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:34.043948 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea48540c1af24bf3b55edec5c52ec133" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:34.044602 29217 raft_consensus.cc:3060] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:07:34.069468 29217 raft_consensus.cc:2468] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 730186a8c70f43f2a57d80163bbc37d2 in term 28.
I20260709 13:07:34.071182 29113 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2; no voters: 
I20260709 13:07:34.072350  5355 raft_consensus.cc:2804] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:07:34.072975  5355 raft_consensus.cc:697] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 LEADER]: Becoming Leader. State: Replica: 730186a8c70f43f2a57d80163bbc37d2, State: Running, Role: LEADER
I20260709 13:07:34.074832  5355 consensus_queue.cc:237] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:34.080811  5436 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 election: Requested vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:34.108418 29185 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:34.109189 29148 raft_consensus.cc:1275] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Refusing update from remote peer 6ad9faa1c14444b29c9db8e71f46e2d6: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 13:07:34.108417 29871 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.110242 29871 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.111768 29871 consensus_peers.cc:597] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:34.117887  5438 consensus_queue.cc:1048] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:07:34.137182 29111 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:34.139729 29111 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:34.144889 29111 consensus_peers.cc:597] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:34.149302 29216 raft_consensus.cc:1275] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:07:34.153199  5433 consensus_queue.cc:1048] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:07:34.202590 29185 consensus_peers.cc:597] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.203164 29111 consensus_peers.cc:597] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:34.204097 29185 consensus_peers.cc:597] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:34.203794 28971 catalog_manager.cc:5697] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 reported cstate change: term changed from 26 to 28, leader changed from a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129) to 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130). New cstate: current_term: 28 leader_uuid: "730186a8c70f43f2a57d80163bbc37d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:34.251142 29871 consensus_peers.cc:597] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:34.290289 29185 consensus_peers.cc:597] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.307606 29871 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.354323 29871 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.409121 29185 consensus_peers.cc:597] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.446496 29111 consensus_peers.cc:597] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.447207 29111 consensus_peers.cc:597] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:34.474069 29871 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.581281 29185 consensus_peers.cc:597] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.612782 29185 consensus_peers.cc:597] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.613382 29185 consensus_peers.cc:597] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:34.619951 29185 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.650110 29185 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:34.664161 28960 maintenance_manager.cc:419] P 9c100c1aaf1b493089df815bded98302: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033363
I20260709 13:07:34.689726 29217 raft_consensus.cc:1275] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Refusing update from remote peer 730186a8c70f43f2a57d80163bbc37d2: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 13:07:34.691350  5355 consensus_queue.cc:1048] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:07:34.706784 29111 consensus_peers.cc:597] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:34.726060 29902 consensus_queue.cc:237] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:34.746362 29111 consensus_peers.cc:597] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:34.807036 29185 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.809376 29871 consensus_peers.cc:597] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:34.847071 29217 raft_consensus.cc:1275] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:07:34.851200  5475 consensus_queue.cc:1048] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:07:34.860548 28959 maintenance_manager.cc:643] P 9c100c1aaf1b493089df815bded98302: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.195s	user 0.036s	sys 0.008s Metrics: {"bytes_written":7916,"delete_count":0,"lbm_write_time_us":2785,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":6,"reinsert_count":0,"update_count":317}
I20260709 13:07:34.863060 28960 maintenance_manager.cc:419] P 9c100c1aaf1b493089df815bded98302: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.377252
W20260709 13:07:34.952015 29185 consensus_peers.cc:597] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:34.952616 29185 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:41631, user_credentials={real_user=slave}} blocked reactor thread for 48078.2us
W20260709 13:07:34.963387 27934 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 29094)
W20260709 13:07:35.004554 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 87166.4us
I20260709 13:07:35.021065  5440 raft_consensus.cc:2955] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Committing config change with OpId 22.16: config changed from index 8 to 16, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
W20260709 13:07:35.027951 29871 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:35.016686 29185 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:35.040364 29111 consensus_peers.cc:597] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:35.042924 29221 raft_consensus.cc:2955] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 8 to 16, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } } }
I20260709 13:07:35.056010 28956 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 79cdfc6a725145348b70034422ef477b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
I20260709 13:07:35.063495 28970 catalog_manager.cc:5697] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: config changed from index 8 to 16, VOTER 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) evicted. New cstate: current_term: 22 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:35.078948 29185 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:35.079878 29185 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:35.106577 29871 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:35.110709 29185 consensus_peers.cc:597] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.124320 29111 consensus_peers.cc:597] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:35.164831 29111 consensus_peers.cc:597] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:35.169629 28955 catalog_manager.cc:4755] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c0c33ea0ff53432c8be5e364242b23b3: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:35.360983 29134 tablet_service.cc:1558] Processing DeleteTablet for tablet 79cdfc6a725145348b70034422ef477b with delete_type TABLET_DATA_TOMBSTONED (TS 730186a8c70f43f2a57d80163bbc37d2 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:36428
I20260709 13:07:35.437355 28959 maintenance_manager.cc:643] P 9c100c1aaf1b493089df815bded98302: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.571s	user 0.122s	sys 0.071s Metrics: {"cfile_cache_miss":10,"cfile_cache_miss_bytes":28909,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":39564,"dirs.run_cpu_time_us":591,"dirs.run_wall_time_us":5248,"lbm_read_time_us":11334,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":25,"lbm_write_time_us":1917,"lbm_writes_lt_1ms":12,"peak_mem_usage":579520,"reinsert_count":0,"spinlock_wait_cycles":2668288,"thread_start_us":109503,"threads_started":2,"update_count":326}
I20260709 13:07:35.440075  5497 tablet_replica.cc:333] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:35.440495 28960 maintenance_manager.cc:419] P 9c100c1aaf1b493089df815bded98302: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:07:35.441263 28959 maintenance_manager.cc:643] P 9c100c1aaf1b493089df815bded98302: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260709 13:07:35.441454  5497 raft_consensus.cc:2243] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.453367  5497 raft_consensus.cc:2272] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.456637  5497 ts_tablet_manager.cc:1916] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:07:35.458807 29871 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:35.496340  5497 ts_tablet_manager.cc:1929] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.15
I20260709 13:07:35.512022  5497 log.cc:1199] T 79cdfc6a725145348b70034422ef477b P 730186a8c70f43f2a57d80163bbc37d2: Deleting WAL directory at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/wals/79cdfc6a725145348b70034422ef477b
W20260709 13:07:35.512703 29111 consensus_peers.cc:597] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:35.517108 28958 catalog_manager.cc:5028] TS 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): tablet 79cdfc6a725145348b70034422ef477b (table test-workload [id=4d8d882ecc6b449a9b7e83c45a0d44b5]) successfully deleted
W20260709 13:07:35.532898 29111 consensus_peers.cc:597] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:35.963936 27934 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 29094)
W20260709 13:07:35.979125 29871 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:36.031849 29871 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.085685 29111 consensus_peers.cc:597] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.131286 29871 consensus_peers.cc:597] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:36.146337 29185 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.166781 29111 consensus_peers.cc:597] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.217535 29185 consensus_peers.cc:597] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:36.263948 29185 consensus_peers.cc:597] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:36.304708 28955 catalog_manager.cc:4755] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c0c33ea0ff53432c8be5e364242b23b3: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:36.361076 29185 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.452339 29871 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.460135 29111 consensus_peers.cc:597] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:36.463687 29871 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.524216 29185 consensus_peers.cc:597] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.529548 29871 consensus_peers.cc:597] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:36.589454 29185 consensus_peers.cc:597] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:36.613926 29871 consensus_peers.cc:597] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.619796 29111 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111) [suppressed 491 similar messages]
W20260709 13:07:36.622742 29111 consensus_peers.cc:597] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:36.644781 29185 consensus_peers.cc:597] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.721658 29185 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.764133 29871 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.795202 29185 consensus_peers.cc:597] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.819787 29185 consensus_peers.cc:597] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.922410 29111 consensus_peers.cc:597] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.961140 29185 consensus_peers.cc:597] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:36.964496 27934 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 29094)
W20260709 13:07:36.999780 29185 consensus_peers.cc:597] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:37.017874 29111 consensus_peers.cc:597] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:37.044124 29111 consensus_peers.cc:597] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:37.075884 29185 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:37.138849 29185 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:37.146502 29185 consensus_peers.cc:597] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:37.247846 29185 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:37.309262 29111 consensus_peers.cc:597] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:37.323887 29185 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:37.367424 29185 consensus_peers.cc:597] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:37.427362 29185 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:37.443032 29111 consensus_peers.cc:597] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:37.451181 29111 consensus_peers.cc:597] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:37.494343 29111 consensus_peers.cc:597] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:37.613070 29185 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:37.912645 29185 consensus_peers.cc:597] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:37.954460 29111 consensus_peers.cc:597] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:37.965862 27934 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 29094)
W20260709 13:07:38.048310 29871 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:38.052757 29871 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:38.053594 29871 consensus_peers.cc:597] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:38.305501 29111 consensus_peers.cc:597] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:38.324720 27934 tablet_server.cc:196] TabletServer@127.27.71.129:0 shutdown complete.
W20260709 13:07:38.410735 28955 catalog_manager.cc:4755] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c0c33ea0ff53432c8be5e364242b23b3: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:38.520272 29871 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:38.550136 29871 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:38.620786 29111 consensus_peers.cc:597] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:38.641039 29871 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:38.659058 29185 consensus_peers.cc:597] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:38.675731 29111 consensus_peers.cc:597] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:38.727234 29185 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:38.771420 27934 tablet_server.cc:179] TabletServer@127.27.71.130:0 shutting down...
W20260709 13:07:38.779063 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 61020.2us
W20260709 13:07:38.787637 29188 consensus_peers.cc:597] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.798010 29185 consensus_peers.cc:597] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:38.806085 29875 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.806903 29875 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.818614 29188 consensus_peers.cc:597] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.837028 29188 consensus_peers.cc:597] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.865236 29188 consensus_peers.cc:597] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.867594 29185 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:38.870724 29111 consensus_peers.cc:597] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:38.883505 29188 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.913478 29188 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.919166 29188 consensus_peers.cc:597] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.918880 29875 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.921381 29188 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.939419 29188 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.944664 29875 consensus_peers.cc:597] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.955156 29185 consensus_peers.cc:597] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:38.976487 29185 consensus_peers.cc:597] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:38.995759 29188 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:38.999547 29186 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:36041, user_credentials={real_user=slave}} blocked reactor thread for 99775.9us
W20260709 13:07:39.010284 29188 consensus_peers.cc:597] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.010993 29188 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:39.030511 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:39.034191 27934 tablet_replica.cc:333] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.035338 27934 raft_consensus.cc:2243] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:39.036950 27934 raft_consensus.cc:2272] T d7517b9e93904801886467474f000b2d P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.040762 27934 tablet_replica.cc:333] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.041854 27934 raft_consensus.cc:2243] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:07:39.043125 27934 raft_consensus.cc:2272] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.046356 27934 tablet_replica.cc:333] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.047338 27934 raft_consensus.cc:2243] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.047973 27934 raft_consensus.cc:2272] T 9a29f89bc1cb471d94fee8d14f56314c P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.052809 27934 tablet_replica.cc:333] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.053807 27934 raft_consensus.cc:2243] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:07:39.055320 27934 raft_consensus.cc:2272] T a57ab99b58a448de88023e7166f78ce2 P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.058890 27934 tablet_replica.cc:333] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.060670 27934 raft_consensus.cc:2243] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.061775 27934 raft_consensus.cc:2272] T b9574de46fcd490e84208e41907200b0 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:39.070355 29185 consensus_peers.cc:597] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:39.071473 27934 tablet_replica.cc:333] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.072499 27934 raft_consensus.cc:2243] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.073312 27934 raft_consensus.cc:2272] T b55b583a63b64ca6832b73f813985e49 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:39.081995 29188 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:39.094292 27934 tablet_replica.cc:333] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.095342 27934 raft_consensus.cc:2243] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:39.105822 29111 consensus_peers.cc:597] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:39.113369 29875 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.114728 29875 consensus_peers.cc:597] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:39.115492 27934 raft_consensus.cc:2272] T ee1e380ba6084a0e9d9978fa9983edc1 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:39.124111 29188 consensus_peers.cc:597] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.125124 29188 consensus_peers.cc:597] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:39.131564 27934 tablet_replica.cc:333] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.138262 27934 raft_consensus.cc:2243] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.144546 27934 raft_consensus.cc:2272] T 5acfd220cbb54cd181c08050de276f65 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:39.125910 29188 consensus_peers.cc:597] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.138661 29875 consensus_peers.cc:597] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:39.157284 27934 tablet_replica.cc:333] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
W20260709 13:07:39.168653 29875 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:39.171772 27934 raft_consensus.cc:2243] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:07:39.182348 27934 raft_consensus.cc:2272] T ea48540c1af24bf3b55edec5c52ec133 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.185112 27934 tablet_replica.cc:333] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.186038 27934 raft_consensus.cc:2243] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.188354 27934 raft_consensus.cc:2272] T 01d4b885cd86484fbaa6c528fc2ed1bc P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:39.189680 29875 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.198560 29871 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:39.191155 27934 tablet_replica.cc:333] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
W20260709 13:07:39.179843 29185 consensus_peers.cc:597] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:39.211891 27934 raft_consensus.cc:2243] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:39.219974 29185 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:39.223394 29185 consensus_peers.cc:597] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:39.225632 29188 consensus_peers.cc:597] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.229703 29875 consensus_peers.cc:597] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.235389 29875 consensus_peers.cc:597] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:39.236131 27934 raft_consensus.cc:2272] T ac412099b9f64d4e8bcceebe1c7ffe87 P 730186a8c70f43f2a57d80163bbc37d2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:39.259960 29871 consensus_peers.cc:597] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:39.262162 27934 tablet_replica.cc:333] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.267518 27934 raft_consensus.cc:2243] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.274734 27934 raft_consensus.cc:2272] T 3c777774468e477ca1ab0352dff56c9c P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:39.285596 29875 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.293365 29188 consensus_peers.cc:597] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.297346 29188 consensus_peers.cc:597] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.298161 29188 consensus_peers.cc:597] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:39.309412 27934 tablet_replica.cc:333] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.317269 27934 raft_consensus.cc:2243] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.328554 27934 raft_consensus.cc:2272] T 984bea8952114e3f9c0a045e7f557e37 P 730186a8c70f43f2a57d80163bbc37d2 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:39.329820 29185 consensus_peers.cc:597] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:39.327909 29871 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:39.350826 27934 tablet_replica.cc:333] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.367812 27934 raft_consensus.cc:2243] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:07:39.383863 29113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 69430.7us
W20260709 13:07:39.406315 29871 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:39.406639 29185 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:39.432605 29111 consensus_peers.cc:597] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:39.441999 27934 raft_consensus.cc:2272] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:39.450657 29188 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:39.461352 29188 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:39.463531 27934 tablet_replica.cc:333] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.464520 27934 raft_consensus.cc:2243] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.465427 27934 raft_consensus.cc:2272] T c24ba590bc46427dbac6cd4a4da67ee8 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.500006 27934 tablet_replica.cc:333] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.508296 27934 raft_consensus.cc:2243] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:39.512794 27934 raft_consensus.cc:2272] T 7f1cace028c94f79adfce2be604fb35a P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.518807 27934 tablet_replica.cc:333] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
W20260709 13:07:39.529342 29111 consensus_peers.cc:597] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:39.529886 27934 raft_consensus.cc:2243] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.550413 27934 raft_consensus.cc:2272] T e9334a7c40c744cbbff4dec0358f721b P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.647861 27934 tablet_replica.cc:333] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.649046 27934 raft_consensus.cc:2243] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:07:39.650033 27934 raft_consensus.cc:2272] T 3aaa941bea174ae08dc8aad56e0425f2 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.753098  5512 raft_consensus.cc:493] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:39.753751  5512 raft_consensus.cc:515] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:39.756911  5512 leader_election.cc:290] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:39.766803 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e40d8d1f334428db02ee54c8467d058" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:39.787253 27934 tablet_replica.cc:333] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.804149 27934 raft_consensus.cc:2243] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.804780 27934 raft_consensus.cc:2272] T c54509bf16d642138312814fefa3f6dc P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.807534 27934 tablet_replica.cc:333] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
W20260709 13:07:39.817412 29111 leader_election.cc:336] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:39.817930 29111 leader_election.cc:304] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:39.825066  5512 raft_consensus.cc:2749] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:39.827494 27934 raft_consensus.cc:2243] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.841574  5512 raft_consensus.cc:493] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:39.842399  5512 raft_consensus.cc:515] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:39.844956  5512 leader_election.cc:290] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:39.846412 27934 raft_consensus.cc:2272] T 082c896dd04e451198eba8392e70fba7 P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.846534 29221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8195256ce88462cab8bb734ddbb65ad" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:39.864281  5512 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:39.864940  5512 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:39.868647  5512 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:39.870893 27934 tablet_replica.cc:333] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.871876 27934 raft_consensus.cc:2243] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:39.873128 27934 raft_consensus.cc:2272] T 12e21412fcd24f95a8e3635fc5bbe138 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.876076  5512 raft_consensus.cc:493] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:39.876605  5512 raft_consensus.cc:515] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:39.880788  5479 raft_consensus.cc:493] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:39.881330 29216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09437a3ae8d46379ccdbb1829b14f27" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:39.881667 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:39.881367  5479 raft_consensus.cc:515] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:39.885155  5479 leader_election.cc:290] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:39.886502 29185 consensus_peers.cc:597] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:39.887173 29185 consensus_peers.cc:597] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:39.888525 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:39.897305 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb329a9e56064f1480f8f42e09ff6679" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:39.898908 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9f522a55d4b4051a75e0bf87cd2619e" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:39.899173 29112 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:39.900027  5479 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:39.904554 29112 leader_election.cc:304] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:39.905720  5479 raft_consensus.cc:2749] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:39.913931  5512 leader_election.cc:290] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:39.917305  5512 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:39.917846  5512 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:39.923219 29871 consensus_peers.cc:597] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:39.923770 29871 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:39.924253 29871 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:39.927100 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:39.937595 27934 tablet_replica.cc:333] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.937799 29910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0c33ea0ff53432c8be5e364242b23b3" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:39.938532 27934 raft_consensus.cc:2243] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.939293 27934 raft_consensus.cc:2272] T 128a7406f55d4400bfd5fadcbc14d065 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:39.941701 29112 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:39.942528  5479 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:39.935129 29111 leader_election.cc:336] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:39.944054  5512 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:39.943619 29111 leader_election.cc:304] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:39.948571 29185 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:39.949254 29185 consensus_peers.cc:597] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:39.951386  5512 raft_consensus.cc:2749] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:39.956794 27934 tablet_replica.cc:333] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:39.970320 27934 raft_consensus.cc:2243] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:39.970958 27934 raft_consensus.cc:2272] T a9f522a55d4b4051a75e0bf87cd2619e P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:39.959820 29185 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:39.969383 29871 consensus_peers.cc:597] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:40.004690 27934 tablet_replica.cc:333] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
W20260709 13:07:40.005759 29185 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:40.006551 29111 leader_election.cc:336] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:40.007460 29111 leader_election.cc:336] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.007877 29111 leader_election.cc:304] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.005934 27934 raft_consensus.cc:2243] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.009003  5512 raft_consensus.cc:2749] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:40.014846  5479 raft_consensus.cc:493] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:40.015512  5479 raft_consensus.cc:515] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:40.017688  5503 raft_consensus.cc:493] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:40.018110  5479 leader_election.cc:290] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:40.018501  5503 raft_consensus.cc:515] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:40.020074 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0c0ac78fc4de7a353a995edd44346" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:40.021271  5503 leader_election.cc:290] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:40.022701 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca58eb3f3b147bc941a1c21818d370c" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:40.026726 27934 raft_consensus.cc:2272] T 43a52106dbee45398d36a633f281eef6 P 730186a8c70f43f2a57d80163bbc37d2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:40.037046 29111 consensus_peers.cc:597] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:40.032922 27934 tablet_replica.cc:333] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.040267 27934 raft_consensus.cc:2243] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.042873 27934 raft_consensus.cc:2272] T cddea8e9cff6495ba269c65b0edf8b6c P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:40.053261 29185 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:40.047900 27934 tablet_replica.cc:333] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.077710 27934 raft_consensus.cc:2243] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.083425  5512 raft_consensus.cc:493] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:40.084080  5512 raft_consensus.cc:515] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:40.084910 27934 raft_consensus.cc:2272] T e250871b4ad74e079a538cdb04065b00 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.086203  5512 leader_election.cc:290] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:40.087337  5512 raft_consensus.cc:493] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:40.087414 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c8a2336e86c4846bf4e69ec61ba3fe0" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:40.088251 27934 tablet_replica.cc:333] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.088034  5512 raft_consensus.cc:515] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:40.089177 27934 raft_consensus.cc:2243] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:40.091089 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c59eab5922dc4c07a3a86e915e3cab7f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:40.091527  5512 leader_election.cc:290] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:40.094446  5503 raft_consensus.cc:493] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:40.096042  5503 raft_consensus.cc:515] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:40.098160  5503 leader_election.cc:290] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:40.099889 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a385068bbb144e6fa0e8e9974398b1b9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:40.106129 29111 leader_election.cc:336] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.095774 27934 raft_consensus.cc:2272] T 88cabbc6d2424a53a76857f415831c36 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.106729 29111 leader_election.cc:304] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.107630  5512 raft_consensus.cc:2749] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:07:40.109341 29111 leader_election.cc:336] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.109639 27934 tablet_replica.cc:333] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.109871 29111 leader_election.cc:304] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.110598 27934 raft_consensus.cc:2243] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:07:40.110723  5512 raft_consensus.cc:2749] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:40.115090 27934 raft_consensus.cc:2272] T 813e480732fd4061837e784d4d768b1f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:40.120383 29111 leader_election.cc:336] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.120985 29111 leader_election.cc:304] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.121776  5512 raft_consensus.cc:493] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:40.122309  5512 raft_consensus.cc:515] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:40.124164  5503 raft_consensus.cc:2749] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:07:40.125183 29111 leader_election.cc:336] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.124466  5512 leader_election.cc:290] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:40.125787 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e68315e143764938be60ec716705c15f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:40.126312 29111 leader_election.cc:304] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.129755  5503 raft_consensus.cc:2749] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:07:40.130336 29871 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:40.146194 29111 leader_election.cc:336] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.146816 29111 leader_election.cc:304] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.147583  5503 raft_consensus.cc:2749] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:40.137384 27934 tablet_replica.cc:333] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.149535 27934 raft_consensus.cc:2243] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.150349 27934 raft_consensus.cc:2272] T e8195256ce88462cab8bb734ddbb65ad P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.155660 27934 tablet_replica.cc:333] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.166531  5503 raft_consensus.cc:493] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:40.167385 27934 raft_consensus.cc:2243] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.167120  5503 raft_consensus.cc:515] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:40.168049 27934 raft_consensus.cc:2272] T c59eab5922dc4c07a3a86e915e3cab7f P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:40.166272 29111 leader_election.cc:336] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.169247 29111 leader_election.cc:304] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 7e78ffbcc98242068a874a5d9e341477, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.169363  5503 leader_election.cc:290] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:40.170143  5512 raft_consensus.cc:2749] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:07:40.170696 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:40.171949 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e4dbd865b648aaa1bce128c9de27af" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:40.173595 29113 leader_election.cc:304] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:40.174921  5503 raft_consensus.cc:2749] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:07:40.184131 27934 tablet_replica.cc:333] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.190093  5503 raft_consensus.cc:493] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 7e78ffbcc98242068a874a5d9e341477)
I20260709 13:07:40.193202  5503 raft_consensus.cc:515] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:40.195349  5503 leader_election.cc:290] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:40.196219 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:40.197576 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be73fc18460b4609ad4da2946e3acb87" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
I20260709 13:07:40.198988 29112 leader_election.cc:304] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:40.199203 27934 raft_consensus.cc:2243] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.200011 27934 raft_consensus.cc:2272] T 344aa5f73f0f4124b1512832a0bca2c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.201962  5503 raft_consensus.cc:2749] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:40.202708 27934 tablet_replica.cc:333] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.211889 27934 raft_consensus.cc:2243] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:40.214002 27934 raft_consensus.cc:2272] T 77b0625d42ef413aae4ec112d9ba450b P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.218930 27934 tablet_replica.cc:333] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.219607  5503 raft_consensus.cc:493] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:40.225378  5503 raft_consensus.cc:515] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:40.227632  5503 leader_election.cc:290] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:40.224727  5512 raft_consensus.cc:493] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:40.228819  5512 raft_consensus.cc:515] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:40.229178 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8373166c3664170b540f729a4e3c93f" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:40.230845  5512 leader_election.cc:290] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:40.232007 27934 raft_consensus.cc:2243] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:07:40.232533 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d506d5cb9c9943c49f64dbb8fa09e5c9" candidate_uuid: "730186a8c70f43f2a57d80163bbc37d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:40.233268 27934 raft_consensus.cc:2272] T 1fdfda5dcdd94083afe1620564f9d376 P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.236310 27934 tablet_replica.cc:333] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.243456 27934 raft_consensus.cc:2243] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.244472 27934 raft_consensus.cc:2272] T c0c33ea0ff53432c8be5e364242b23b3 P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:40.249490 29111 leader_election.cc:336] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.250039 29111 leader_election.cc:304] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.250944  5512 raft_consensus.cc:2749] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:07:40.251257 29111 leader_election.cc:336] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.256786 27934 tablet_replica.cc:333] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.257860 27934 raft_consensus.cc:2243] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.258749 27934 raft_consensus.cc:2272] T b09437a3ae8d46379ccdbb1829b14f27 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.261786 27934 tablet_replica.cc:333] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.272560 27934 raft_consensus.cc:2243] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.273540 27934 raft_consensus.cc:2272] T cb329a9e56064f1480f8f42e09ff6679 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.276523 27934 tablet_replica.cc:333] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.277289 27934 raft_consensus.cc:2243] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.278090 27934 raft_consensus.cc:2272] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 730186a8c70f43f2a57d80163bbc37d2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.251729 29111 leader_election.cc:304] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 730186a8c70f43f2a57d80163bbc37d2; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.278911  5512 raft_consensus.cc:2749] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:40.282021 27934 tablet_replica.cc:333] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.288748 27934 raft_consensus.cc:2243] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.289764 27934 raft_consensus.cc:2272] T 9ca58eb3f3b147bc941a1c21818d370c P 730186a8c70f43f2a57d80163bbc37d2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.293020 27934 tablet_replica.cc:333] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.293859 27934 raft_consensus.cc:2243] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:40.299208  5520 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:40.299840  5520 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:40.300436 27934 raft_consensus.cc:2272] T e96bceabcbf747e0acec327bc1abc729 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.312778 27934 tablet_replica.cc:333] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.313687 27934 raft_consensus.cc:2243] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:07:40.314954 27934 raft_consensus.cc:2272] T 40b726ca47da439b9be7ad6d531144fe P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.315333  5520 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
I20260709 13:07:40.317716 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:40.320037 29188 leader_election.cc:336] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:40.320482 29188 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:40.322213  5520 raft_consensus.cc:2749] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:07:40.324136 29185 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:40.324903 27934 tablet_replica.cc:333] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.325752 27934 raft_consensus.cc:2243] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.326489 27934 raft_consensus.cc:2272] T a9e4dbd865b648aaa1bce128c9de27af P 730186a8c70f43f2a57d80163bbc37d2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.329443 27934 tablet_replica.cc:333] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.330179 27934 raft_consensus.cc:2243] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.330900 27934 raft_consensus.cc:2272] T d8373166c3664170b540f729a4e3c93f P 730186a8c70f43f2a57d80163bbc37d2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.332522  5509 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:40.333254  5509 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:40.343706 27934 tablet_replica.cc:333] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.344651 27934 raft_consensus.cc:2243] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.345355 27934 raft_consensus.cc:2272] T e68315e143764938be60ec716705c15f P 730186a8c70f43f2a57d80163bbc37d2 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.345448  5505 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:40.351375 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:40.352233 29217 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 26.
I20260709 13:07:40.351719  5505 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:40.354494  5505 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:40.355964 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:40.357131 27934 tablet_replica.cc:333] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.357940  5515 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:07:40.358443  5515 raft_consensus.cc:493] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:40.358939  5515 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:40.358057 27934 raft_consensus.cc:2243] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.362524 27934 raft_consensus.cc:2272] T cdb0c0ac78fc4de7a353a995edd44346 P 730186a8c70f43f2a57d80163bbc37d2 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.365500 27934 tablet_replica.cc:333] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.366261 27934 raft_consensus.cc:2243] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.366918 27934 raft_consensus.cc:2272] T 2e40d8d1f334428db02ee54c8467d058 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.368927  5509 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:40.369845 27934 tablet_replica.cc:333] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.370642 27934 raft_consensus.cc:2243] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:40.376466 27934 raft_consensus.cc:2272] T 44d88fbf78394409bcacfadb40f377d6 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:40.383322 29871 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.406251  5515 raft_consensus.cc:515] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:40.408612 29874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:45851, user_credentials={real_user=slave}} blocked reactor thread for 44751.2us
I20260709 13:07:40.427417 27934 tablet_replica.cc:333] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.428295 27934 raft_consensus.cc:2243] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.428966 27934 raft_consensus.cc:2272] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 730186a8c70f43f2a57d80163bbc37d2 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:40.431492 29871 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:40.434808 29875 leader_election.cc:336] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:40.435794 29875 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:40.436183 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.437050  5522 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:07:40.442466 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0107c7a468e44f6b9ebd1ced2ae1ed1" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
W20260709 13:07:40.442656 29875 leader_election.cc:336] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:40.443073 29217 raft_consensus.cc:3060] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:07:40.446225  5420 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:40.446738  5420 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:40.448676  5420 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:40.450372 29217 raft_consensus.cc:2468] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 27.
I20260709 13:07:40.452510  5519 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:40.452668 29874 leader_election.cc:304] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:40.453086  5519 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:40.455225  5523 raft_consensus.cc:2804] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:07:40.455760  5523 raft_consensus.cc:697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 27 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
I20260709 13:07:40.455243  5519 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:40.457280  5523 consensus_queue.cc:237] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:40.459944 29188 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:40.463418 29185 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.463884 29185 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.466952  5420 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:07:40.481729 29188 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:07:40.484182 29185 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.484635 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.485327  5519 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:40.488273 27934 tablet_replica.cc:333] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.489524 27934 raft_consensus.cc:2243] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:40.490108 29875 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 47357us
I20260709 13:07:40.490790 27934 raft_consensus.cc:2272] T a385068bbb144e6fa0e8e9974398b1b9 P 730186a8c70f43f2a57d80163bbc37d2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:40.494282  5515 leader_election.cc:290] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 27 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:40.496171 28970 catalog_manager.cc:5697] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 26 to 27, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 27 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:40.524860 27934 tablet_replica.cc:333] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2: stopping tablet replica
I20260709 13:07:40.525835 27934 raft_consensus.cc:2243] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:40.526582 27934 raft_consensus.cc:2272] T be73fc18460b4609ad4da2946e3acb87 P 730186a8c70f43f2a57d80163bbc37d2 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:40.636885 29185 consensus_peers.cc:597] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:40.809432  5519 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:40.810056  5519 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:40.815619  5420 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:40.816146  5420 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:40.834172  5420 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
I20260709 13:07:40.836735  5519 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132:36041)
W20260709 13:07:40.842031 29188 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:40.848371 29188 leader_election.cc:336] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:40.858484 29906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "7e78ffbcc98242068a874a5d9e341477" is_pre_election: true
W20260709 13:07:40.864195 29185 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.864732 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.865659  5519 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:40.890416 29186 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, 7e78ffbcc98242068a874a5d9e341477
I20260709 13:07:40.891270  5420 raft_consensus.cc:2749] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:07:40.981204  5519 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:40.981781  5519 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:40.984110  5519 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:40.990352 29871 consensus_peers.cc:597] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:40.994539 29188 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:40.995680 29185 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:40.996155 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:40.998131  5420 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:41.009889 29875 consensus_peers.cc:597] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:41.114833 29217 raft_consensus.cc:1275] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:07:41.119740  5523 consensus_queue.cc:1048] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 13:07:41.124567 29871 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:41.180555 29875 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:41.227694  5519 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:41.228894  5519 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:41.233217  5519 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:41.242057  5537 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:41.243551  5537 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:41.245872  5537 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:41.257301  5522 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:07:41.260725 29188 consensus_peers.cc:597] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:41.260890  5505 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:41.261929  5505 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:41.269485 29875 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.275406 29188 leader_election.cc:336] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:41.276305 29188 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:41.258112  5522 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:41.287168  5505 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:41.288448  5522 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:41.289857 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" is_pre_election: true
I20260709 13:07:41.291240 29217 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 28.
W20260709 13:07:41.292603 29185 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:41.293617 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:41.296465  5505 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:07:41.297606  5520 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:41.298434  5520 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:41.302098  5520 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:41.304899 29188 consensus_peers.cc:597] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.306955 29185 leader_election.cc:336] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:41.307516 29185 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:41.308759 29188 leader_election.cc:336] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:41.309199  5535 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:07:41.313633 29185 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:41.314193 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:41.304195  5505 raft_consensus.cc:493] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:41.316412  5505 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:07:41.316992  5535 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:07:41.317864 29185 consensus_peers.cc:597] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:41.322330  5540 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:41.323249  5540 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:41.325651  5540 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:41.335135 29188 consensus_peers.cc:597] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.341470 29871 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:41.342402 29185 leader_election.cc:336] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:41.342573 29188 consensus_peers.cc:597] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:41.342880 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:41.343739  5537 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:07:41.346905 29185 consensus_peers.cc:597] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:41.348156 29875 leader_election.cc:336] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:41.353595 29871 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:41.356247 29875 leader_election.cc:336] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:41.361495 29871 leader_election.cc:336] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:41.361984 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:41.362851  5540 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:41.363858 29871 leader_election.cc:336] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:41.370217 29875 leader_election.cc:336] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:41.370841 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:41.371704  5540 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:41.374902  5505 raft_consensus.cc:515] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:41.378450 29217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaa941bea174ae08dc8aad56e0425f2" candidate_uuid: "7e78ffbcc98242068a874a5d9e341477" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6"
I20260709 13:07:41.379109 29217 raft_consensus.cc:3060] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:07:41.383780 29188 consensus_peers.cc:597] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:41.389074  5505 leader_election.cc:290] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 election: Requested vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:41.389377 29217 raft_consensus.cc:2468] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e78ffbcc98242068a874a5d9e341477 in term 29.
I20260709 13:07:41.390781 29874 leader_election.cc:304] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 7e78ffbcc98242068a874a5d9e341477; no voters: 
I20260709 13:07:41.394384  5540 raft_consensus.cc:2804] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:07:41.395114  5540 raft_consensus.cc:697] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 LEADER]: Becoming Leader. State: Replica: 7e78ffbcc98242068a874a5d9e341477, State: Running, Role: LEADER
W20260709 13:07:41.396189 29875 leader_election.cc:336] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:41.396522  5540 consensus_queue.cc:237] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:41.432615 29188 consensus_peers.cc:597] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:41.435263  5535 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:41.437486  5535 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:41.440029  5535 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:41.445828 29185 consensus_peers.cc:597] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:41.449157 29188 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:41.456348 29185 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:41.456967 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:41.461402  5535 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:07:41.482774 29188 consensus_peers.cc:597] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.491012 29875 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:41.502997  5537 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:41.503859  5537 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:41.506599  5537 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:41.511654 28971 catalog_manager.cc:5697] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 reported cstate change: term changed from 28 to 29, leader changed from 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130) to 7e78ffbcc98242068a874a5d9e341477 (127.27.71.132). New cstate: current_term: 29 leader_uuid: "7e78ffbcc98242068a874a5d9e341477" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.522545  5535 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:41.523730  5535 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:41.526046  5535 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:41.534778 29185 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:41.541356 29188 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:41.544497 29875 consensus_peers.cc:597] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.570228 29188 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.570657 29875 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.572408 29185 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:41.573103 29185 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:41.574275 29185 consensus_peers.cc:597] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:41.574965  5535 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:07:41.570930 29188 leader_election.cc:336] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:41.583310 29185 leader_election.cc:336] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:41.585822 29188 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.590407 29188 consensus_peers.cc:597] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.591004 29188 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.591439 29188 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.593403 29188 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:41.595429 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:41.596306  5537 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:41.599000 29185 consensus_peers.cc:597] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:41.599218 29875 consensus_peers.cc:597] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.626286 29185 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111) [suppressed 644 similar messages]
W20260709 13:07:41.638610 29188 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.646095 29875 consensus_peers.cc:597] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.658638 29871 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:41.672963 29185 consensus_peers.cc:597] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:41.691834 29871 consensus_peers.cc:597] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:41.691883 29188 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.756057 29188 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.756721 29875 consensus_peers.cc:597] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.773797 29875 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.775084 29188 consensus_peers.cc:597] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.793676 29188 consensus_peers.cc:597] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.820941 29185 consensus_peers.cc:597] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:41.825109 29188 consensus_peers.cc:597] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:41.833036 29221 raft_consensus.cc:1275] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Refusing update from remote peer 7e78ffbcc98242068a874a5d9e341477: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 13:07:41.835162  5540 consensus_queue.cc:1048] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:07:41.842429 29875 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.854244 27934 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 29169)
W20260709 13:07:41.884541 29188 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.935370 29185 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:41.941277 29185 consensus_peers.cc:597] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:41.971063 29188 consensus_peers.cc:597] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.982300 29875 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:41.982930 29875 consensus_peers.cc:597] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:41.982946  5537 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:41.990489  5537 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:41.993557  5537 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:41.996665 29875 consensus_peers.cc:597] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:42.006482 29185 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:42.007179 29185 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:42.013057 29188 consensus_peers.cc:597] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:42.028096 29871 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:42.042881 29871 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:42.043627 29871 consensus_peers.cc:597] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:42.044238 29188 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:42.044696 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:42.045537  5548 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:07:42.267064  5553 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:42.267683  5553 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:42.271241  5553 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:42.281697 29871 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:42.282703 29875 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:42.283124 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:42.284413  5553 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:07:42.289945 29185 consensus_peers.cc:597] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:42.319458 29185 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:42.358007 29185 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:42.380792 29871 consensus_peers.cc:597] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:42.401070 29185 consensus_peers.cc:597] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:42.404734 29871 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:42.409565 29185 consensus_peers.cc:597] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:42.507597  5537 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:42.508388  5537 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:42.516072  5537 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:42.522053 29188 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:42.523528 29185 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:42.522765 29871 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:42.524143 29185 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:42.526003  5520 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:42.528463 29185 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:42.533174  5520 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:42.533828  5520 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:42.578300 29188 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:42.581564 29185 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:42.582170 29185 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:42.583253  5537 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:07:42.590294  5520 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
W20260709 13:07:42.595162 29871 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:42.605154 28955 catalog_manager.cc:4755] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c0c33ea0ff53432c8be5e364242b23b3: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:42.798797 29185 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:42.831729  5520 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:42.832355  5520 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:42.834729  5520 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:42.840581 29185 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:42.841287 29185 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:42.850575 29188 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:42.851155 29188 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:42.854792 27934 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 29169)
W20260709 13:07:42.868570 29871 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:42.869194  5549 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:42.879024  5549 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:42.881735  5549 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:42.884121  5537 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:07:42.896099 29871 leader_election.cc:336] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:42.872687  5520 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:07:42.900375 29875 leader_election.cc:336] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:42.901803 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:42.900906  5520 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:42.904691  5520 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:42.905220  5549 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:42.917743 29185 leader_election.cc:336] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:42.951037 29188 leader_election.cc:336] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:42.951660 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:42.952493  5520 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:07:43.088377 29185 consensus_peers.cc:597] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:43.170780  5520 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:43.171489  5520 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:43.176218  5520 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:43.182294 29188 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:43.187250 29185 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:43.187767 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:43.188512  5548 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:43.213424  5548 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:43.213990  5548 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:43.217038  5548 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:43.224079 29188 leader_election.cc:336] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:43.234504  5556 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:43.235185  5556 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:43.239499 29185 leader_election.cc:336] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:43.237531  5556 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:43.241381  5548 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:43.241671 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:43.242094  5548 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:43.242509  5520 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:43.249578 29871 leader_election.cc:336] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:43.250667 29875 leader_election.cc:336] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:43.251291 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:43.252876  5548 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:43.253777  5556 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:07:43.258561 29188 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:43.397763  5557 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:43.398433  5557 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:43.402159  5557 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:43.411160 29188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:35337, user_credentials={real_user=slave}} blocked reactor thread for 141818us
W20260709 13:07:43.425269 29188 leader_election.cc:336] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:43.464728 29185 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:43.465296 29185 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:43.466117  5558 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:43.466748 29185 leader_election.cc:336] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:43.468060 29185 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:43.469854  5548 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:07:43.518110 29871 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:43.525195 29875 consensus_peers.cc:597] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:43.543107 29871 consensus_peers.cc:597] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:43.601264 29875 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:43.657337 29188 consensus_peers.cc:597] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:43.663698 29871 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:43.711378 29185 consensus_peers.cc:597] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:43.725165  5559 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:07:43.725776 29188 consensus_peers.cc:597] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:43.725845  5559 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:43.729609  5559 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:43.736668 29185 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:43.740989 29188 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:43.747869 29188 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:43.748389 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:43.749106  5559 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:07:43.770073 29185 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:43.873852 27934 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 29169)
W20260709 13:07:43.887200 29188 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:43.903738 29188 consensus_peers.cc:597] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:43.931571 29185 consensus_peers.cc:597] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:43.957676 29185 consensus_peers.cc:597] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:43.966856 29188 consensus_peers.cc:597] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:43.976943 29875 consensus_peers.cc:597] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:43.989636 29875 consensus_peers.cc:597] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:43.993933 29188 consensus_peers.cc:597] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.003342 29871 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:44.044152 29875 consensus_peers.cc:597] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.058954 29188 consensus_peers.cc:597] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.059618 29188 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.067579 29875 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.071166 29185 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.096776 29875 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.105877 29188 consensus_peers.cc:597] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.106451 29188 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.128404 29875 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.136206 29188 consensus_peers.cc:597] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.147008 29188 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.153475 29185 consensus_peers.cc:597] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.158272 29875 consensus_peers.cc:597] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.205169 29871 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.206321 29875 consensus_peers.cc:597] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.228490 29871 consensus_peers.cc:597] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:07:44.291296 29185 consensus_peers.cc:597] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.296658 29871 consensus_peers.cc:597] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.299183 29185 consensus_peers.cc:597] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.299793 29875 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.300266 29188 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:44.300927  5561 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:44.302570  5561 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:44.304948  5561 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:44.314193 29871 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:44.321924 29875 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:44.322928 29875 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:44.322489 29185 consensus_peers.cc:597] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:44.324339  5561 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:07:44.332432 29188 consensus_peers.cc:597] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.360750 29243 debug-util.cc:398] Leaking SignalData structure 0x7b08003aaf60 after lost signal to thread 27937
W20260709 13:07:44.362016 29243 debug-util.cc:398] Leaking SignalData structure 0x7b0800281540 after lost signal to thread 29005
W20260709 13:07:44.363454 29243 debug-util.cc:398] Leaking SignalData structure 0x7b0800408b40 after lost signal to thread 29246
W20260709 13:07:44.364745 29243 debug-util.cc:398] Leaking SignalData structure 0x7b0800306c20 after lost signal to thread 29935
W20260709 13:07:44.373109 29188 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.388691 29185 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.399574 29188 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.400379 29875 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.449429 29875 consensus_peers.cc:597] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:44.469769 29188 consensus_peers.cc:597] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.470341 29188 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:44.511720  5561 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:44.512336  5561 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:44.552124 29871 leader_election.cc:336] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:44.555136 29185 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.563675 29188 consensus_peers.cc:597] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.567763 29188 consensus_peers.cc:597] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:44.571498  5561 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:44.576766 29185 consensus_peers.cc:597] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.619680 29875 leader_election.cc:336] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:44.621498 29875 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:44.630724  5561 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:44.634729 29871 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.637567 29188 consensus_peers.cc:597] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:44.731014 29185 consensus_peers.cc:597] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.731539 29185 consensus_peers.cc:597] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:44.737385  5557 raft_consensus.cc:493] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:44.738061  5557 raft_consensus.cc:515] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:44.742959  5557 leader_election.cc:290] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:44.745425 29185 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:44.745662 29871 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:44.762404 29188 leader_election.cc:336] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:44.771986 29188 leader_election.cc:304] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:44.784114  5557 raft_consensus.cc:2749] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:07:44.823092 29875 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:44.836133  5557 raft_consensus.cc:493] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:44.836755  5557 raft_consensus.cc:515] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:44.838943  5557 leader_election.cc:290] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:44.843679 29185 consensus_peers.cc:597] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.847973 29185 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:44.863173 29188 leader_election.cc:336] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:44.863751 29188 leader_election.cc:304] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:44.864622  5557 raft_consensus.cc:2749] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:07:44.888827 29185 consensus_peers.cc:597] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:44.923444  5557 raft_consensus.cc:493] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:07:44.927946 29871 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:44.924608  5557 raft_consensus.cc:515] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
W20260709 13:07:44.941581 29871 consensus_peers.cc:597] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:44.938298  5557 leader_election.cc:290] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:44.957525 29185 leader_election.cc:336] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:44.959188 29188 leader_election.cc:336] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:44.963104 29188 leader_election.cc:304] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:44.964093  5557 raft_consensus.cc:2749] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:07:44.968542 29185 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:44.970927 29871 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:45.021624 29871 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:45.093362  5557 raft_consensus.cc:493] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:45.093878  5557 raft_consensus.cc:515] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:45.100888  5557 leader_election.cc:290] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:45.103106 29188 leader_election.cc:336] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:45.107034 29185 leader_election.cc:336] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:45.107874 29185 leader_election.cc:304] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:45.109163  5557 raft_consensus.cc:2749] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:45.159441 29185 consensus_peers.cc:597] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:07:45.193845  5564 raft_consensus.cc:493] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:45.194492  5564 raft_consensus.cc:515] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:45.198266  5564 leader_election.cc:290] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:45.204221 29871 leader_election.cc:336] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:45.205317 29875 leader_election.cc:336] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:45.209062 29875 leader_election.cc:304] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:45.218344  5560 raft_consensus.cc:2749] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:07:45.237095 29185 consensus_peers.cc:597] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:45.251158  5559 raft_consensus.cc:493] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:45.252038  5559 raft_consensus.cc:515] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:45.257872  5559 leader_election.cc:290] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:45.271893 29188 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:45.280098 29185 leader_election.cc:336] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:45.280707 29185 leader_election.cc:304] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:45.287009  5559 raft_consensus.cc:2749] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:07:45.314770 29871 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:45.357594  5559 raft_consensus.cc:493] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:45.358229  5559 raft_consensus.cc:515] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:45.377949  5559 leader_election.cc:290] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:45.384220 29185 leader_election.cc:336] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:45.490779 29185 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:45.560436 29188 leader_election.cc:336] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:45.561106 29188 leader_election.cc:304] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:45.564332  5559 raft_consensus.cc:2749] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:07:45.606518 29185 consensus_peers.cc:597] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:45.634064 29875 consensus_peers.cc:597] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:45.669391 27934 tablet_server.cc:196] TabletServer@127.27.71.130:0 shutdown complete.
I20260709 13:07:45.669631  5565 raft_consensus.cc:493] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:45.670348  5565 raft_consensus.cc:515] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:45.672976  5565 leader_election.cc:290] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:45.680971 29188 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:45.688275 29185 leader_election.cc:336] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:45.689774 29185 leader_election.cc:304] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:45.691579  5565 raft_consensus.cc:2749] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:07:45.880646 27934 tablet_server.cc:179] TabletServer@127.27.71.131:0 shutting down...
W20260709 13:07:45.903712 29874 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:45.914037  5559 raft_consensus.cc:493] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:45.914764  5559 raft_consensus.cc:515] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:45.917488  5559 leader_election.cc:290] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631)
W20260709 13:07:45.923727 29185 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:45.928536 29188 leader_election.cc:336] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:45.929030 29188 leader_election.cc:304] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:45.930375  5565 raft_consensus.cc:2749] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:07:45.945477 29871 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:45.970305 29874 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:45.973737 29874 consensus_peers.cc:597] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:46.004199 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:46.007419 27934 tablet_replica.cc:333] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.008358 27934 raft_consensus.cc:2243] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.009135 27934 raft_consensus.cc:2272] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.016618 27934 tablet_replica.cc:333] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.017611 27934 raft_consensus.cc:2243] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:46.017925 29874 consensus_peers.cc:597] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:46.018460 27934 raft_consensus.cc:2272] T d7517b9e93904801886467474f000b2d P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.021633 27934 tablet_replica.cc:333] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.030988  5565 raft_consensus.cc:493] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:46.031625  5565 raft_consensus.cc:515] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:46.032603 27934 raft_consensus.cc:2243] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.034006 27934 raft_consensus.cc:2272] T 9a29f89bc1cb471d94fee8d14f56314c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.034162  5565 leader_election.cc:290] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
I20260709 13:07:46.037101 27934 tablet_replica.cc:333] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.038103 27934 raft_consensus.cc:2243] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.039321 27934 raft_consensus.cc:2272] T 023c90d48e094012bc60b1e600fffa14 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:46.044936 29874 consensus_peers.cc:597] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:46.049593 29874 consensus_peers.cc:597] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:46.053947 27934 tablet_replica.cc:333] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.054958 27934 raft_consensus.cc:2243] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.056293 27934 raft_consensus.cc:2272] T b55b583a63b64ca6832b73f813985e49 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.059373 27934 tablet_replica.cc:333] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.060322 27934 raft_consensus.cc:2243] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:07:46.061497 29185 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:46.067866 29188 leader_election.cc:336] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:46.068434 29188 leader_election.cc:304] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:46.069486 29875 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:46.061641 27934 raft_consensus.cc:2272] T ee1e380ba6084a0e9d9978fa9983edc1 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.074086  5565 raft_consensus.cc:2749] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:07:46.082297 27934 tablet_replica.cc:333] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.083510 27934 raft_consensus.cc:2243] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.085034 27934 raft_consensus.cc:2272] T 5acfd220cbb54cd181c08050de276f65 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.096099 27934 tablet_replica.cc:333] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.096884  5559 raft_consensus.cc:493] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
W20260709 13:07:46.097187 29871 consensus_peers.cc:597] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:46.097591  5559 raft_consensus.cc:515] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } }
I20260709 13:07:46.099788  5559 leader_election.cc:290] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:46.100374 29188 consensus_peers.cc:597] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.103134 29874 consensus_peers.cc:597] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:46.108206 29185 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:46.108888 29185 consensus_peers.cc:597] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:46.111470 29188 leader_election.cc:336] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:46.117928 29188 leader_election.cc:304] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ad9faa1c14444b29c9db8e71f46e2d6; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:46.110008 27934 raft_consensus.cc:2243] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.118955  5557 raft_consensus.cc:2749] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:07:46.120124 27934 raft_consensus.cc:2272] T ea48540c1af24bf3b55edec5c52ec133 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:46.122041 29188 consensus_peers.cc:597] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.149762 29188 consensus_peers.cc:597] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.160739 29871 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:07:46.166061 29874 consensus_peers.cc:597] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:46.172421 29874 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:46.163542 27934 tablet_replica.cc:333] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.174031 27934 raft_consensus.cc:2243] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.175165 27934 raft_consensus.cc:2272] T d5d7d7e7e55b46f89b835614737ee91b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:46.188306 29874 consensus_peers.cc:597] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:46.193744 27934 tablet_replica.cc:333] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.194811 27934 raft_consensus.cc:2243] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.196544 27934 raft_consensus.cc:2272] T 01d4b885cd86484fbaa6c528fc2ed1bc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.200556 27934 tablet_replica.cc:333] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.201839 27934 raft_consensus.cc:2243] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.202702 27934 raft_consensus.cc:2272] T 9e3e05d4f95940f5a4cac560d26d0ec8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.205974 27934 tablet_replica.cc:333] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.206858 27934 raft_consensus.cc:2243] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.207607 27934 raft_consensus.cc:2272] T 3842b36ffd134cd28c657475df48e741 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.210702 27934 tablet_replica.cc:333] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.211887 27934 raft_consensus.cc:2243] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.213361 27934 raft_consensus.cc:2272] T c24ba590bc46427dbac6cd4a4da67ee8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.224962 27934 tablet_replica.cc:333] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.225878 27934 raft_consensus.cc:2243] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.227852 27934 raft_consensus.cc:2272] T 79cdfc6a725145348b70034422ef477b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.231067 27934 tablet_replica.cc:333] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.232035 27934 raft_consensus.cc:2243] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.232832 27934 raft_consensus.cc:2272] T 7f1cace028c94f79adfce2be604fb35a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.236105 27934 tablet_replica.cc:333] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.237010 27934 raft_consensus.cc:2243] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.238385 27934 raft_consensus.cc:2272] T c54509bf16d642138312814fefa3f6dc P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.241529 27934 tablet_replica.cc:333] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.242379 27934 raft_consensus.cc:2243] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.243809 27934 raft_consensus.cc:2272] T 082c896dd04e451198eba8392e70fba7 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.246798 27934 tablet_replica.cc:333] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.247686 27934 raft_consensus.cc:2243] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.249125 27934 raft_consensus.cc:2272] T 128a7406f55d4400bfd5fadcbc14d065 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.252184 27934 tablet_replica.cc:333] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.253090 27934 raft_consensus.cc:2243] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.253918 27934 raft_consensus.cc:2272] T 12e21412fcd24f95a8e3635fc5bbe138 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.257061 27934 tablet_replica.cc:333] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.258030 27934 raft_consensus.cc:2243] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.258808 27934 raft_consensus.cc:2272] T a9f522a55d4b4051a75e0bf87cd2619e P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:46.271869 29874 consensus_peers.cc:597] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:46.276764 29875 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:46.282768 27934 tablet_replica.cc:333] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.283932 27934 raft_consensus.cc:2243] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.285490 27934 raft_consensus.cc:2272] T e9334a7c40c744cbbff4dec0358f721b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:46.320375 29874 consensus_peers.cc:597] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:46.321161 29874 consensus_peers.cc:597] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:46.349639 29185 consensus_peers.cc:597] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:46.368506 29188 consensus_peers.cc:597] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.384716 29874 consensus_peers.cc:597] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:46.393805 27934 tablet_replica.cc:333] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.394860 27934 raft_consensus.cc:2243] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.395637 27934 raft_consensus.cc:2272] T b27f1b9ed9964227ac908c2897d935e9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:46.413960 29188 consensus_peers.cc:597] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:46.422621 27934 tablet_replica.cc:333] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.423753 27934 raft_consensus.cc:2243] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.424515 27934 raft_consensus.cc:2272] T 3aaa941bea174ae08dc8aad56e0425f2 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.432569 27934 tablet_replica.cc:333] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
W20260709 13:07:46.439891 29188 consensus_peers.cc:597] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:46.441293 27934 raft_consensus.cc:2243] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:07:46.442088 29875 consensus_peers.cc:597] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:46.443198 27934 raft_consensus.cc:2272] T 43a52106dbee45398d36a633f281eef6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:46.443393 29871 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:07:46.451035 27934 tablet_replica.cc:333] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.452255 27934 raft_consensus.cc:2243] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.453047 27934 raft_consensus.cc:2272] T 348ae3050d97486fa402a5da2bbdce46 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.458432 27934 tablet_replica.cc:333] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.459215 27934 raft_consensus.cc:2243] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.460521 27934 raft_consensus.cc:2272] T e8195256ce88462cab8bb734ddbb65ad P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:46.478166 29185 consensus_peers.cc:597] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:46.478961 27934 tablet_replica.cc:333] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.479813 27934 raft_consensus.cc:2243] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.481335 27934 raft_consensus.cc:2272] T c59eab5922dc4c07a3a86e915e3cab7f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.484190 27934 tablet_replica.cc:333] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.484951 27934 raft_consensus.cc:2243] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.486188 27934 raft_consensus.cc:2272] T cddea8e9cff6495ba269c65b0edf8b6c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.488812 27934 tablet_replica.cc:333] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.489537 27934 raft_consensus.cc:2243] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.490336 27934 raft_consensus.cc:2272] T 813e480732fd4061837e784d4d768b1f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.493023 27934 tablet_replica.cc:333] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.493741 27934 raft_consensus.cc:2243] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.494994 27934 raft_consensus.cc:2272] T b09437a3ae8d46379ccdbb1829b14f27 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.497646 27934 tablet_replica.cc:333] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.498358 27934 raft_consensus.cc:2243] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.500007 27934 raft_consensus.cc:2272] T cb329a9e56064f1480f8f42e09ff6679 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.502614 27934 tablet_replica.cc:333] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.503332 27934 raft_consensus.cc:2243] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.504874 27934 raft_consensus.cc:2272] T 2c02e27bf25948329604215b11034f44 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.507521 27934 tablet_replica.cc:333] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.508507 27934 raft_consensus.cc:2243] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.509680 27934 raft_consensus.cc:2272] T c0c33ea0ff53432c8be5e364242b23b3 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.512214 27934 tablet_replica.cc:333] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.512992 27934 raft_consensus.cc:2243] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.513561 27934 raft_consensus.cc:2272] T 344aa5f73f0f4124b1512832a0bca2c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.517215 27934 tablet_replica.cc:333] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.517954 27934 raft_consensus.cc:2243] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:46.520220 29932 debug-util.cc:398] Leaking SignalData structure 0x7b080019cf00 after lost signal to thread 27937
W20260709 13:07:46.521311 29932 debug-util.cc:398] Leaking SignalData structure 0x7b08002e4ac0 after lost signal to thread 29005
I20260709 13:07:46.518546 27934 raft_consensus.cc:2272] T 77b0625d42ef413aae4ec112d9ba450b P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:46.522670 29932 debug-util.cc:398] Leaking SignalData structure 0x7b08002e4ce0 after lost signal to thread 29246
W20260709 13:07:46.523841 29932 debug-util.cc:398] Leaking SignalData structure 0x7b080018d3c0 after lost signal to thread 29935
W20260709 13:07:46.525161 29188 consensus_peers.cc:597] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:46.526947 27934 tablet_replica.cc:333] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.527765 27934 raft_consensus.cc:2243] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.533304 27934 raft_consensus.cc:2272] T 1fdfda5dcdd94083afe1620564f9d376 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.537874 27934 tablet_replica.cc:333] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.538733 27934 raft_consensus.cc:2243] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.540024 27934 raft_consensus.cc:2272] T ac412099b9f64d4e8bcceebe1c7ffe87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.554582 27934 tablet_replica.cc:333] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.555562 27934 raft_consensus.cc:2243] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.556344 27934 raft_consensus.cc:2272] T cc31aa85851e41609957a765b2d40774 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.559484 27934 tablet_replica.cc:333] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.560786 27934 raft_consensus.cc:2243] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.561695 27934 raft_consensus.cc:2272] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.564538 27934 tablet_replica.cc:333] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
W20260709 13:07:46.569017 29875 consensus_peers.cc:597] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:46.570058 27934 raft_consensus.cc:2243] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.571866 27934 raft_consensus.cc:2272] T 2e40d8d1f334428db02ee54c8467d058 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.574900 27934 tablet_replica.cc:333] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.575666 27934 raft_consensus.cc:2243] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:46.576113 29875 consensus_peers.cc:597] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:46.576423 27934 raft_consensus.cc:2272] T 44d88fbf78394409bcacfadb40f377d6 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.579629 27934 tablet_replica.cc:333] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.580860 27934 raft_consensus.cc:2243] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.582115 27934 raft_consensus.cc:2272] T cdb0c0ac78fc4de7a353a995edd44346 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.584980 27934 tablet_replica.cc:333] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.585726 27934 raft_consensus.cc:2243] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.586890 27934 raft_consensus.cc:2272] T 9ca58eb3f3b147bc941a1c21818d370c P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.589758 27934 tablet_replica.cc:333] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.590533 27934 raft_consensus.cc:2243] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.591822 27934 raft_consensus.cc:2272] T 2fae9bf99a434b83a73451cee1c56e6a P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.594559 27934 tablet_replica.cc:333] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
W20260709 13:07:46.594683 29188 consensus_peers.cc:597] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:46.595222 27934 raft_consensus.cc:2243] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.596556 27934 raft_consensus.cc:2272] T d506d5cb9c9943c49f64dbb8fa09e5c9 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.604084 27934 tablet_replica.cc:333] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
W20260709 13:07:46.604552 29871 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:07:46.604846 27934 raft_consensus.cc:2243] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:07:46.606225 27934 raft_consensus.cc:2272] T d8373166c3664170b540f729a4e3c93f P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.608923 27934 tablet_replica.cc:333] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.609800 27934 raft_consensus.cc:2243] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.610495 27934 raft_consensus.cc:2272] T 40b726ca47da439b9be7ad6d531144fe P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.612965 27934 tablet_replica.cc:333] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.613603 27934 raft_consensus.cc:2243] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.614254 27934 raft_consensus.cc:2272] T a9e4dbd865b648aaa1bce128c9de27af P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:46.616981 27934 tablet_replica.cc:333] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6: stopping tablet replica
I20260709 13:07:46.617619 27934 raft_consensus.cc:2243] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:46.618423 27934 raft_consensus.cc:2272] T be73fc18460b4609ad4da2946e3acb87 P 6ad9faa1c14444b29c9db8e71f46e2d6 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:46.626789 29875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111) [suppressed 754 similar messages]
W20260709 13:07:46.652918 29875 consensus_peers.cc:597] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.653491 29875 consensus_peers.cc:597] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.668336 29875 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.766585 29874 connection.cc:769] client connection to 127.27.71.131:45851 send error: Network error: sendmsg error: Connection reset by peer (error 104)
W20260709 13:07:46.767011 29871 consensus_peers.cc:597] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:46.803530 29875 consensus_peers.cc:597] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:46.814164 29871 consensus_peers.cc:597] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:47.024060 29875 consensus_peers.cc:597] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:47.091099 29875 consensus_peers.cc:597] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.146032 29875 consensus_peers.cc:597] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.146651 29875 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:07:47.224906 29871 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:07:47.228324 29871 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:07:47.269032  5561 raft_consensus.cc:493] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:47.269563  5561 raft_consensus.cc:515] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:47.278693 29875 leader_election.cc:336] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
I20260709 13:07:47.280514  5561 leader_election.cc:290] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:47.283825  5561 raft_consensus.cc:493] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:47.284315  5561 raft_consensus.cc:515] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
W20260709 13:07:47.285238 29874 leader_election.cc:336] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111)
I20260709 13:07:47.285760 29874 leader_election.cc:304] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:47.286527  5564 raft_consensus.cc:2749] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:47.287177  5561 leader_election.cc:290] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:47.290903 29875 leader_election.cc:336] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:47.305320 29874 leader_election.cc:336] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111)
I20260709 13:07:47.305907 29874 leader_election.cc:304] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:47.306710  5561 raft_consensus.cc:2749] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:47.343858 29871 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:07:47.511799  5561 raft_consensus.cc:493] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:47.512429  5561 raft_consensus.cc:515] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:47.514554  5561 leader_election.cc:290] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
I20260709 13:07:47.515029  5561 raft_consensus.cc:493] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:47.515483  5561 raft_consensus.cc:515] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:47.520329  5561 leader_election.cc:290] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:47.528928 29874 leader_election.cc:336] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111)
W20260709 13:07:47.531286 29875 leader_election.cc:336] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:47.537420 29874 leader_election.cc:336] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111)
I20260709 13:07:47.537879 29874 leader_election.cc:304] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:47.538569  5561 raft_consensus.cc:2749] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:07:47.543277 29871 leader_election.cc:336] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:47.543768 29871 leader_election.cc:304] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:47.544652  5561 raft_consensus.cc:2749] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:47.560770 29871 consensus_peers.cc:597] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:07:47.656205 29871 consensus_peers.cc:597] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:07:47.702657  5561 raft_consensus.cc:493] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:47.703205  5561 raft_consensus.cc:515] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:47.706430  5561 leader_election.cc:290] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:47.710876 29871 leader_election.cc:336] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:47.712699  5564 raft_consensus.cc:493] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:47.718180  5564 raft_consensus.cc:515] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:47.720345  5564 leader_election.cc:290] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:47.723093 29874 leader_election.cc:336] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111)
I20260709 13:07:47.725968 29874 leader_election.cc:304] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:47.733400  5569 raft_consensus.cc:2749] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:07:47.729773  5561 raft_consensus.cc:493] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:47.734485  5561 raft_consensus.cc:515] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } attrs { replace: true } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:47.737551  5561 leader_election.cc:290] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:47.748369 29875 leader_election.cc:336] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:47.761745 29875 leader_election.cc:336] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:47.781262 29871 leader_election.cc:336] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:47.770354 29874 leader_election.cc:336] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111)
I20260709 13:07:47.783074 29874 leader_election.cc:304] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, 730186a8c70f43f2a57d80163bbc37d2
I20260709 13:07:47.782322 29871 leader_election.cc:304] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:47.789681  5561 raft_consensus.cc:2749] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:07:47.790470  5561 raft_consensus.cc:2749] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:07:47.791142 29874 leader_election.cc:336] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111)
I20260709 13:07:47.811702  5564 raft_consensus.cc:493] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6ad9faa1c14444b29c9db8e71f46e2d6)
I20260709 13:07:47.812332  5564 raft_consensus.cc:515] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "6ad9faa1c14444b29c9db8e71f46e2d6" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45851 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:47.814558  5564 leader_election.cc:290] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851)
W20260709 13:07:47.833813 29871 consensus_peers.cc:597] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:07:47.835155 29871 leader_election.cc:336] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
W20260709 13:07:47.853276 29874 leader_election.cc:336] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111)
I20260709 13:07:47.855504 29874 leader_election.cc:304] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 6ad9faa1c14444b29c9db8e71f46e2d6, a936878f59c840a6bd176a1d2d7347b1
W20260709 13:07:47.855047 29871 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:07:47.856779  5570 raft_consensus.cc:2749] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:07:48.109467  5570 raft_consensus.cc:493] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:48.110010  5570 raft_consensus.cc:515] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:48.112196  5570 leader_election.cc:290] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:48.121731 29875 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:48.122537 29871 leader_election.cc:336] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:48.122922 29871 leader_election.cc:304] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:48.124107  5570 raft_consensus.cc:2749] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:07:48.171495 29875 consensus_peers.cc:597] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:07:48.173346  5570 raft_consensus.cc:493] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 730186a8c70f43f2a57d80163bbc37d2)
I20260709 13:07:48.173839  5570 raft_consensus.cc:515] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a936878f59c840a6bd176a1d2d7347b1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41631 } } peers { permanent_uuid: "730186a8c70f43f2a57d80163bbc37d2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 35337 } } peers { permanent_uuid: "7e78ffbcc98242068a874a5d9e341477" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 36041 } attrs { promote: false } }
I20260709 13:07:48.176196  5570 leader_election.cc:290] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631), 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337)
W20260709 13:07:48.185626 29875 leader_election.cc:336] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111)
W20260709 13:07:48.190574 29871 leader_election.cc:336] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:48.192893 29871 leader_election.cc:304] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e78ffbcc98242068a874a5d9e341477; no voters: 730186a8c70f43f2a57d80163bbc37d2, a936878f59c840a6bd176a1d2d7347b1
I20260709 13:07:48.194651  5570 raft_consensus.cc:2749] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:07:48.217908 27934 tablet_server.cc:196] TabletServer@127.27.71.131:0 shutdown complete.
W20260709 13:07:48.219172 29874 consensus_peers.cc:597] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:48.335577 29874 consensus_peers.cc:597] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:48.357347 29874 consensus_peers.cc:597] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:48.394475 29874 consensus_peers.cc:597] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:48.395401 27934 tablet_server.cc:179] TabletServer@127.27.71.132:0 shutting down...
W20260709 13:07:48.409052 29874 consensus_peers.cc:597] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:48.464429 29871 consensus_peers.cc:597] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:07:48.481010 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:48.482228 27934 tablet_replica.cc:333] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.483459 27934 raft_consensus.cc:2243] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.484663 27934 raft_consensus.cc:2272] T c0c33ea0ff53432c8be5e364242b23b3 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.488365 27934 tablet_replica.cc:333] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.489343 27934 raft_consensus.cc:2243] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.490321 27934 raft_consensus.cc:2272] T cb329a9e56064f1480f8f42e09ff6679 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.494083 27934 tablet_replica.cc:333] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.495234 27934 raft_consensus.cc:2243] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.496668 27934 raft_consensus.cc:2272] T 348ae3050d97486fa402a5da2bbdce46 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.500245 27934 tablet_replica.cc:333] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.501389 27934 raft_consensus.cc:2243] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.503058 27934 raft_consensus.cc:2272] T e68315e143764938be60ec716705c15f P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.506639 27934 tablet_replica.cc:333] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.507678 27934 raft_consensus.cc:2243] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.508548 27934 raft_consensus.cc:2272] T 2c02e27bf25948329604215b11034f44 P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.511875 27934 tablet_replica.cc:333] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.512894 27934 raft_consensus.cc:2243] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.513691 27934 raft_consensus.cc:2272] T a57ab99b58a448de88023e7166f78ce2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.517366 27934 tablet_replica.cc:333] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.518467 27934 raft_consensus.cc:2243] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.519289 27934 raft_consensus.cc:2272] T 5acfd220cbb54cd181c08050de276f65 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.522810 27934 tablet_replica.cc:333] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.523952 27934 raft_consensus.cc:2243] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:48.533869 29874 consensus_peers.cc:597] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:48.536301 27934 raft_consensus.cc:2272] T 023c90d48e094012bc60b1e600fffa14 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.539736 27934 tablet_replica.cc:333] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.540755 27934 raft_consensus.cc:2243] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.542155 27934 raft_consensus.cc:2272] T 984bea8952114e3f9c0a045e7f557e37 P 7e78ffbcc98242068a874a5d9e341477 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.545199 27934 tablet_replica.cc:333] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.546105 27934 raft_consensus.cc:2243] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.546770 27934 raft_consensus.cc:2272] T 88cabbc6d2424a53a76857f415831c36 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:48.550134 29875 consensus_peers.cc:597] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 730186a8c70f43f2a57d80163bbc37d2 (127.27.71.130:35337): Couldn't send request to peer 730186a8c70f43f2a57d80163bbc37d2. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:35337: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:07:48.560091 27934 tablet_replica.cc:333] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
W20260709 13:07:48.561049 29874 consensus_peers.cc:597] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 -> Peer 6ad9faa1c14444b29c9db8e71f46e2d6 (127.27.71.131:45851): Couldn't send request to peer 6ad9faa1c14444b29c9db8e71f46e2d6. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.131:45851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:48.561074 27934 raft_consensus.cc:2243] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.562635 27934 raft_consensus.cc:2272] T 01d4b885cd86484fbaa6c528fc2ed1bc P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.565976 27934 tablet_replica.cc:333] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.567202 27934 raft_consensus.cc:2243] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.568512 27934 raft_consensus.cc:2272] T a9f522a55d4b4051a75e0bf87cd2619e P 7e78ffbcc98242068a874a5d9e341477 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.571735 27934 tablet_replica.cc:333] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.572675 27934 raft_consensus.cc:2243] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.574010 27934 raft_consensus.cc:2272] T d5d7d7e7e55b46f89b835614737ee91b P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.577107 27934 tablet_replica.cc:333] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.578071 27934 raft_consensus.cc:2243] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.579005 27934 raft_consensus.cc:2272] T 2fae9bf99a434b83a73451cee1c56e6a P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.582358 27934 tablet_replica.cc:333] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
W20260709 13:07:48.596477 29871 consensus_peers.cc:597] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 -> Peer a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): Couldn't send request to peer a936878f59c840a6bd176a1d2d7347b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:07:48.583431 27934 raft_consensus.cc:2243] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.614264 27934 raft_consensus.cc:2272] T 79cdfc6a725145348b70034422ef477b P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.617901 27934 tablet_replica.cc:333] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.626241 27934 raft_consensus.cc:2243] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.627782 27934 raft_consensus.cc:2272] T cc31aa85851e41609957a765b2d40774 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.631085 27934 tablet_replica.cc:333] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.632112 27934 raft_consensus.cc:2243] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.633559 27934 raft_consensus.cc:2272] T e250871b4ad74e079a538cdb04065b00 P 7e78ffbcc98242068a874a5d9e341477 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.636760 27934 tablet_replica.cc:333] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.637703 27934 raft_consensus.cc:2243] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.639034 27934 raft_consensus.cc:2272] T 344aa5f73f0f4124b1512832a0bca2c9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.642560 27934 tablet_replica.cc:333] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.646029 27934 raft_consensus.cc:2243] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.647315 27934 raft_consensus.cc:2272] T c54509bf16d642138312814fefa3f6dc P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.650350 27934 tablet_replica.cc:333] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.651263 27934 raft_consensus.cc:2243] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.652081 27934 raft_consensus.cc:2272] T 128a7406f55d4400bfd5fadcbc14d065 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.655280 27934 tablet_replica.cc:333] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.663231 27934 raft_consensus.cc:2243] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.664678 27934 raft_consensus.cc:2272] T a9e4dbd865b648aaa1bce128c9de27af P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.681531 27934 tablet_replica.cc:333] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.682518 27934 raft_consensus.cc:2243] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.683521 27934 raft_consensus.cc:2272] T e96bceabcbf747e0acec327bc1abc729 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.686509 27934 tablet_replica.cc:333] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.687374 27934 raft_consensus.cc:2243] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.688656 27934 raft_consensus.cc:2272] T d2ee8b4c1bf649b9b89ad6243f3890a8 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.691772 27934 tablet_replica.cc:333] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.692689 27934 raft_consensus.cc:2243] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.694128 27934 raft_consensus.cc:2272] T 3c777774468e477ca1ab0352dff56c9c P 7e78ffbcc98242068a874a5d9e341477 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.697319 27934 tablet_replica.cc:333] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.698215 27934 raft_consensus.cc:2243] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.699632 27934 raft_consensus.cc:2272] T b9574de46fcd490e84208e41907200b0 P 7e78ffbcc98242068a874a5d9e341477 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.710207 27934 tablet_replica.cc:333] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.711076 27934 raft_consensus.cc:2243] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.713670 27934 raft_consensus.cc:2272] T be73fc18460b4609ad4da2946e3acb87 P 7e78ffbcc98242068a874a5d9e341477 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.716804 27934 tablet_replica.cc:333] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.717656 27934 raft_consensus.cc:2243] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.719014 27934 raft_consensus.cc:2272] T 3aaa941bea174ae08dc8aad56e0425f2 P 7e78ffbcc98242068a874a5d9e341477 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.721899 27934 tablet_replica.cc:333] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.722730 27934 raft_consensus.cc:2243] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.724056 27934 raft_consensus.cc:2272] T 3842b36ffd134cd28c657475df48e741 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.734165 27934 tablet_replica.cc:333] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.735038 27934 raft_consensus.cc:2243] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.736266 27934 raft_consensus.cc:2272] T b27f1b9ed9964227ac908c2897d935e9 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.739362 27934 tablet_replica.cc:333] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.740185 27934 raft_consensus.cc:2243] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.741448 27934 raft_consensus.cc:2272] T a385068bbb144e6fa0e8e9974398b1b9 P 7e78ffbcc98242068a874a5d9e341477 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.744424 27934 tablet_replica.cc:333] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.745262 27934 raft_consensus.cc:2243] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.746640 27934 raft_consensus.cc:2272] T d0107c7a468e44f6b9ebd1ced2ae1ed1 P 7e78ffbcc98242068a874a5d9e341477 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.749799 27934 tablet_replica.cc:333] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477: stopping tablet replica
I20260709 13:07:48.750561 27934 raft_consensus.cc:2243] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:07:48.751824 27934 raft_consensus.cc:2272] T 8c8a2336e86c4846bf4e69ec61ba3fe0 P 7e78ffbcc98242068a874a5d9e341477 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.440696 27934 tablet_server.cc:196] TabletServer@127.27.71.132:0 shutdown complete.
I20260709 13:07:49.524971 27934 master.cc:562] Master@127.27.71.190:37537 shutting down...
W20260709 13:07:49.595124 29019 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:07:49.595523 29019 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:07:49.595744 29019 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:07:50.851514 28955 catalog_manager.cc:4755] TS a936878f59c840a6bd176a1d2d7347b1 (127.27.71.129:41631): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c0c33ea0ff53432c8be5e364242b23b3: Network error: Client connection negotiation failed: client connection to 127.27.71.129:41631: connect: Connection refused (error 111)
I20260709 13:07:51.102160 27934 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:51.102897 27934 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:51.103256 27934 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9c100c1aaf1b493089df815bded98302: stopping tablet replica
W20260709 13:07:51.734388 29002 debug-util.cc:398] Leaking SignalData structure 0x7b0800174820 after lost signal to thread 27937
W20260709 13:07:51.735080 29002 debug-util.cc:398] Leaking SignalData structure 0x7b08000394a0 after lost signal to thread 29005
I20260709 13:07:52.085599 27934 master.cc:584] Master@127.27.71.190:37537 shutdown complete.
I20260709 13:07:52.146989 27934 test_util.cc:182] -----------------------------------------------
I20260709 13:07:52.147220 27934 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602179482400-27934-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (268202 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:07:52.151263 27934 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.71.190:45203
I20260709 13:07:52.152547 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:52.159118  5574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.159711  5577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.160173  5575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:52.160749 27934 server_base.cc:1061] running on GCE node
I20260709 13:07:52.164834 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:52.165043 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:52.165233 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602472165218 us; error 0 us; skew 500 ppm
I20260709 13:07:52.165817 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:52.168583 27934 webserver.cc:533] Webserver started at http://127.27.71.190:34939/ using document root <none> and password file <none>
I20260709 13:07:52.169118 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:52.169304 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:52.169611 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:52.170951 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/master-0-root/instance:
uuid: "c783fb2b5e494764af12cd3d23090672"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-mlt7"
I20260709 13:07:52.176026 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:07:52.180317  5582 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.181166 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 13:07:52.181502 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/master-0-root
uuid: "c783fb2b5e494764af12cd3d23090672"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-mlt7"
I20260709 13:07:52.181821 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:52.201864 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:52.203203 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:52.246865 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.190:45203
I20260709 13:07:52.246964  5633 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.190:45203 every 8 connection(s)
I20260709 13:07:52.251379  5634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:52.265726  5634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672: Bootstrap starting.
I20260709 13:07:52.271106  5634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:52.275848  5634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672: No bootstrap required, opened a new log
I20260709 13:07:52.278558  5634 raft_consensus.cc:359] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c783fb2b5e494764af12cd3d23090672" member_type: VOTER }
I20260709 13:07:52.279150  5634 raft_consensus.cc:385] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:52.279407  5634 raft_consensus.cc:740] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c783fb2b5e494764af12cd3d23090672, State: Initialized, Role: FOLLOWER
I20260709 13:07:52.280112  5634 consensus_queue.cc:260] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c783fb2b5e494764af12cd3d23090672" member_type: VOTER }
I20260709 13:07:52.280653  5634 raft_consensus.cc:399] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:52.280913  5634 raft_consensus.cc:493] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:52.281226  5634 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:52.286795  5634 raft_consensus.cc:515] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c783fb2b5e494764af12cd3d23090672" member_type: VOTER }
I20260709 13:07:52.287410  5634 leader_election.cc:304] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c783fb2b5e494764af12cd3d23090672; no voters: 
I20260709 13:07:52.288733  5634 leader_election.cc:290] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:52.289211  5637 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:52.290685  5637 raft_consensus.cc:697] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [term 1 LEADER]: Becoming Leader. State: Replica: c783fb2b5e494764af12cd3d23090672, State: Running, Role: LEADER
I20260709 13:07:52.291381  5637 consensus_queue.cc:237] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c783fb2b5e494764af12cd3d23090672" member_type: VOTER }
I20260709 13:07:52.292029  5634 sys_catalog.cc:565] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:52.301797  5639 sys_catalog.cc:455] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c783fb2b5e494764af12cd3d23090672. Latest consensus state: current_term: 1 leader_uuid: "c783fb2b5e494764af12cd3d23090672" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c783fb2b5e494764af12cd3d23090672" member_type: VOTER } }
I20260709 13:07:52.302644  5639 sys_catalog.cc:458] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:52.301949  5638 sys_catalog.cc:455] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c783fb2b5e494764af12cd3d23090672" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c783fb2b5e494764af12cd3d23090672" member_type: VOTER } }
I20260709 13:07:52.303890  5638 sys_catalog.cc:458] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:52.304831  5647 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:52.310269  5647 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:52.311332 27934 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:52.319787  5647 catalog_manager.cc:1383] Generated new cluster ID: 6b4e9e6e2e8b4f8ca18df82efaff6a3a
I20260709 13:07:52.320161  5647 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:52.333865  5647 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:52.335358  5647 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:52.348171  5647 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672: Generated new TSK 0
I20260709 13:07:52.348883  5647 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:52.378186 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:52.384853  5655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.386780  5656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.387532  5658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:52.389289 27934 server_base.cc:1061] running on GCE node
I20260709 13:07:52.392427 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:52.392649 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:52.392840 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602472392825 us; error 0 us; skew 500 ppm
I20260709 13:07:52.393500 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:52.396090 27934 webserver.cc:533] Webserver started at http://127.27.71.129:46079/ using document root <none> and password file <none>
I20260709 13:07:52.396636 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:52.396838 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:52.397131 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:52.398341 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/instance:
uuid: "d9952df48cf245608db9181dedad0aeb"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-mlt7"
I20260709 13:07:52.403157 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:07:52.406802  5663 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.407614 27934 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:52.407979 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-0-root
uuid: "d9952df48cf245608db9181dedad0aeb"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-mlt7"
I20260709 13:07:52.408264 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:52.418686 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:52.420166 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:52.421922 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:52.424489 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:52.424685 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.425017 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:52.425174 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.470271 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.129:40419
I20260709 13:07:52.470438  5725 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.129:40419 every 8 connection(s)
I20260709 13:07:52.475821 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:52.485514  5730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.486063  5731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.489207  5733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:52.492188 27934 server_base.cc:1061] running on GCE node
I20260709 13:07:52.494239  5726 heartbeater.cc:344] Connected to a master server at 127.27.71.190:45203
I20260709 13:07:52.494746  5726 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:52.495874  5726 heartbeater.cc:507] Master 127.27.71.190:45203 requested a full tablet report, sending...
I20260709 13:07:52.496935 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:52.497182 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:52.497417 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602472497390 us; error 0 us; skew 500 ppm
I20260709 13:07:52.498288 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:52.498752  5599 ts_manager.cc:194] Registered new tserver with Master: d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:52.501421 27934 webserver.cc:533] Webserver started at http://127.27.71.130:43273/ using document root <none> and password file <none>
I20260709 13:07:52.501739  5599 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39338
I20260709 13:07:52.502126 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:52.502447 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:52.502789 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:52.504524 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/instance:
uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-mlt7"
I20260709 13:07:52.509297 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:52.513119  5738 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.513960 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:07:52.514242 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-1-root
uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-mlt7"
I20260709 13:07:52.514499 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:52.536509 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:52.537788 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:52.539250 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:52.541555 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:52.541735 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.542047 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:52.542207 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.587051 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.130:39429
I20260709 13:07:52.587170  5800 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.130:39429 every 8 connection(s)
I20260709 13:07:52.592483 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:52.601995  5805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.603304  5804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.605477  5807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:52.606611 27934 server_base.cc:1061] running on GCE node
I20260709 13:07:52.607816 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:52.608152 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:52.608371 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602472608353 us; error 0 us; skew 500 ppm
I20260709 13:07:52.608698  5801 heartbeater.cc:344] Connected to a master server at 127.27.71.190:45203
I20260709 13:07:52.609144 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:52.609169  5801 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:52.610491  5801 heartbeater.cc:507] Master 127.27.71.190:45203 requested a full tablet report, sending...
I20260709 13:07:52.612854 27934 webserver.cc:533] Webserver started at http://127.27.71.131:44549/ using document root <none> and password file <none>
I20260709 13:07:52.613276  5599 ts_manager.cc:194] Registered new tserver with Master: 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:52.613435 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:52.613780 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:52.614054 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:52.614894  5599 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39344
I20260709 13:07:52.615360 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/instance:
uuid: "52b726da57c24dd2aa72488ebe2ff21e"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-mlt7"
I20260709 13:07:52.620756 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.001s
I20260709 13:07:52.624472  5812 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.625393 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:07:52.625737 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-2-root
uuid: "52b726da57c24dd2aa72488ebe2ff21e"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-mlt7"
I20260709 13:07:52.626056 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:52.641343 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:52.642689 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:52.644277 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:52.646581 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:52.646818 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.647065 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:52.647243 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.690673 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.131:37585
I20260709 13:07:52.690801  5875 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.131:37585 every 8 connection(s)
I20260709 13:07:52.695832 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:52.703075  5879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:52.704244  5880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:52.709863  5876 heartbeater.cc:344] Connected to a master server at 127.27.71.190:45203
I20260709 13:07:52.710455  5876 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:52.710814 27934 server_base.cc:1061] running on GCE node
W20260709 13:07:52.710852  5883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:52.711614  5876 heartbeater.cc:507] Master 127.27.71.190:45203 requested a full tablet report, sending...
I20260709 13:07:52.712183 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:52.712452 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:52.712673 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602472712654 us; error 0 us; skew 500 ppm
I20260709 13:07:52.713376 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:52.713860  5599 ts_manager.cc:194] Registered new tserver with Master: 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:52.715474  5599 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39354
I20260709 13:07:52.716085 27934 webserver.cc:533] Webserver started at http://127.27.71.132:45187/ using document root <none> and password file <none>
I20260709 13:07:52.716714 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:52.716974 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:52.717259 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:52.718433 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-3-root/instance:
uuid: "792907f253c744aa9392e8d572f88e06"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-mlt7"
I20260709 13:07:52.723503 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:07:52.726912  5887 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.727679 27934 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:07:52.728070 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-3-root
uuid: "792907f253c744aa9392e8d572f88e06"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-mlt7"
I20260709 13:07:52.728384 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:52.746531 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:52.747891 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:52.749516 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:52.751818 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:52.752092 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.752341 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:52.752522 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:52.797966 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.132:45241
I20260709 13:07:52.798096  5949 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.132:45241 every 8 connection(s)
I20260709 13:07:52.813270  5950 heartbeater.cc:344] Connected to a master server at 127.27.71.190:45203
I20260709 13:07:52.813761  5950 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:52.814532  5950 heartbeater.cc:507] Master 127.27.71.190:45203 requested a full tablet report, sending...
I20260709 13:07:52.816633  5599 ts_manager.cc:194] Registered new tserver with Master: 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:52.817800 27934 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016571714s
I20260709 13:07:52.818046  5599 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39364
I20260709 13:07:52.848776  5599 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39378:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:07:53.117403  5914 tablet_service.cc:1511] Processing CreateTablet for tablet 892918414c974de38c7050ccd7151f94 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:07:53.118269  5912 tablet_service.cc:1511] Processing CreateTablet for tablet a9e48769916844ce8d464a5bd1bda6de (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:07:53.119119  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 892918414c974de38c7050ccd7151f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.119827  5912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e48769916844ce8d464a5bd1bda6de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.120220  5915 tablet_service.cc:1511] Processing CreateTablet for tablet a9dfeaefea774ae19834e0a619330eb5 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:07:53.121635  5915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9dfeaefea774ae19834e0a619330eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.123561  5913 tablet_service.cc:1511] Processing CreateTablet for tablet e78f9ad8be2a4dd6bf2aa1d922e471a2 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:07:53.125134  5913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e78f9ad8be2a4dd6bf2aa1d922e471a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.126757  5841 tablet_service.cc:1511] Processing CreateTablet for tablet a9dfeaefea774ae19834e0a619330eb5 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:07:53.128247  5841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9dfeaefea774ae19834e0a619330eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.129220  5766 tablet_service.cc:1511] Processing CreateTablet for tablet 77f29b60a32f4f79a3f1bd59278b00bd (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:07:53.130722  5766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f29b60a32f4f79a3f1bd59278b00bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.131705  5840 tablet_service.cc:1511] Processing CreateTablet for tablet 77f29b60a32f4f79a3f1bd59278b00bd (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:07:53.132759  5765 tablet_service.cc:1511] Processing CreateTablet for tablet 892918414c974de38c7050ccd7151f94 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:07:53.134255  5765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 892918414c974de38c7050ccd7151f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.134761  5691 tablet_service.cc:1511] Processing CreateTablet for tablet a9dfeaefea774ae19834e0a619330eb5 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:07:53.133253  5840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f29b60a32f4f79a3f1bd59278b00bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.135299  5839 tablet_service.cc:1511] Processing CreateTablet for tablet e78f9ad8be2a4dd6bf2aa1d922e471a2 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:07:53.118862  5911 tablet_service.cc:1511] Processing CreateTablet for tablet 40ba6b5dcd674422a95eeaad0efd1e45 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:07:53.137722  5764 tablet_service.cc:1511] Processing CreateTablet for tablet e78f9ad8be2a4dd6bf2aa1d922e471a2 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:07:53.138285  5911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40ba6b5dcd674422a95eeaad0efd1e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.139293  5764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e78f9ad8be2a4dd6bf2aa1d922e471a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.139365  5839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e78f9ad8be2a4dd6bf2aa1d922e471a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.141094  5762 tablet_service.cc:1511] Processing CreateTablet for tablet a9e48769916844ce8d464a5bd1bda6de (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:07:53.142728  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e48769916844ce8d464a5bd1bda6de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.145170  5690 tablet_service.cc:1511] Processing CreateTablet for tablet 77f29b60a32f4f79a3f1bd59278b00bd (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:07:53.145871  5838 tablet_service.cc:1511] Processing CreateTablet for tablet 2ddbd585b0264150a1fbd1b5cb8b5ebb (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:07:53.147483  5838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ddbd585b0264150a1fbd1b5cb8b5ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.147305  5763 tablet_service.cc:1511] Processing CreateTablet for tablet 2ddbd585b0264150a1fbd1b5cb8b5ebb (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:07:53.150490  5763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ddbd585b0264150a1fbd1b5cb8b5ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.137907  5688 tablet_service.cc:1511] Processing CreateTablet for tablet 2ddbd585b0264150a1fbd1b5cb8b5ebb (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:07:53.139659  5687 tablet_service.cc:1511] Processing CreateTablet for tablet a9e48769916844ce8d464a5bd1bda6de (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:07:53.153285  5691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9dfeaefea774ae19834e0a619330eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.153415  5837 tablet_service.cc:1511] Processing CreateTablet for tablet 40ba6b5dcd674422a95eeaad0efd1e45 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:07:53.154989  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40ba6b5dcd674422a95eeaad0efd1e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.159176  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ddbd585b0264150a1fbd1b5cb8b5ebb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.161842  5690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f29b60a32f4f79a3f1bd59278b00bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.165843  5687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e48769916844ce8d464a5bd1bda6de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.149037  5689 tablet_service.cc:1511] Processing CreateTablet for tablet 892918414c974de38c7050ccd7151f94 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:07:53.179067  5689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 892918414c974de38c7050ccd7151f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.205050  5686 tablet_service.cc:1511] Processing CreateTablet for tablet 40ba6b5dcd674422a95eeaad0efd1e45 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:07:53.207077  5686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40ba6b5dcd674422a95eeaad0efd1e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.230064  5761 tablet_service.cc:1511] Processing CreateTablet for tablet c7f81180a2c94fe5874f1a402e44c52f (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:07:53.231454  5761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f81180a2c94fe5874f1a402e44c52f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.236577  5836 tablet_service.cc:1511] Processing CreateTablet for tablet c7f81180a2c94fe5874f1a402e44c52f (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:07:53.240132  5685 tablet_service.cc:1511] Processing CreateTablet for tablet 7f95c9f03a2848e29a1ed53170f6bc85 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:07:53.241636  5685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f95c9f03a2848e29a1ed53170f6bc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.246438  5910 tablet_service.cc:1511] Processing CreateTablet for tablet c7f81180a2c94fe5874f1a402e44c52f (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:07:53.252130  5910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f81180a2c94fe5874f1a402e44c52f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.255677  5760 tablet_service.cc:1511] Processing CreateTablet for tablet 7f95c9f03a2848e29a1ed53170f6bc85 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:07:53.257344  5760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f95c9f03a2848e29a1ed53170f6bc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.257048  5835 tablet_service.cc:1511] Processing CreateTablet for tablet 7f95c9f03a2848e29a1ed53170f6bc85 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:07:53.238638  5836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f81180a2c94fe5874f1a402e44c52f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.264926  5835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f95c9f03a2848e29a1ed53170f6bc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.292348  5970 tablet_bootstrap.cc:492] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
W20260709 13:07:53.306011  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:53.345268  5913 tablet_service.cc:1511] Processing CreateTablet for tablet 3c979e51cdc44dac9ae6b2117e8ac0c5 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:07:53.346760  5913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c979e51cdc44dac9ae6b2117e8ac0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.368772  5971 tablet_bootstrap.cc:492] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:53.374692  5684 tablet_service.cc:1511] Processing CreateTablet for tablet 3c979e51cdc44dac9ae6b2117e8ac0c5 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:07:53.376784  5972 tablet_bootstrap.cc:492] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:53.377605  5684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c979e51cdc44dac9ae6b2117e8ac0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.384934  5765 tablet_service.cc:1511] Processing CreateTablet for tablet 3c979e51cdc44dac9ae6b2117e8ac0c5 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:07:53.386581  5837 tablet_service.cc:1511] Processing CreateTablet for tablet 96bcc183bf584d4e8fbd4d0a87c2e2ea (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:07:53.387585  5765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c979e51cdc44dac9ae6b2117e8ac0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.398218  5841 tablet_service.cc:1511] Processing CreateTablet for tablet e2d7f76b05b8419783f1a8f16afaec07 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:07:53.392262  5912 tablet_service.cc:1511] Processing CreateTablet for tablet 96bcc183bf584d4e8fbd4d0a87c2e2ea (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:07:53.393392  5683 tablet_service.cc:1511] Processing CreateTablet for tablet 96bcc183bf584d4e8fbd4d0a87c2e2ea (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:07:53.400668  5683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96bcc183bf584d4e8fbd4d0a87c2e2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.401117  5763 tablet_service.cc:1511] Processing CreateTablet for tablet e2d7f76b05b8419783f1a8f16afaec07 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:07:53.402626  5763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2d7f76b05b8419783f1a8f16afaec07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.408901  5912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96bcc183bf584d4e8fbd4d0a87c2e2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.410420  5970 tablet_bootstrap.cc:654] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:53.388074  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96bcc183bf584d4e8fbd4d0a87c2e2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.416579  5841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2d7f76b05b8419783f1a8f16afaec07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.422827  5973 tablet_bootstrap.cc:492] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:53.424856  5971 tablet_bootstrap.cc:654] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:53.433449  5973 tablet_bootstrap.cc:654] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:53.445873  5972 tablet_bootstrap.cc:654] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:53.446944  5973 tablet_bootstrap.cc:492] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:53.447561  5973 ts_tablet_manager.cc:1403] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.013s
I20260709 13:07:53.451017  5973 raft_consensus.cc:359] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:53.451818  5973 raft_consensus.cc:385] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.452159  5973 raft_consensus.cc:740] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.453881  5973 consensus_queue.cc:260] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:53.456107  5726 heartbeater.cc:499] Master 127.27.71.190:45203 was elected leader, sending a full tablet report...
I20260709 13:07:53.456447  5913 tablet_service.cc:1511] Processing CreateTablet for tablet e2d7f76b05b8419783f1a8f16afaec07 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:07:53.457161  5687 tablet_service.cc:1511] Processing CreateTablet for tablet 75595b8e97ef436992a8acba2c9724a7 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:07:53.457823  5913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2d7f76b05b8419783f1a8f16afaec07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.461103  5973 ts_tablet_manager.cc:1434] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:07:53.462333  5973 tablet_bootstrap.cc:492] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:53.464802  5762 tablet_service.cc:1511] Processing CreateTablet for tablet 75595b8e97ef436992a8acba2c9724a7 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:07:53.466223  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75595b8e97ef436992a8acba2c9724a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.466084  5837 tablet_service.cc:1511] Processing CreateTablet for tablet 75595b8e97ef436992a8acba2c9724a7 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:07:53.467363  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75595b8e97ef436992a8acba2c9724a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.468495  5691 tablet_service.cc:1511] Processing CreateTablet for tablet 2e47f79b0ad84696b05ddc28ce9bef20 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:07:53.469787  5691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e47f79b0ad84696b05ddc28ce9bef20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.469614  5760 tablet_service.cc:1511] Processing CreateTablet for tablet 2e47f79b0ad84696b05ddc28ce9bef20 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:07:53.470829  5760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e47f79b0ad84696b05ddc28ce9bef20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.473989  5914 tablet_service.cc:1511] Processing CreateTablet for tablet 2e47f79b0ad84696b05ddc28ce9bef20 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:07:53.475322  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e47f79b0ad84696b05ddc28ce9bef20. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:53.475350  5727 tablet.cc:2406] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:07:53.477262  5970 tablet_bootstrap.cc:492] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:53.477769  5970 ts_tablet_manager.cc:1403] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.186s	user 0.014s	sys 0.000s
I20260709 13:07:53.481178  5970 raft_consensus.cc:359] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:53.481943  5970 raft_consensus.cc:385] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.482268  5970 raft_consensus.cc:740] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.483562  5687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75595b8e97ef436992a8acba2c9724a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.490469  5973 tablet_bootstrap.cc:654] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:53.501561  5970 consensus_queue.cc:260] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:53.474045  5971 tablet_bootstrap.cc:492] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:53.505580  5971 ts_tablet_manager.cc:1403] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.137s	user 0.010s	sys 0.003s
I20260709 13:07:53.521314  5801 heartbeater.cc:499] Master 127.27.71.190:45203 was elected leader, sending a full tablet report...
I20260709 13:07:53.521849  5970 ts_tablet_manager.cc:1434] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.005s
I20260709 13:07:53.527230  5971 raft_consensus.cc:359] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:53.530349  5970 tablet_bootstrap.cc:492] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:53.535792  5971 raft_consensus.cc:385] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.544938  5971 raft_consensus.cc:740] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.545208  5912 tablet_service.cc:1511] Processing CreateTablet for tablet 9f08c26e76934d3c94ef7060e15caed0 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:07:53.546384  5973 tablet_bootstrap.cc:492] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:53.546613  5912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f08c26e76934d3c94ef7060e15caed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.546077  5971 consensus_queue.cc:260] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:53.546938  5973 ts_tablet_manager.cc:1403] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.085s	user 0.012s	sys 0.002s
I20260709 13:07:53.548921  5837 tablet_service.cc:1511] Processing CreateTablet for tablet 9f08c26e76934d3c94ef7060e15caed0 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:07:53.551393  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f08c26e76934d3c94ef7060e15caed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.552870  5973 raft_consensus.cc:359] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:53.549314  5970 tablet_bootstrap.cc:654] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:53.553277  5691 tablet_service.cc:1511] Processing CreateTablet for tablet 9f08c26e76934d3c94ef7060e15caed0 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:07:53.554951  5691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f08c26e76934d3c94ef7060e15caed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.553676  5973 raft_consensus.cc:385] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.559449  5836 tablet_service.cc:1511] Processing CreateTablet for tablet adfec13bcbce4db48b5f0dba7a8e5345 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:07:53.560129  5973 raft_consensus.cc:740] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.562265  5973 consensus_queue.cc:260] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:53.563701  5836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adfec13bcbce4db48b5f0dba7a8e5345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.566455  5971 ts_tablet_manager.cc:1434] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.059s	user 0.004s	sys 0.009s
I20260709 13:07:53.568046  5876 heartbeater.cc:499] Master 127.27.71.190:45203 was elected leader, sending a full tablet report...
I20260709 13:07:53.570997  5914 tablet_service.cc:1511] Processing CreateTablet for tablet adfec13bcbce4db48b5f0dba7a8e5345 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:07:53.572680  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adfec13bcbce4db48b5f0dba7a8e5345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.576697  5687 tablet_service.cc:1511] Processing CreateTablet for tablet 32b6cadc1b9d441983391c6e69f42beb (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:07:53.578173  5687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b6cadc1b9d441983391c6e69f42beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.581101  5973 ts_tablet_manager.cc:1434] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.033s	user 0.001s	sys 0.006s
I20260709 13:07:53.582216  5973 tablet_bootstrap.cc:492] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:53.588891  5972 tablet_bootstrap.cc:492] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:53.589133  5973 tablet_bootstrap.cc:654] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:53.589493  5972 ts_tablet_manager.cc:1403] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.213s	user 0.015s	sys 0.012s
I20260709 13:07:53.592422  5971 tablet_bootstrap.cc:492] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:53.596586  5762 tablet_service.cc:1511] Processing CreateTablet for tablet adfec13bcbce4db48b5f0dba7a8e5345 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:07:53.598047  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adfec13bcbce4db48b5f0dba7a8e5345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.601536  5972 raft_consensus.cc:359] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:53.602387  5972 raft_consensus.cc:385] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.602708  5972 raft_consensus.cc:740] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.604277  5972 consensus_queue.cc:260] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:53.606463  5981 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.607009  5981 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:53.599999  5971 tablet_bootstrap.cc:654] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:53.617468  5950 heartbeater.cc:499] Master 127.27.71.190:45203 was elected leader, sending a full tablet report...
I20260709 13:07:53.625124  5981 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:53.644891  5980 raft_consensus.cc:493] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.645519  5980 raft_consensus.cc:515] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:53.646848  5760 tablet_service.cc:1511] Processing CreateTablet for tablet 32b6cadc1b9d441983391c6e69f42beb (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:07:53.648722  5760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b6cadc1b9d441983391c6e69f42beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.618755  5972 ts_tablet_manager.cc:1434] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.005s
I20260709 13:07:53.672124  5973 tablet_bootstrap.cc:492] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:53.663043  5914 tablet_service.cc:1511] Processing CreateTablet for tablet a1b715b22a0343fbb87245b6bf0fb522 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:07:53.672828  5973 ts_tablet_manager.cc:1403] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.091s	user 0.020s	sys 0.024s
I20260709 13:07:53.673803  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b715b22a0343fbb87245b6bf0fb522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.674836  5683 tablet_service.cc:1511] Processing CreateTablet for tablet a1b715b22a0343fbb87245b6bf0fb522 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:07:53.676746  5683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b715b22a0343fbb87245b6bf0fb522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.676594  5973 raft_consensus.cc:359] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:53.677595  5973 raft_consensus.cc:385] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.677706  5970 tablet_bootstrap.cc:492] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:53.677944  5980 leader_election.cc:290] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:53.678310  5970 ts_tablet_manager.cc:1403] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.156s	user 0.018s	sys 0.015s
I20260709 13:07:53.681669  5970 raft_consensus.cc:359] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:53.682443  5970 raft_consensus.cc:385] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.682809  5970 raft_consensus.cc:740] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.697433  5763 tablet_service.cc:1511] Processing CreateTablet for tablet a1b715b22a0343fbb87245b6bf0fb522 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:07:53.698915  5763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b715b22a0343fbb87245b6bf0fb522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.702097  5913 tablet_service.cc:1511] Processing CreateTablet for tablet a3f19012a0a24bd88cecafe95b9b461c (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:07:53.703516  5913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3f19012a0a24bd88cecafe95b9b461c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.704663  5688 tablet_service.cc:1511] Processing CreateTablet for tablet a3f19012a0a24bd88cecafe95b9b461c (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:07:53.705973  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3f19012a0a24bd88cecafe95b9b461c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.709812  5972 tablet_bootstrap.cc:492] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:53.683777  5970 consensus_queue.cc:260] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:53.714385  5970 ts_tablet_manager.cc:1434] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:07:53.684310  5973 raft_consensus.cc:740] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.741055  5970 tablet_bootstrap.cc:492] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:53.727667  5837 tablet_service.cc:1511] Processing CreateTablet for tablet a3f19012a0a24bd88cecafe95b9b461c (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:07:53.742297  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3f19012a0a24bd88cecafe95b9b461c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.746640  5841 tablet_service.cc:1511] Processing CreateTablet for tablet a62aede00e774cabb36368e7202cdff9 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:07:53.747116  5762 tablet_service.cc:1511] Processing CreateTablet for tablet a62aede00e774cabb36368e7202cdff9 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:07:53.748194  5841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a62aede00e774cabb36368e7202cdff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.748448  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a62aede00e774cabb36368e7202cdff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.730154  5971 tablet_bootstrap.cc:492] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:53.740707  5973 consensus_queue.cc:260] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:53.719067  5836 tablet_service.cc:1511] Processing CreateTablet for tablet 32b6cadc1b9d441983391c6e69f42beb (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:07:53.752627  5836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b6cadc1b9d441983391c6e69f42beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.753486  5973 ts_tablet_manager.cc:1434] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.000s
I20260709 13:07:53.754679  5973 tablet_bootstrap.cc:492] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:53.757109  5912 tablet_service.cc:1511] Processing CreateTablet for tablet a62aede00e774cabb36368e7202cdff9 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:07:53.758525  5912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a62aede00e774cabb36368e7202cdff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.762153  5973 tablet_bootstrap.cc:654] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:53.764254  5971 ts_tablet_manager.cc:1403] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.172s	user 0.021s	sys 0.009s
I20260709 13:07:53.768532  5691 tablet_service.cc:1511] Processing CreateTablet for tablet 365d76e5c79a4b0b8b65ad7465deac93 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:07:53.774632  5691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 365d76e5c79a4b0b8b65ad7465deac93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.783396  5970 tablet_bootstrap.cc:654] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:53.791270  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:53.784380  5972 tablet_bootstrap.cc:654] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:53.794449  5813 leader_election.cc:343] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:53.767751  5971 raft_consensus.cc:359] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:53.797473  5971 raft_consensus.cc:385] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.799805  5971 raft_consensus.cc:740] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.801462  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:53.803830  5765 tablet_service.cc:1511] Processing CreateTablet for tablet 365d76e5c79a4b0b8b65ad7465deac93 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:07:53.805701  5765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 365d76e5c79a4b0b8b65ad7465deac93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.806470  5971 consensus_queue.cc:260] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:53.807529  5851 raft_consensus.cc:2468] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:07:53.809957  5983 raft_consensus.cc:493] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.814702  5983 raft_consensus.cc:515] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:53.809908  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:07:53.817564  5816 leader_election.cc:343] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:53.818035  5816 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:53.819190  5981 raft_consensus.cc:2749] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:53.820520  5983 leader_election.cc:290] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:53.821765  5976 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:53.822371  5976 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:53.838068  5836 tablet_service.cc:1511] Processing CreateTablet for tablet 106cdfafda414cf6981dbcaf1e28cc9d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:07:53.837926  5683 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee9614d828d4ea8ab18a900dbc063c1 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:07:53.839463  5683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee9614d828d4ea8ab18a900dbc063c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.828845  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:07:53.842551  5816 leader_election.cc:343] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:53.839475  5836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 106cdfafda414cf6981dbcaf1e28cc9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.827845  5976 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:53.845746  5914 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee9614d828d4ea8ab18a900dbc063c1 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:07:53.853665  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee9614d828d4ea8ab18a900dbc063c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.855574  5910 tablet_service.cc:1511] Processing CreateTablet for tablet 106cdfafda414cf6981dbcaf1e28cc9d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:07:53.857743  5910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 106cdfafda414cf6981dbcaf1e28cc9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.861130  5760 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee9614d828d4ea8ab18a900dbc063c1 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:07:53.862818  5760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee9614d828d4ea8ab18a900dbc063c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.869463  5688 tablet_service.cc:1511] Processing CreateTablet for tablet 106cdfafda414cf6981dbcaf1e28cc9d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:07:53.870553  5766 tablet_service.cc:1511] Processing CreateTablet for tablet 8ba1e6d2a3344631b647a567e5280513 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:07:53.872077  5766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba1e6d2a3344631b647a567e5280513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.872185  5838 tablet_service.cc:1511] Processing CreateTablet for tablet 8ba1e6d2a3344631b647a567e5280513 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:07:53.815723  5742 leader_election.cc:304] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:07:53.874032  5838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba1e6d2a3344631b647a567e5280513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.875070  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 106cdfafda414cf6981dbcaf1e28cc9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.879876  5971 ts_tablet_manager.cc:1434] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.115s	user 0.004s	sys 0.007s
I20260709 13:07:53.887795  5971 tablet_bootstrap.cc:492] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:53.895205  5973 tablet_bootstrap.cc:492] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:53.895823  5973 ts_tablet_manager.cc:1403] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.141s	user 0.017s	sys 0.007s
I20260709 13:07:53.897037  5980 raft_consensus.cc:2804] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:53.896893  5915 tablet_service.cc:1511] Processing CreateTablet for tablet 8ba1e6d2a3344631b647a567e5280513 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:07:53.897578  5980 raft_consensus.cc:493] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:53.898034  5980 raft_consensus.cc:3060] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.898353  5915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ba1e6d2a3344631b647a567e5280513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.876380  5837 tablet_service.cc:1511] Processing CreateTablet for tablet 365d76e5c79a4b0b8b65ad7465deac93 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:07:53.905478  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 365d76e5c79a4b0b8b65ad7465deac93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.906535  5687 tablet_service.cc:1511] Processing CreateTablet for tablet fbbd1771e7664e0cb3441c84672d037d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:07:53.906792  5980 raft_consensus.cc:515] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:07:53.896957  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 81559.2us
I20260709 13:07:53.908088  5687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbbd1771e7664e0cb3441c84672d037d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.899593  5973 raft_consensus.cc:359] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:53.921950  5763 tablet_service.cc:1511] Processing CreateTablet for tablet fbbd1771e7664e0cb3441c84672d037d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:07:53.923183  5972 tablet_bootstrap.cc:492] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:53.923461  5763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbbd1771e7664e0cb3441c84672d037d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.923799  5972 ts_tablet_manager.cc:1403] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.214s	user 0.029s	sys 0.029s
I20260709 13:07:53.938210  5980 leader_election.cc:290] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:53.939283  5835 tablet_service.cc:1511] Processing CreateTablet for tablet fbbd1771e7664e0cb3441c84672d037d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:07:53.944614  5835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbbd1771e7664e0cb3441c84672d037d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.939630  5913 tablet_service.cc:1511] Processing CreateTablet for tablet 8aa6b3cf215a40a5b6bc25bd3dddc408 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:07:53.946193  5913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8aa6b3cf215a40a5b6bc25bd3dddc408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.941039  5972 raft_consensus.cc:359] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:53.919449  5973 raft_consensus.cc:385] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.950490  5973 raft_consensus.cc:740] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.951339  5973 consensus_queue.cc:260] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:53.949471  5972 raft_consensus.cc:385] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:53.953444  5688 tablet_service.cc:1511] Processing CreateTablet for tablet 8aa6b3cf215a40a5b6bc25bd3dddc408 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:07:53.963196  5972 raft_consensus.cc:740] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:53.964874  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8aa6b3cf215a40a5b6bc25bd3dddc408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.965615  5972 consensus_queue.cc:260] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:53.971863  5972 ts_tablet_manager.cc:1434] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.046s	user 0.002s	sys 0.005s
I20260709 13:07:53.978353  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:53.979075  5851 raft_consensus.cc:3060] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:53.979665  5691 tablet_service.cc:1511] Processing CreateTablet for tablet 8cb2353dae2d41588ef00b83e6cc10d1 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:07:53.981140  5691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cb2353dae2d41588ef00b83e6cc10d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.992151  5973 ts_tablet_manager.cc:1434] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.096s	user 0.007s	sys 0.001s
I20260709 13:07:53.997896  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:53.998646  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:54.001621  5762 tablet_service.cc:1511] Processing CreateTablet for tablet 8aa6b3cf215a40a5b6bc25bd3dddc408 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:07:54.003022  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8aa6b3cf215a40a5b6bc25bd3dddc408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.004956  5912 tablet_service.cc:1511] Processing CreateTablet for tablet 8cb2353dae2d41588ef00b83e6cc10d1 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:07:54.005723  5970 tablet_bootstrap.cc:492] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:54.006321  5970 ts_tablet_manager.cc:1403] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.265s	user 0.011s	sys 0.005s
I20260709 13:07:54.006421  5912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cb2353dae2d41588ef00b83e6cc10d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:53.998682  5924 raft_consensus.cc:3060] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.009109  5761 tablet_service.cc:1511] Processing CreateTablet for tablet 674fd655eaf14b10bc2e013c27c40557 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:07:54.010577  5761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674fd655eaf14b10bc2e013c27c40557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.012300  5971 tablet_bootstrap.cc:654] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.014272  5970 raft_consensus.cc:359] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.018020  5970 raft_consensus.cc:385] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.018389  5970 raft_consensus.cc:740] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.023804  5973 tablet_bootstrap.cc:492] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:54.022840  5970 consensus_queue.cc:260] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.029188  5838 tablet_service.cc:1511] Processing CreateTablet for tablet 8cb2353dae2d41588ef00b83e6cc10d1 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:07:54.030311  5976 raft_consensus.cc:493] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.030951  5838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cb2353dae2d41588ef00b83e6cc10d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.030864  5976 raft_consensus.cc:515] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.031839  5970 ts_tablet_manager.cc:1434] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.003s
I20260709 13:07:54.032968  5970 tablet_bootstrap.cc:492] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:54.033509  5976 leader_election.cc:290] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:54.034307  5851 raft_consensus.cc:2468] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:07:54.036358  5742 leader_election.cc:304] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:07:54.039520  5980 raft_consensus.cc:2804] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:54.055083  5980 raft_consensus.cc:697] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:07:54.056692  5976 raft_consensus.cc:493] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.061993  5984 raft_consensus.cc:493] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.062938  5841 tablet_service.cc:1511] Processing CreateTablet for tablet 674fd655eaf14b10bc2e013c27c40557 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:07:54.062716  5686 tablet_service.cc:1511] Processing CreateTablet for tablet e982efc662b84721b51ef9101f0cf5a8 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:07:54.065052  5911 tablet_service.cc:1511] Processing CreateTablet for tablet 674fd655eaf14b10bc2e013c27c40557 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:07:54.068125  5972 tablet_bootstrap.cc:492] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:54.076907  5686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e982efc662b84721b51ef9101f0cf5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.080627  5841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674fd655eaf14b10bc2e013c27c40557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.082270  5973 tablet_bootstrap.cc:654] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.083886  5970 tablet_bootstrap.cc:654] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.071744  5984 raft_consensus.cc:515] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:54.084448  5766 tablet_service.cc:1511] Processing CreateTablet for tablet e982efc662b84721b51ef9101f0cf5a8 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:07:54.085960  5766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e982efc662b84721b51ef9101f0cf5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.099077  5911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674fd655eaf14b10bc2e013c27c40557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.100593  5971 tablet_bootstrap.cc:492] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:54.100687  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:54.101321  5971 ts_tablet_manager.cc:1403] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.214s	user 0.009s	sys 0.013s
I20260709 13:07:54.104930  5815 leader_election.cc:304] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:54.070329  5976 raft_consensus.cc:515] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.106380  5983 raft_consensus.cc:2749] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:54.107703  5976 leader_election.cc:290] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:54.125793  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:54.107836  5971 raft_consensus.cc:359] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.128311  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:54.128854  5971 raft_consensus.cc:385] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.129145  5775 raft_consensus.cc:2468] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:54.117781  5973 tablet_bootstrap.cc:492] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:54.129251  5971 raft_consensus.cc:740] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.130143  5973 ts_tablet_manager.cc:1403] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.107s	user 0.019s	sys 0.001s
I20260709 13:07:54.131038  5666 leader_election.cc:304] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:54.131208  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:54.132411  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:54.132452  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:54.134529  5973 raft_consensus.cc:359] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:07:54.135082  5666 leader_election.cc:343] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.135309  5973 raft_consensus.cc:385] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.135659  5973 raft_consensus.cc:740] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.136482  5973 consensus_queue.cc:260] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:54.137355  5976 raft_consensus.cc:2804] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:54.141603  5976 raft_consensus.cc:493] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.122017  5972 tablet_bootstrap.cc:654] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.141979  5976 raft_consensus.cc:3060] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.124706  5924 raft_consensus.cc:2468] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:07:54.130918  5971 consensus_queue.cc:260] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.149348  5980 consensus_queue.cc:237] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:54.151134  5971 ts_tablet_manager.cc:1434] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.006s
I20260709 13:07:54.152125  5850 raft_consensus.cc:2468] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:54.093935  5984 leader_election.cc:290] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:54.155891  5971 tablet_bootstrap.cc:492] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:54.163094  5971 tablet_bootstrap.cc:654] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.166621  5976 raft_consensus.cc:515] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.182431  5910 tablet_service.cc:1511] Processing CreateTablet for tablet 400faa6e9aed42f19cda604767b955c2 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:07:54.183873  5910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 400faa6e9aed42f19cda604767b955c2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:54.184190  5667 leader_election.cc:343] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:54.184985  5667 leader_election.cc:343] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.185428  5667 leader_election.cc:304] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:54.256965  5837 tablet_service.cc:1511] Processing CreateTablet for tablet e982efc662b84721b51ef9101f0cf5a8 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:07:54.257975  5976 leader_election.cc:290] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:54.258548  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e982efc662b84721b51ef9101f0cf5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.258850  6000 raft_consensus.cc:493] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.259392  6000 raft_consensus.cc:515] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:54.274602  6000 leader_election.cc:290] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:54.275400  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:07:54.277158  5742 leader_election.cc:343] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.280591  5766 tablet_service.cc:1511] Processing CreateTablet for tablet 400faa6e9aed42f19cda604767b955c2 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:07:54.282370  5766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 400faa6e9aed42f19cda604767b955c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.283700  5836 tablet_service.cc:1511] Processing CreateTablet for tablet 759acc6b90aa41a1b6c9053bd20846c1 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:07:54.285316  5836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 759acc6b90aa41a1b6c9053bd20846c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.289232  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:54.290143  5776 raft_consensus.cc:3060] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.306546  5688 tablet_service.cc:1511] Processing CreateTablet for tablet 400faa6e9aed42f19cda604767b955c2 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
W20260709 13:07:54.310643  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:54.312063  6008 raft_consensus.cc:2749] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:54.312601  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 400faa6e9aed42f19cda604767b955c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.317451  5973 ts_tablet_manager.cc:1434] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.186s	user 0.007s	sys 0.003s
I20260709 13:07:54.321002  5973 tablet_bootstrap.cc:492] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:54.329579  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:54.332173  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:54.340310  5850 raft_consensus.cc:3060] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.341939  5761 tablet_service.cc:1511] Processing CreateTablet for tablet dd06784a6f3244309ebdd41cf8127741 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:07:54.345402  5776 raft_consensus.cc:2468] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:54.350466  5970 tablet_bootstrap.cc:492] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:54.351110  5970 ts_tablet_manager.cc:1403] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.318s	user 0.034s	sys 0.018s
I20260709 13:07:54.351236  5666 leader_election.cc:304] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:54.352341  6011 raft_consensus.cc:2804] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:54.352669  5761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd06784a6f3244309ebdd41cf8127741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.352885  6011 raft_consensus.cc:697] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:54.356737  6011 consensus_queue.cc:237] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.357576  5971 tablet_bootstrap.cc:492] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:54.358127  5971 ts_tablet_manager.cc:1403] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.202s	user 0.003s	sys 0.010s
W20260709 13:07:54.353341  5667 leader_election.cc:343] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.354511  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:54.360229  5667 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:54.359622  5599 catalog_manager.cc:5697] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 1 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:54.361191  6008 raft_consensus.cc:2749] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:54.362617  5888 leader_election.cc:343] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.362974  5973 tablet_bootstrap.cc:654] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.354570  5970 raft_consensus.cc:359] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:54.363626  5970 raft_consensus.cc:385] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.363998  5970 raft_consensus.cc:740] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.364890  5970 consensus_queue.cc:260] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:54.367496  5850 raft_consensus.cc:2468] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:54.369017  5970 ts_tablet_manager.cc:1434] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:07:54.366719  5837 tablet_service.cc:1511] Processing CreateTablet for tablet dd06784a6f3244309ebdd41cf8127741 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:07:54.368839  5913 tablet_service.cc:1511] Processing CreateTablet for tablet dd06784a6f3244309ebdd41cf8127741 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:07:54.373153  5683 tablet_service.cc:1511] Processing CreateTablet for tablet 43ed8e83113241bda21a6e92ed10ac9e (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:07:54.380568  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd06784a6f3244309ebdd41cf8127741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.375627  5835 tablet_service.cc:1511] Processing CreateTablet for tablet 43ed8e83113241bda21a6e92ed10ac9e (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:07:54.381757  5835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43ed8e83113241bda21a6e92ed10ac9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.378101  5762 tablet_service.cc:1511] Processing CreateTablet for tablet 43ed8e83113241bda21a6e92ed10ac9e (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:07:54.382695  6008 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.383232  6008 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:54.384084  5911 tablet_service.cc:1511] Processing CreateTablet for tablet 759acc6b90aa41a1b6c9053bd20846c1 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:07:54.384689  5913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd06784a6f3244309ebdd41cf8127741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.385545  5911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 759acc6b90aa41a1b6c9053bd20846c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.380705  5683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43ed8e83113241bda21a6e92ed10ac9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.380780  5972 tablet_bootstrap.cc:492] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:54.385910  6008 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:54.390977  5914 tablet_service.cc:1511] Processing CreateTablet for tablet 422b96951eda4c1c8b8d86612ac6c163 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:07:54.392638  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:54.391031  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43ed8e83113241bda21a6e92ed10ac9e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:54.397897  5667 leader_election.cc:343] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.395727  5972 ts_tablet_manager.cc:1403] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.350s	user 0.024s	sys 0.032s
I20260709 13:07:54.398516  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:54.404888  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:07:54.406353  5890 leader_election.cc:343] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.406862  5890 leader_election.cc:304] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:54.407866  5984 raft_consensus.cc:2749] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:54.406818  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422b96951eda4c1c8b8d86612ac6c163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.405994  5972 raft_consensus.cc:359] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:54.398940  5765 tablet_service.cc:1511] Processing CreateTablet for tablet 422b96951eda4c1c8b8d86612ac6c163 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:07:54.412210  5970 tablet_bootstrap.cc:492] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:54.413707  5765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422b96951eda4c1c8b8d86612ac6c163. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:54.424208  5666 leader_election.cc:343] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.424914  5666 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:54.425849  6008 raft_consensus.cc:2749] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:54.429080  5691 tablet_service.cc:1511] Processing CreateTablet for tablet 422b96951eda4c1c8b8d86612ac6c163 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:07:54.431048  5691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422b96951eda4c1c8b8d86612ac6c163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.407369  5686 tablet_service.cc:1511] Processing CreateTablet for tablet 759acc6b90aa41a1b6c9053bd20846c1 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:07:54.410320  5972 raft_consensus.cc:385] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.450825  5686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 759acc6b90aa41a1b6c9053bd20846c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.453176  5841 tablet_service.cc:1511] Processing CreateTablet for tablet d76ed139f0d94b8484f5b5101634fb1b (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:07:54.454057  5971 raft_consensus.cc:359] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:54.454931  5971 raft_consensus.cc:385] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.455394  5971 raft_consensus.cc:740] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.454630  5841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d76ed139f0d94b8484f5b5101634fb1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.457240  5971 consensus_queue.cc:260] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:54.445783  5972 raft_consensus.cc:740] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.468844  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:54.471939  5701 raft_consensus.cc:2468] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:07:54.473353  5739 leader_election.cc:304] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:54.474823  5980 raft_consensus.cc:2804] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:54.475205  5980 raft_consensus.cc:493] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.475541  5980 raft_consensus.cc:3060] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.473371  5972 consensus_queue.cc:260] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:54.487555  5972 ts_tablet_manager.cc:1434] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.085s	user 0.007s	sys 0.001s
I20260709 13:07:54.488868  5972 tablet_bootstrap.cc:492] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:54.488731  5911 tablet_service.cc:1511] Processing CreateTablet for tablet d76ed139f0d94b8484f5b5101634fb1b (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:07:54.491158  5911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d76ed139f0d94b8484f5b5101634fb1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.493459  5980 raft_consensus.cc:515] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:54.498360  5973 tablet_bootstrap.cc:492] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:54.503786  5973 ts_tablet_manager.cc:1403] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.183s	user 0.002s	sys 0.034s
I20260709 13:07:54.507668  5973 raft_consensus.cc:359] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:54.511190  5973 raft_consensus.cc:385] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.511615  5973 raft_consensus.cc:740] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.514371  5973 consensus_queue.cc:260] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:54.517213  5973 ts_tablet_manager.cc:1434] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.006s
I20260709 13:07:54.518388  5973 tablet_bootstrap.cc:492] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:54.508744  5597 catalog_manager.cc:5697] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:54.528607  5762 tablet_service.cc:1511] Processing CreateTablet for tablet d8f9da7df7034a9f99e973517496cd22 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:07:54.529978  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f9da7df7034a9f99e973517496cd22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.532140  5973 tablet_bootstrap.cc:654] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.538053  5980 leader_election.cc:290] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:54.538291  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:54.539214  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:54.539903  5701 raft_consensus.cc:3060] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:54.540066  5742 leader_election.cc:343] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.544387  5686 tablet_service.cc:1511] Processing CreateTablet for tablet d76ed139f0d94b8484f5b5101634fb1b (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:07:54.547122  5841 tablet_service.cc:1511] Processing CreateTablet for tablet d8f9da7df7034a9f99e973517496cd22 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:07:54.549321  5913 tablet_service.cc:1511] Processing CreateTablet for tablet d8f9da7df7034a9f99e973517496cd22 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:07:54.550354  5971 ts_tablet_manager.cc:1434] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.192s	user 0.002s	sys 0.009s
I20260709 13:07:54.551115  5913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f9da7df7034a9f99e973517496cd22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.555323  5970 tablet_bootstrap.cc:654] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.558734  5686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d76ed139f0d94b8484f5b5101634fb1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.559751  5971 tablet_bootstrap.cc:492] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:54.563560  5841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f9da7df7034a9f99e973517496cd22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.567003  5701 raft_consensus.cc:2468] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:07:54.568624  5739 leader_election.cc:304] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:54.569494  5980 raft_consensus.cc:2804] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:54.570039  5980 raft_consensus.cc:697] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:07:54.570915  5980 consensus_queue.cc:237] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:54.593516  5971 tablet_bootstrap.cc:654] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.594573  5972 tablet_bootstrap.cc:654] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.603839  5688 tablet_service.cc:1511] Processing CreateTablet for tablet deecf916d19d4f5ca9ac8200dc931c5c (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:07:54.605568  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deecf916d19d4f5ca9ac8200dc931c5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.618981  5761 tablet_service.cc:1511] Processing CreateTablet for tablet deecf916d19d4f5ca9ac8200dc931c5c (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:07:54.623245  5973 tablet_bootstrap.cc:492] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:54.623894  5973 ts_tablet_manager.cc:1403] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.106s	user 0.020s	sys 0.002s
I20260709 13:07:54.623558  5835 tablet_service.cc:1511] Processing CreateTablet for tablet deecf916d19d4f5ca9ac8200dc931c5c (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:07:54.626060  5835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deecf916d19d4f5ca9ac8200dc931c5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.626822  5973 raft_consensus.cc:359] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:54.627403  5973 raft_consensus.cc:385] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.627641  5973 raft_consensus.cc:740] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.635064  5761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deecf916d19d4f5ca9ac8200dc931c5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.645224  5980 consensus_queue.cc:1048] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:07:54.649061  5841 tablet_service.cc:1511] Processing CreateTablet for tablet 3e04051483df4ae3bf4c3f5ab037fda0 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:07:54.650444  5841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e04051483df4ae3bf4c3f5ab037fda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.656627  5686 tablet_service.cc:1511] Processing CreateTablet for tablet dcc03c962c314adf95650bc3bf8e9d34 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:07:54.658087  5686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcc03c962c314adf95650bc3bf8e9d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.666359  5911 tablet_service.cc:1511] Processing CreateTablet for tablet dcc03c962c314adf95650bc3bf8e9d34 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:07:54.668591  5911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcc03c962c314adf95650bc3bf8e9d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.639791  5973 consensus_queue.cc:260] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:54.675877  5970 tablet_bootstrap.cc:492] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:54.677210  5762 tablet_service.cc:1511] Processing CreateTablet for tablet dcc03c962c314adf95650bc3bf8e9d34 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:07:54.679896  5972 tablet_bootstrap.cc:492] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:54.686632  5972 ts_tablet_manager.cc:1403] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.198s	user 0.015s	sys 0.017s
I20260709 13:07:54.693101  5913 tablet_service.cc:1511] Processing CreateTablet for tablet 3e04051483df4ae3bf4c3f5ab037fda0 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:07:54.695341  5913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e04051483df4ae3bf4c3f5ab037fda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.695443  5691 tablet_service.cc:1511] Processing CreateTablet for tablet 3e04051483df4ae3bf4c3f5ab037fda0 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:07:54.695791  5765 tablet_service.cc:1511] Processing CreateTablet for tablet 440b21f4e5f94cf2a96ffb8b2a631d60 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:07:54.696902  5691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e04051483df4ae3bf4c3f5ab037fda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.689062  5970 ts_tablet_manager.cc:1403] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.277s	user 0.031s	sys 0.021s
I20260709 13:07:54.705385  5838 tablet_service.cc:1511] Processing CreateTablet for tablet 440b21f4e5f94cf2a96ffb8b2a631d60 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:07:54.706843  5838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 440b21f4e5f94cf2a96ffb8b2a631d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.708178  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcc03c962c314adf95650bc3bf8e9d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.709189  5973 ts_tablet_manager.cc:1434] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.085s	user 0.007s	sys 0.001s
I20260709 13:07:54.710965  5973 tablet_bootstrap.cc:492] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:54.709758  5765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 440b21f4e5f94cf2a96ffb8b2a631d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.711138  5914 tablet_service.cc:1511] Processing CreateTablet for tablet 440b21f4e5f94cf2a96ffb8b2a631d60 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:07:54.712539  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 440b21f4e5f94cf2a96ffb8b2a631d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.717731  5972 raft_consensus.cc:359] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:54.718611  5972 raft_consensus.cc:385] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.718963  5972 raft_consensus.cc:740] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.724874  5970 raft_consensus.cc:359] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.740206  5970 raft_consensus.cc:385] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.740620  5970 raft_consensus.cc:740] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.741437  5970 consensus_queue.cc:260] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.742667  5837 tablet_service.cc:1511] Processing CreateTablet for tablet 5051e38577e54ce780fad112f2fcd202 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:07:54.728300  5971 tablet_bootstrap.cc:492] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:54.744016  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5051e38577e54ce780fad112f2fcd202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.744345  5971 ts_tablet_manager.cc:1403] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.185s	user 0.015s	sys 0.013s
I20260709 13:07:54.740613  5972 consensus_queue.cc:260] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:54.749945  5910 tablet_service.cc:1511] Processing CreateTablet for tablet 0e03b5d529f34213b58b61ece24ce512 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:07:54.750422  5972 ts_tablet_manager.cc:1434] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.063s	user 0.005s	sys 0.000s
I20260709 13:07:54.747576  5971 raft_consensus.cc:359] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.751538  5971 raft_consensus.cc:385] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.751929  5971 raft_consensus.cc:740] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.752825  5971 consensus_queue.cc:260] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.743724  5970 ts_tablet_manager.cc:1434] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.000s
I20260709 13:07:54.754617  5688 tablet_service.cc:1511] Processing CreateTablet for tablet 5051e38577e54ce780fad112f2fcd202 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:07:54.755582  5766 tablet_service.cc:1511] Processing CreateTablet for tablet 5051e38577e54ce780fad112f2fcd202 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:07:54.756286  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5051e38577e54ce780fad112f2fcd202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.756955  5766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5051e38577e54ce780fad112f2fcd202. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.757972  5973 tablet_bootstrap.cc:654] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.767189  5971 ts_tablet_manager.cc:1434] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:07:54.758496  5761 tablet_service.cc:1511] Processing CreateTablet for tablet 0e03b5d529f34213b58b61ece24ce512 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:07:54.760888  6006 consensus_queue.cc:1048] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:54.761610  5683 tablet_service.cc:1511] Processing CreateTablet for tablet 0e03b5d529f34213b58b61ece24ce512 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:07:54.772235  5683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e03b5d529f34213b58b61ece24ce512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.772630  5761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e03b5d529f34213b58b61ece24ce512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.784116  5973 tablet_bootstrap.cc:492] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:54.784662  5973 ts_tablet_manager.cc:1403] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.074s	user 0.007s	sys 0.005s
I20260709 13:07:54.785005  6011 raft_consensus.cc:493] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.785549  6011 raft_consensus.cc:515] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:54.787513  5973 raft_consensus.cc:359] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.788338  5973 raft_consensus.cc:385] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:54.788635  5973 raft_consensus.cc:740] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:54.789630  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:54.790442  5925 raft_consensus.cc:2468] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:54.790482  6006 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.790613  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:54.791170  6006 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:54.791851  5667 leader_election.cc:304] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:54.792706  6008 raft_consensus.cc:2804] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:54.793301  6008 raft_consensus.cc:493] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.793687  6008 raft_consensus.cc:3060] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.763343  5597 catalog_manager.cc:5697] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 1 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:54.792712  5666 leader_election.cc:343] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.793839  6006 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:54.789570  5973 consensus_queue.cc:260] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.796072  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:54.762818  5910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e03b5d529f34213b58b61ece24ce512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.798059  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:54.787849  6011 leader_election.cc:290] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:07:54.800458  5742 leader_election.cc:343] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:54.801147  5742 leader_election.cc:343] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.801548  5742 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:54.802603  6006 raft_consensus.cc:2749] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:54.810530  6008 raft_consensus.cc:515] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:54.762177  5970 tablet_bootstrap.cc:492] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:54.848526  5983 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.849076  5983 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:54.849720  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:54.849644  5686 tablet_service.cc:1511] Processing CreateTablet for tablet b37c67a7062e4372b72045052c828e2d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:07:54.850441  5925 raft_consensus.cc:3060] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.851161  5686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b37c67a7062e4372b72045052c828e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.851164  5983 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:54.853229  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:54.853513  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:54.854130  5701 raft_consensus.cc:2468] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:07:54.854269  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:54.854998  5923 raft_consensus.cc:2468] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:07:54.856628  5913 tablet_service.cc:1511] Processing CreateTablet for tablet b37c67a7062e4372b72045052c828e2d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:07:54.857221  5762 tablet_service.cc:1511] Processing CreateTablet for tablet ffd891d1488c409d87be1300b3e91325 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:07:54.857827  5913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b37c67a7062e4372b72045052c828e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.858364  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd891d1488c409d87be1300b3e91325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.860693  5813 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:54.861485  5835 tablet_service.cc:1511] Processing CreateTablet for tablet ffd891d1488c409d87be1300b3e91325 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:07:54.862851  5835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd891d1488c409d87be1300b3e91325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.861769  5983 raft_consensus.cc:2804] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:54.863307  5983 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.863750  5983 raft_consensus.cc:3060] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.853260  5841 tablet_service.cc:1511] Processing CreateTablet for tablet b37c67a7062e4372b72045052c828e2d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
W20260709 13:07:54.865090  5666 leader_election.cc:343] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:54.865726  5841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b37c67a7062e4372b72045052c828e2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.868618  6011 consensus_queue.cc:1048] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:54.844658  5971 tablet_bootstrap.cc:492] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:54.875708  6011 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.876309  6011 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.885931  5973 ts_tablet_manager.cc:1434] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.101s	user 0.036s	sys 0.019s
I20260709 13:07:54.887401  5973 tablet_bootstrap.cc:492] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:54.894650  6031 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.895241  6031 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:54.898525  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:54.899252  5849 raft_consensus.cc:2468] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:54.900584  5667 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:54.901278  5973 tablet_bootstrap.cc:654] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.878844  6008 leader_election.cc:290] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:54.911474  6008 raft_consensus.cc:2804] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:54.915645  6008 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.916057  5972 tablet_bootstrap.cc:492] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:54.916066  6008 raft_consensus.cc:3060] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.918928  5970 tablet_bootstrap.cc:654] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.920675  6011 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:54.923195  5688 tablet_service.cc:1511] Processing CreateTablet for tablet 2bba78f968ad41538018aacddcc5d9d7 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:07:54.924670  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bba78f968ad41538018aacddcc5d9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.925076  5972 tablet_bootstrap.cc:654] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.928402  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:54.929103  5776 raft_consensus.cc:2468] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:54.930995  5971 tablet_bootstrap.cc:654] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:54.931352  5913 tablet_service.cc:1511] Processing CreateTablet for tablet ffd891d1488c409d87be1300b3e91325 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:07:54.932166  6031 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:54.932550  5913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffd891d1488c409d87be1300b3e91325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.933588  5762 tablet_service.cc:1511] Processing CreateTablet for tablet 2bba78f968ad41538018aacddcc5d9d7 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:07:54.902350  5984 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.934746  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bba78f968ad41538018aacddcc5d9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.936690  5666 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:54.938341  6033 raft_consensus.cc:493] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:54.938894  6033 raft_consensus.cc:515] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:54.941234  5925 raft_consensus.cc:2468] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:54.941982  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:54.960685  5701 raft_consensus.cc:2468] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:07:54.965969  5813 leader_election.cc:304] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:54.974527  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:54.974988  5691 tablet_service.cc:1511] Processing CreateTablet for tablet 4890d4f381d74b37a99616c1b1dc65d4 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:07:54.975293  5761 tablet_service.cc:1511] Processing CreateTablet for tablet 4890d4f381d74b37a99616c1b1dc65d4 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:07:54.954228  6008 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:54.976776  5841 tablet_service.cc:1511] Processing CreateTablet for tablet 2bba78f968ad41538018aacddcc5d9d7 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:07:54.977955  5841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bba78f968ad41538018aacddcc5d9d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.977823  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:54.952559  5984 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:54.980746  5984 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:54.981040  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:54.980669  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:54.982084  5848 raft_consensus.cc:3060] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.976807  5761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4890d4f381d74b37a99616c1b1dc65d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.982487  5925 raft_consensus.cc:2468] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:54.958323  6031 raft_consensus.cc:2804] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:54.981276  5667 leader_election.cc:304] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:07:54.986933  6033 leader_election.cc:290] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:54.987273  6031 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:54.993415  6031 raft_consensus.cc:3060] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:54.996227  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:54.995990  6033 raft_consensus.cc:2804] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:54.977905  6008 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:55.012015  5913 tablet_service.cc:1511] Processing CreateTablet for tablet 4890d4f381d74b37a99616c1b1dc65d4 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:07:55.013800  5913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4890d4f381d74b37a99616c1b1dc65d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.006954  5691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4890d4f381d74b37a99616c1b1dc65d4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:55.018407  5667 leader_election.cc:343] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.007387  6011 raft_consensus.cc:2804] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:55.026942  6011 raft_consensus.cc:697] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:55.027849  6011 consensus_queue.cc:237] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:54.990765  5983 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:55.003451  6033 raft_consensus.cc:493] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:55.010950  5815 leader_election.cc:343] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.032980  6033 raft_consensus.cc:3060] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.031634  5848 raft_consensus.cc:2468] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:55.033774  5683 tablet_service.cc:1511] Processing CreateTablet for tablet 11ac5c3627ca49348e39094121c135c4 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:07:55.034489  5762 tablet_service.cc:1511] Processing CreateTablet for tablet 098c5f84c14148eabebfe76f9ccfc1df (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:07:55.037992  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098c5f84c14148eabebfe76f9ccfc1df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.023118  6031 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:55.040954  5983 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:55.059316  5973 tablet_bootstrap.cc:492] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:55.061236  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:55.043635  5914 tablet_service.cc:1511] Processing CreateTablet for tablet 11ac5c3627ca49348e39094121c135c4 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:07:55.068769  5973 ts_tablet_manager.cc:1403] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.182s	user 0.012s	sys 0.004s
I20260709 13:07:55.041801  5837 tablet_service.cc:1511] Processing CreateTablet for tablet 11ac5c3627ca49348e39094121c135c4 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:07:55.068924  5775 raft_consensus.cc:3060] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.050374  5971 tablet_bootstrap.cc:492] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:55.070895  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ac5c3627ca49348e39094121c135c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.060344  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:55.048966  5683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ac5c3627ca49348e39094121c135c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.057188  6008 consensus_queue.cc:1048] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:07:55.057890  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:55.076802  5925 raft_consensus.cc:3060] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.077356  5971 ts_tablet_manager.cc:1403] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.239s	user 0.018s	sys 0.007s
I20260709 13:07:55.042104  5838 tablet_service.cc:1511] Processing CreateTablet for tablet 098c5f84c14148eabebfe76f9ccfc1df (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:07:55.086972  5838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098c5f84c14148eabebfe76f9ccfc1df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.090162  5766 tablet_service.cc:1511] Processing CreateTablet for tablet 30f9347b11ec44d1b07c475379a51881 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:07:55.091619  5766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30f9347b11ec44d1b07c475379a51881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.061189  5911 tablet_service.cc:1511] Processing CreateTablet for tablet 098c5f84c14148eabebfe76f9ccfc1df (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:07:55.104436  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:55.104902  5701 raft_consensus.cc:2393] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:07:55.105216  5700 raft_consensus.cc:3060] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.112018  5973 raft_consensus.cc:359] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:55.112751  5973 raft_consensus.cc:385] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.113008  5973 raft_consensus.cc:740] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.113387  5912 tablet_service.cc:1511] Processing CreateTablet for tablet 12df4d7a5aad4ece999b2b3722b294e5 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:07:55.114023  5973 consensus_queue.cc:260] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:55.113387  5835 tablet_service.cc:1511] Processing CreateTablet for tablet 30f9347b11ec44d1b07c475379a51881 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:07:55.060981  5667 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:55.116040  6008 raft_consensus.cc:2804] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:55.116297  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ac5c3627ca49348e39094121c135c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.116698  6008 raft_consensus.cc:697] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:55.117702  6008 consensus_queue.cc:237] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:55.119660  5911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098c5f84c14148eabebfe76f9ccfc1df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.095745  6031 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:55.116032  5835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30f9347b11ec44d1b07c475379a51881. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:55.116187  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 55295.1us
I20260709 13:07:55.116060  5972 tablet_bootstrap.cc:492] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:55.122740  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:55.111164  5688 tablet_service.cc:1511] Processing CreateTablet for tablet 30f9347b11ec44d1b07c475379a51881 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:07:55.124842  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30f9347b11ec44d1b07c475379a51881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.122743  5912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12df4d7a5aad4ece999b2b3722b294e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.126819  5971 raft_consensus.cc:359] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:55.127766  5971 raft_consensus.cc:385] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.128108  5971 raft_consensus.cc:740] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.128887  5971 consensus_queue.cc:260] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:55.131198  5971 ts_tablet_manager.cc:1434] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.000s
W20260709 13:07:55.133158  5667 leader_election.cc:343] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.136593  5973 ts_tablet_manager.cc:1434] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.001s
I20260709 13:07:55.140216  5691 tablet_service.cc:1511] Processing CreateTablet for tablet 12df4d7a5aad4ece999b2b3722b294e5 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:07:55.141736  5691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12df4d7a5aad4ece999b2b3722b294e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.123423  5972 ts_tablet_manager.cc:1403] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.219s	user 0.008s	sys 0.043s
I20260709 13:07:55.146123  5973 tablet_bootstrap.cc:492] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:55.178166  5971 tablet_bootstrap.cc:492] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:55.202322  5972 raft_consensus.cc:359] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:55.203161  5972 raft_consensus.cc:385] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.203496  5972 raft_consensus.cc:740] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.206959  5970 tablet_bootstrap.cc:492] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:55.207545  5970 ts_tablet_manager.cc:1403] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.446s	user 0.016s	sys 0.014s
I20260709 13:07:55.211088  6033 raft_consensus.cc:515] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:55.214496  6033 leader_election.cc:290] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:55.180483  5925 raft_consensus.cc:2468] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:07:55.221864  5775 raft_consensus.cc:2468] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:55.222620  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:55.228533  5851 raft_consensus.cc:2393] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:07:55.229832  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:55.230136  5891 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:55.231467  5984 raft_consensus.cc:3060] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.232300  5970 raft_consensus.cc:359] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:55.233132  5970 raft_consensus.cc:385] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.233453  5970 raft_consensus.cc:740] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.234259  5970 consensus_queue.cc:260] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:55.239627  5984 raft_consensus.cc:2749] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:55.241374  5815 leader_election.cc:343] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:55.242434  5742 consensus_peers.cc:597] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Couldn't send request to peer 792907f253c744aa9392e8d572f88e06. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:55.244874  5972 consensus_queue.cc:260] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:55.247614  5972 ts_tablet_manager.cc:1434] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.095s	user 0.007s	sys 0.000s
I20260709 13:07:55.252099  5816 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:07:55.253113  6033 raft_consensus.cc:2804] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:55.253654  6033 raft_consensus.cc:697] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:07:55.254545  6033 consensus_queue.cc:237] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:55.258122  5972 tablet_bootstrap.cc:492] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:55.260465  5971 tablet_bootstrap.cc:654] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.260550  6035 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.261349  6035 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:55.265259  5973 tablet_bootstrap.cc:654] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.266809  5972 tablet_bootstrap.cc:654] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.269398  5700 raft_consensus.cc:2468] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:07:55.272209  6035 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:55.277448  5970 ts_tablet_manager.cc:1434] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.000s
I20260709 13:07:55.278038  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:55.279203  5775 raft_consensus.cc:2393] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:07:55.279276  5970 tablet_bootstrap.cc:492] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:55.301949  5972 tablet_bootstrap.cc:492] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:55.302590  5972 ts_tablet_manager.cc:1403] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.007s
I20260709 13:07:55.318161  5970 tablet_bootstrap.cc:654] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.321300  5838 tablet_service.cc:1511] Processing CreateTablet for tablet de4f21d87306453b81a417014e41e4a4 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:07:55.324108  5838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de4f21d87306453b81a417014e41e4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.326761  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:55.327718  5701 raft_consensus.cc:2393] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:07:55.328609  5972 raft_consensus.cc:359] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:55.328881  5914 tablet_service.cc:1511] Processing CreateTablet for tablet de4f21d87306453b81a417014e41e4a4 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:07:55.329386  5972 raft_consensus.cc:385] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.329280  5813 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:55.329855  5972 raft_consensus.cc:740] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.330117  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de4f21d87306453b81a417014e41e4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.330643  6033 raft_consensus.cc:3060] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.330830  5972 consensus_queue.cc:260] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:55.333329  5972 ts_tablet_manager.cc:1434] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 13:07:55.334417  5972 tablet_bootstrap.cc:492] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:55.339313  6033 raft_consensus.cc:2749] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:55.341285  5972 tablet_bootstrap.cc:654] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.342607  5971 tablet_bootstrap.cc:492] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:55.343266  5971 ts_tablet_manager.cc:1403] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.192s	user 0.008s	sys 0.009s
I20260709 13:07:55.344568  5766 tablet_service.cc:1511] Processing CreateTablet for tablet 12df4d7a5aad4ece999b2b3722b294e5 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:07:55.345968  5766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12df4d7a5aad4ece999b2b3722b294e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.348709  5971 raft_consensus.cc:359] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:55.349802  5971 raft_consensus.cc:385] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.350203  5971 raft_consensus.cc:740] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.361342  5666 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:55.374954  6011 raft_consensus.cc:2804] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:55.375574  6011 raft_consensus.cc:697] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:55.376570  6011 consensus_queue.cc:237] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:55.392169  5973 tablet_bootstrap.cc:492] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:55.392824  5973 ts_tablet_manager.cc:1403] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.251s	user 0.012s	sys 0.001s
I20260709 13:07:55.394630  5838 tablet_service.cc:1511] Processing CreateTablet for tablet 057217b46a0d4d188d28fecbdf744762 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:07:55.426864  5691 tablet_service.cc:1511] Processing CreateTablet for tablet de4f21d87306453b81a417014e41e4a4 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:07:55.428673  5688 tablet_service.cc:1511] Processing CreateTablet for tablet 8842506bb6124bf9adf9b13c6300c796 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:07:55.363246  5970 tablet_bootstrap.cc:492] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:55.642525  5970 ts_tablet_manager.cc:1403] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.364s	user 0.018s	sys 0.003s
W20260709 13:07:55.374015  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:55.585052  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:55.644807  5700 raft_consensus.cc:3060] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.645699  5970 raft_consensus.cc:359] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:55.646502  5970 raft_consensus.cc:385] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.646792  5970 raft_consensus.cc:740] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.585588  5691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de4f21d87306453b81a417014e41e4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.586414  5835 tablet_service.cc:1511] Processing CreateTablet for tablet 8842506bb6124bf9adf9b13c6300c796 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:07:55.609364  5766 tablet_service.cc:1511] Processing CreateTablet for tablet 057217b46a0d4d188d28fecbdf744762 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:07:55.650636  5766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 057217b46a0d4d188d28fecbdf744762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.651389  6031 consensus_queue.cc:1048] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:55.610411  5762 tablet_service.cc:1511] Processing CreateTablet for tablet 8842506bb6124bf9adf9b13c6300c796 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:07:55.653805  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8842506bb6124bf9adf9b13c6300c796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.610872  5973 raft_consensus.cc:359] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:55.655164  5973 raft_consensus.cc:385] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.655638  5973 raft_consensus.cc:740] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.656785  5973 consensus_queue.cc:260] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:55.657407  5970 consensus_queue.cc:260] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:55.659300  5973 ts_tablet_manager.cc:1434] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.266s	user 0.008s	sys 0.000s
I20260709 13:07:55.620137  6006 raft_consensus.cc:493] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.620141  6008 raft_consensus.cc:493] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.659986  5970 ts_tablet_manager.cc:1434] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.012s
I20260709 13:07:55.660194  6006 raft_consensus.cc:515] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:55.660305  6008 raft_consensus.cc:515] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:55.661146  5970 tablet_bootstrap.cc:492] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:55.627616  5838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 057217b46a0d4d188d28fecbdf744762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.662524  6006 leader_election.cc:290] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:55.629040  5984 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.629058  6011 consensus_queue.cc:1048] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:55.668805  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:55.671212  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:55.672516  5851 raft_consensus.cc:2393] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:07:55.673121  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:55.673429  6034 consensus_queue.cc:1048] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:07:55.675673  5667 leader_election.cc:343] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.630070  5980 consensus_queue.cc:1048] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:55.631203  5666 consensus_peers.cc:597] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:55.676985  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 45925.9us
I20260709 13:07:55.677007  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:55.635248  5972 tablet_bootstrap.cc:492] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:55.677945  5972 ts_tablet_manager.cc:1403] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.344s	user 0.034s	sys 0.027s
I20260709 13:07:55.667438  5984 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:55.662647  6008 leader_election.cc:290] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:55.680352  5742 leader_election.cc:304] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:55.680847  5835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8842506bb6124bf9adf9b13c6300c796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.681331  5972 raft_consensus.cc:359] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:55.682097  5972 raft_consensus.cc:385] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.682418  5972 raft_consensus.cc:740] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.682597  5911 tablet_service.cc:1511] Processing CreateTablet for tablet 27bf52f47b87456c94c45846de53788d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:07:55.683346  5980 raft_consensus.cc:3060] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.641640  5597 catalog_manager.cc:5697] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:55.684001  5911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27bf52f47b87456c94c45846de53788d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.664049  5973 tablet_bootstrap.cc:492] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:55.683239  5972 consensus_queue.cc:260] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:55.361815  5971 consensus_queue.cc:260] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:55.665560  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8842506bb6124bf9adf9b13c6300c796. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:55.688438  5667 leader_election.cc:343] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.764218  5667 leader_election.cc:304] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:55.764488  5762 tablet_service.cc:1511] Processing CreateTablet for tablet b1e618980cb143c69eb4b401ba701662 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
W20260709 13:07:55.765249  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 76864.1us
I20260709 13:07:55.765775  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1e618980cb143c69eb4b401ba701662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.690245  5984 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:55.691584  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:55.768503  6034 raft_consensus.cc:2749] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:55.704151  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:55.770543  5775 raft_consensus.cc:2468] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
W20260709 13:07:55.771322  5891 leader_election.cc:343] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.716830  5683 tablet_service.cc:1511] Processing CreateTablet for tablet 27bf52f47b87456c94c45846de53788d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:07:55.772737  5683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27bf52f47b87456c94c45846de53788d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.718010  5761 tablet_service.cc:1511] Processing CreateTablet for tablet 27bf52f47b87456c94c45846de53788d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:07:55.776772  5761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27bf52f47b87456c94c45846de53788d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.719244  5837 tablet_service.cc:1511] Processing CreateTablet for tablet c1280a1f51344b42b4410fc2852da38d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:07:55.780705  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1280a1f51344b42b4410fc2852da38d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.699666  5700 raft_consensus.cc:2468] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:07:55.720767  5595 catalog_manager.cc:5697] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 1 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:55.783051  5984 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.783625  5984 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:55.721499  5912 tablet_service.cc:1511] Processing CreateTablet for tablet c1280a1f51344b42b4410fc2852da38d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:07:55.749756  5970 tablet_bootstrap.cc:654] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.784315  5813 leader_election.cc:304] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:07:55.785621  5912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1280a1f51344b42b4410fc2852da38d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.752180  5838 tablet_service.cc:1511] Processing CreateTablet for tablet b1e618980cb143c69eb4b401ba701662 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:07:55.754225  5980 raft_consensus.cc:2749] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:55.756357  5686 tablet_service.cc:1511] Processing CreateTablet for tablet c1280a1f51344b42b4410fc2852da38d (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:07:55.756846  6057 raft_consensus.cc:493] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.761974  5972 ts_tablet_manager.cc:1434] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.084s	user 0.011s	sys 0.014s
I20260709 13:07:55.706265  5914 tablet_service.cc:1511] Processing CreateTablet for tablet 057217b46a0d4d188d28fecbdf744762 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:07:55.800870  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 057217b46a0d4d188d28fecbdf744762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.789501  5971 ts_tablet_manager.cc:1434] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.446s	user 0.008s	sys 0.007s
I20260709 13:07:55.816177  5890 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:55.810813  6033 raft_consensus.cc:2804] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:55.822903  5970 tablet_bootstrap.cc:492] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:55.823472  5970 ts_tablet_manager.cc:1403] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.163s	user 0.020s	sys 0.011s
I20260709 13:07:55.824271  6059 raft_consensus.cc:2804] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.824699  6059 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.825035  6059 raft_consensus.cc:3060] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.789674  6034 consensus_queue.cc:1048] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:07:55.836624  6035 raft_consensus.cc:493] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.840579  6035 raft_consensus.cc:515] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:55.841113  5686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1280a1f51344b42b4410fc2852da38d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.813016  6057 raft_consensus.cc:515] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:55.841150  5972 tablet_bootstrap.cc:492] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:55.854905  5970 raft_consensus.cc:359] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:55.856041  5970 raft_consensus.cc:385] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.856395  5970 raft_consensus.cc:740] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
W20260709 13:07:55.827999  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 43781.2us
I20260709 13:07:55.868070  5972 tablet_bootstrap.cc:654] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.864748  5838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1e618980cb143c69eb4b401ba701662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.876227  5970 consensus_queue.cc:260] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:55.898080  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:55.898944  5700 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:55.900012  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:55.901042  6035 leader_election.cc:290] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:55.904981  5914 tablet_service.cc:1511] Processing CreateTablet for tablet b1e618980cb143c69eb4b401ba701662 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:07:55.906589  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1e618980cb143c69eb4b401ba701662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.903462  6035 consensus_queue.cc:1048] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:07:55.923740  5888 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:55.890522  6033 raft_consensus.cc:697] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:07:55.927413  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:55.929963  5973 tablet_bootstrap.cc:654] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.933928  6033 consensus_queue.cc:237] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:55.935135  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:55.936172  6070 raft_consensus.cc:2804] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.938028  5688 tablet_service.cc:1511] Processing CreateTablet for tablet aa162ebfa5094c25a3a311044956d167 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:07:55.939596  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa162ebfa5094c25a3a311044956d167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.913887  6057 leader_election.cc:290] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:55.936638  6070 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.983296  6070 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.965690  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:55.969441  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:55.988304  5775 raft_consensus.cc:2468] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:07:55.970116  5972 tablet_bootstrap.cc:492] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:55.989270  5972 ts_tablet_manager.cc:1403] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.148s	user 0.011s	sys 0.013s
I20260709 13:07:55.991034  5815 leader_election.cc:304] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:07:55.972229  5970 ts_tablet_manager.cc:1434] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.148s	user 0.011s	sys 0.007s
I20260709 13:07:55.992201  6074 raft_consensus.cc:2804] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.992588  6074 raft_consensus.cc:493] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.992911  6074 raft_consensus.cc:3060] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.993012  5970 tablet_bootstrap.cc:492] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:55.993359  5837 tablet_service.cc:1511] Processing CreateTablet for tablet aa162ebfa5094c25a3a311044956d167 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:07:55.995095  5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa162ebfa5094c25a3a311044956d167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.971196  6059 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:55.997983  6059 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:55.978518  5761 tablet_service.cc:1511] Processing CreateTablet for tablet aa162ebfa5094c25a3a311044956d167 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:07:55.998538  5923 raft_consensus.cc:3060] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.001266  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:56.001467  5914 tablet_service.cc:1511] Processing CreateTablet for tablet 3e83357d1cc74a0292e36bd8947af234 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:07:56.002673  5914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e83357d1cc74a0292e36bd8947af234. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:56.002856  5891 leader_election.cc:343] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.004680  5835 tablet_service.cc:1511] Processing CreateTablet for tablet 3ada8ac4d923408997f9511c6910789c (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:07:56.005756  5761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa162ebfa5094c25a3a311044956d167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.005673  5890 leader_election.cc:304] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:56.007200  5972 raft_consensus.cc:359] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.007658  6035 consensus_queue.cc:1048] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.005721  5762 tablet_service.cc:1511] Processing CreateTablet for tablet 3e83357d1cc74a0292e36bd8947af234 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:07:56.008486  5683 tablet_service.cc:1511] Processing CreateTablet for tablet 3e83357d1cc74a0292e36bd8947af234 (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:07:56.009579  5762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e83357d1cc74a0292e36bd8947af234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.009809  5683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e83357d1cc74a0292e36bd8947af234. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:56.012097  5816 leader_election.cc:343] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.982846  5984 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:56.006086  5835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ada8ac4d923408997f9511c6910789c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.981493  5971 tablet_bootstrap.cc:492] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:56.017846  6074 raft_consensus.cc:515] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:07:56.020421  5890 leader_election.cc:343] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.021718  6034 consensus_queue.cc:1048] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.036285  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:56.037834  5776 raft_consensus.cc:3060] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.040306  6074 leader_election.cc:290] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:56.008013  5972 raft_consensus.cc:385] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.046964  5972 raft_consensus.cc:740] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.048054  5972 consensus_queue.cc:260] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.052852  5972 ts_tablet_manager.cc:1434] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.000s
I20260709 13:07:56.054196  5972 tablet_bootstrap.cc:492] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:56.042516  5970 tablet_bootstrap.cc:654] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.022074  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:56.056501  5775 raft_consensus.cc:3060] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.045310  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:56.055684  6073 raft_consensus.cc:493] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.061573  6073 raft_consensus.cc:515] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.068773  6073 leader_election.cc:290] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:56.070050  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:56.071007  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:56.073314  5742 leader_election.cc:304] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:56.073962  5971 tablet_bootstrap.cc:654] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:56.075346  5816 leader_election.cc:343] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.075368  6073 raft_consensus.cc:2749] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.081854  5775 raft_consensus.cc:2468] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:56.082056  5912 tablet_service.cc:1511] Processing CreateTablet for tablet 3ada8ac4d923408997f9511c6910789c (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:07:56.086756  5688 tablet_service.cc:1511] Processing CreateTablet for tablet 3ada8ac4d923408997f9511c6910789c (DEFAULT_TABLE table=test-workload [id=893410cbe7174f71a91ee5b45c5d6037]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:07:56.088829  5688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ada8ac4d923408997f9511c6910789c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.097110  5912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ada8ac4d923408997f9511c6910789c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:56.131860  5970 tablet_bootstrap.cc:492] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:56.132789  5970 ts_tablet_manager.cc:1403] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.140s	user 0.010s	sys 0.006s
I20260709 13:07:56.166482  6031 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.171566  6031 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.175953  6031 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:56.178252  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:56.179680  5972 tablet_bootstrap.cc:654] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.186556  5970 raft_consensus.cc:359] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.186939  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:56.187351  5970 raft_consensus.cc:385] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.187676  5970 raft_consensus.cc:740] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.188565  5970 consensus_queue.cc:260] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:07:56.190284  5666 leader_election.cc:343] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.059670  6070 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.206564  5970 ts_tablet_manager.cc:1434] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.073s	user 0.006s	sys 0.004s
I20260709 13:07:56.207582  5970 tablet_bootstrap.cc:492] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:56.178874  5890 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:56.217913  6075 raft_consensus.cc:2804] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.218475  6075 raft_consensus.cc:697] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
W20260709 13:07:56.208185  5667 leader_election.cc:343] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.224437  5973 tablet_bootstrap.cc:492] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:56.224993  5973 ts_tablet_manager.cc:1403] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.561s	user 0.025s	sys 0.001s
I20260709 13:07:56.225809  5775 raft_consensus.cc:3060] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.228039  5667 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
W20260709 13:07:56.228864  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 50091.3us
I20260709 13:07:56.232585  6011 raft_consensus.cc:2749] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.219329  6075 consensus_queue.cc:237] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.243435  6070 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:56.243937  6059 raft_consensus.cc:2749] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.245718  5776 raft_consensus.cc:2468] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:07:56.247407  5815 leader_election.cc:304] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:56.258442  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:56.259146  5701 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.242713  5973 raft_consensus.cc:359] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.263201  5973 raft_consensus.cc:385] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.264751  5973 raft_consensus.cc:740] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.269853  5973 consensus_queue.cc:260] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.258378  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:56.276168  6033 raft_consensus.cc:2804] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.275503  5596 catalog_manager.cc:5697] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } }
I20260709 13:07:56.277464  6033 raft_consensus.cc:697] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:07:56.276623  5776 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.285774  6033 consensus_queue.cc:237] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.280469  5973 ts_tablet_manager.cc:1434] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.002s
I20260709 13:07:56.289173  5973 tablet_bootstrap.cc:492] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:56.282032  5972 tablet_bootstrap.cc:492] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:56.281944  5701 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:56.284125  5970 tablet_bootstrap.cc:654] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.290475  5972 ts_tablet_manager.cc:1403] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.237s	user 0.041s	sys 0.024s
I20260709 13:07:56.295516  5599 catalog_manager.cc:5697] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } }
I20260709 13:07:56.297819  5888 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:56.302246  5972 raft_consensus.cc:359] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.302811  6075 raft_consensus.cc:2804] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.302942  5973 tablet_bootstrap.cc:654] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.303035  5972 raft_consensus.cc:385] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.303694  5972 raft_consensus.cc:740] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.303781  6075 raft_consensus.cc:697] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:07:56.304497  5972 consensus_queue.cc:260] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.304752  6075 consensus_queue.cc:237] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.304070  5599 catalog_manager.cc:5697] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 1 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:56.307091  5972 ts_tablet_manager.cc:1434] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:56.298549  5776 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:56.308178  5972 tablet_bootstrap.cc:492] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:56.313074  5971 tablet_bootstrap.cc:492] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:56.313952  5971 ts_tablet_manager.cc:1403] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.333s	user 0.016s	sys 0.005s
I20260709 13:07:56.323037  5595 catalog_manager.cc:5697] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 1 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:56.330034  5972 tablet_bootstrap.cc:654] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.332149  5970 tablet_bootstrap.cc:492] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:56.332693  5970 ts_tablet_manager.cc:1403] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.125s	user 0.003s	sys 0.009s
I20260709 13:07:56.334398  5971 raft_consensus.cc:359] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.335484  5971 raft_consensus.cc:385] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.335807  5971 raft_consensus.cc:740] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.336612  5971 consensus_queue.cc:260] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.339133  5971 ts_tablet_manager.cc:1434] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:07:56.340204  5971 tablet_bootstrap.cc:492] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:56.345805  5973 tablet_bootstrap.cc:492] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:56.346421  5973 ts_tablet_manager.cc:1403] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.057s	user 0.021s	sys 0.012s
I20260709 13:07:56.346509  5971 tablet_bootstrap.cc:654] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.349416  5973 raft_consensus.cc:359] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.350409  5973 raft_consensus.cc:385] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.350750  5973 raft_consensus.cc:740] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.351981  5973 consensus_queue.cc:260] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.356146  5970 raft_consensus.cc:359] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.357244  5970 raft_consensus.cc:385] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.357589  5970 raft_consensus.cc:740] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.358927  5971 tablet_bootstrap.cc:492] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:56.358536  5970 consensus_queue.cc:260] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.359565  5971 ts_tablet_manager.cc:1403] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.005s
I20260709 13:07:56.361306  5599 catalog_manager.cc:5697] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 1 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.362648  5971 raft_consensus.cc:359] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.363410  5971 raft_consensus.cc:385] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.363726  5971 raft_consensus.cc:740] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.363523  5595 catalog_manager.cc:5697] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 1 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.368034  5970 ts_tablet_manager.cc:1434] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:07:56.364663  5971 consensus_queue.cc:260] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.369058  5970 tablet_bootstrap.cc:492] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:56.366010  5973 ts_tablet_manager.cc:1434] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:07:56.370132  5973 tablet_bootstrap.cc:492] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:56.375162  6033 consensus_queue.cc:1048] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.376905  5970 tablet_bootstrap.cc:654] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.377862  5973 tablet_bootstrap.cc:654] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.379070  6075 raft_consensus.cc:493] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.379604  6075 raft_consensus.cc:515] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.381791  6033 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.382303  6033 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.383947  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:56.384639  5700 raft_consensus.cc:2468] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:56.384917  6033 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:56.385202  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:56.386010  5775 raft_consensus.cc:2393] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:07:56.386070  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:56.386245  5888 leader_election.cc:304] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:56.386978  5972 tablet_bootstrap.cc:492] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:56.387190  6059 raft_consensus.cc:2804] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.387460  5972 ts_tablet_manager.cc:1403] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.080s	user 0.011s	sys 0.031s
I20260709 13:07:56.387544  6059 raft_consensus.cc:493] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.388068  6059 raft_consensus.cc:3060] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.389144  5971 ts_tablet_manager.cc:1434] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.003s
I20260709 13:07:56.390152  5971 tablet_bootstrap.cc:492] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:56.388753  5816 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:56.393172  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:56.394052  5848 raft_consensus.cc:2468] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:56.395660  6075 leader_election.cc:290] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:56.396106  6065 raft_consensus.cc:3060] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.396457  5972 raft_consensus.cc:359] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.397173  5972 raft_consensus.cc:385] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.397454  5972 raft_consensus.cc:740] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.398298  5972 consensus_queue.cc:260] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.403712  6065 raft_consensus.cc:2749] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.405481  5971 tablet_bootstrap.cc:654] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.406541  6074 consensus_queue.cc:1048] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.408766  5972 ts_tablet_manager.cc:1434] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:07:56.409787  5972 tablet_bootstrap.cc:492] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:56.412840  5970 tablet_bootstrap.cc:492] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:56.413336  5970 ts_tablet_manager.cc:1403] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.044s	user 0.006s	sys 0.019s
I20260709 13:07:56.414623  6059 raft_consensus.cc:515] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.417110  5970 raft_consensus.cc:359] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.418524  6059 leader_election.cc:290] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:56.419587  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:56.418108  5970 raft_consensus.cc:385] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.419770  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:56.428036  5970 raft_consensus.cc:740] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.428184  5701 raft_consensus.cc:3060] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.429127  5972 tablet_bootstrap.cc:654] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.420598  5848 raft_consensus.cc:3060] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.444092  5970 consensus_queue.cc:260] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.447608  5970 ts_tablet_manager.cc:1434] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.000s
I20260709 13:07:56.449003  5970 tablet_bootstrap.cc:492] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:56.457406  5848 raft_consensus.cc:2468] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:56.458835  5891 leader_election.cc:304] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:07:56.459784  6059 raft_consensus.cc:2804] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.460314  6059 raft_consensus.cc:697] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:07:56.461488  6059 consensus_queue.cc:237] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.469269  5701 raft_consensus.cc:2468] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:56.471774  5973 tablet_bootstrap.cc:492] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:56.479025  5970 tablet_bootstrap.cc:654] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.483115  5970 tablet_bootstrap.cc:492] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:56.483469  5970 ts_tablet_manager.cc:1403] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.035s	user 0.000s	sys 0.010s
I20260709 13:07:56.485101  6059 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.485504  5970 raft_consensus.cc:359] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.486057  5970 raft_consensus.cc:385] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.486255  5970 raft_consensus.cc:740] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.485819  6059 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.486822  5970 consensus_queue.cc:260] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.489230  5970 ts_tablet_manager.cc:1434] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.005s
I20260709 13:07:56.490145  5970 tablet_bootstrap.cc:492] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:56.490108  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:56.491075  5775 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:56.492749  5890 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:07:56.493961  6075 raft_consensus.cc:2804] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.494443  6075 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.494841  6075 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.497020  5970 tablet_bootstrap.cc:654] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.501811  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:56.502513  5701 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:56.505571  5595 catalog_manager.cc:5697] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 1 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.508150  6059 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:07:56.509150  5722 debug-util.cc:398] Leaking SignalData structure 0x7b0800060a20 after lost signal to thread 27937
W20260709 13:07:56.510428  5722 debug-util.cc:398] Leaking SignalData structure 0x7b080026a340 after lost signal to thread 5633
W20260709 13:07:56.511865  5722 debug-util.cc:398] Leaking SignalData structure 0x7b0800199680 after lost signal to thread 5725
W20260709 13:07:56.513120  5722 debug-util.cc:398] Leaking SignalData structure 0x7b0800110180 after lost signal to thread 5800
W20260709 13:07:56.514372  5722 debug-util.cc:398] Leaking SignalData structure 0x7b08003145e0 after lost signal to thread 5875
W20260709 13:07:56.515637  5722 debug-util.cc:398] Leaking SignalData structure 0x7b0800016460 after lost signal to thread 5949
I20260709 13:07:56.524750  5970 tablet_bootstrap.cc:492] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:56.525285  5970 ts_tablet_manager.cc:1403] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.008s
I20260709 13:07:56.526772  5973 ts_tablet_manager.cc:1403] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.157s	user 0.015s	sys 0.002s
I20260709 13:07:56.526480  6075 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.528120  5970 raft_consensus.cc:359] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.529788  5973 raft_consensus.cc:359] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.530081  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:56.530844  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:56.531366  5701 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.531621  5775 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.539969  5970 raft_consensus.cc:385] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.540138  6075 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:56.540292  5973 raft_consensus.cc:385] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.540443  5970 raft_consensus.cc:740] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.541095  5973 raft_consensus.cc:740] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.540202  5701 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:56.541592  5970 consensus_queue.cc:260] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.542562  5973 consensus_queue.cc:260] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.543305  5888 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:56.544582  5775 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:56.545135  6075 raft_consensus.cc:2804] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.545852  5971 tablet_bootstrap.cc:492] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:56.546363  5971 ts_tablet_manager.cc:1403] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.156s	user 0.014s	sys 0.051s
I20260709 13:07:56.546507  6075 raft_consensus.cc:697] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:07:56.547461  6075 consensus_queue.cc:237] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.549269  5971 raft_consensus.cc:359] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.550091  5971 raft_consensus.cc:385] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.550344  5971 raft_consensus.cc:740] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.551179  5973 ts_tablet_manager.cc:1434] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 13:07:56.551203  5971 consensus_queue.cc:260] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.554006  5970 ts_tablet_manager.cc:1434] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:07:56.555076  5970 tablet_bootstrap.cc:492] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:56.556206  5971 ts_tablet_manager.cc:1434] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:07:56.557729  5971 tablet_bootstrap.cc:492] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:56.553980  5973 tablet_bootstrap.cc:492] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:56.562080  5972 tablet_bootstrap.cc:492] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:56.562616  5972 ts_tablet_manager.cc:1403] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.153s	user 0.015s	sys 0.048s
I20260709 13:07:56.562732  5971 tablet_bootstrap.cc:654] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.564644  5970 tablet_bootstrap.cc:654] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.568485  5973 tablet_bootstrap.cc:654] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.570072  5972 raft_consensus.cc:359] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.570883  5972 raft_consensus.cc:385] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.571156  5972 raft_consensus.cc:740] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.574285  5972 consensus_queue.cc:260] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.576730  5972 ts_tablet_manager.cc:1434] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 13:07:56.577716  5972 tablet_bootstrap.cc:492] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:56.584894  5972 tablet_bootstrap.cc:654] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.586975  5970 tablet_bootstrap.cc:492] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:56.587595  5970 ts_tablet_manager.cc:1403] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.000s
I20260709 13:07:56.588824  5971 tablet_bootstrap.cc:492] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:56.589349  5971 ts_tablet_manager.cc:1403] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.003s
I20260709 13:07:56.590975  5970 raft_consensus.cc:359] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.591794  5970 raft_consensus.cc:385] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.592257  5971 raft_consensus.cc:359] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.592976  5971 raft_consensus.cc:385] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.593309  5971 raft_consensus.cc:740] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.592271  5970 raft_consensus.cc:740] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.594210  5971 consensus_queue.cc:260] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.595009  5970 consensus_queue.cc:260] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.597020  5597 catalog_manager.cc:5697] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 1 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.598035  6073 raft_consensus.cc:493] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.598615  6073 raft_consensus.cc:515] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.600884  6073 leader_election.cc:290] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:56.601215  5970 ts_tablet_manager.cc:1434] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:07:56.602255  5970 tablet_bootstrap.cc:492] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:56.603372  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:56.604116  5701 raft_consensus.cc:2468] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:07:56.605297  5971 ts_tablet_manager.cc:1434] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:07:56.606024  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:56.607249  5739 leader_election.cc:304] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
W20260709 13:07:56.609333  5742 leader_election.cc:343] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.614262  5971 tablet_bootstrap.cc:492] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:56.616611  5973 tablet_bootstrap.cc:492] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:56.617136  5973 ts_tablet_manager.cc:1403] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.000s
I20260709 13:07:56.619814  5970 tablet_bootstrap.cc:654] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.621157  6073 raft_consensus.cc:2804] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.621289  6059 consensus_queue.cc:1048] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.621647  6073 raft_consensus.cc:493] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.622025  6073 raft_consensus.cc:3060] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.625216  5971 tablet_bootstrap.cc:654] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.632136  6011 raft_consensus.cc:493] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.632938  6011 raft_consensus.cc:515] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.633451  5973 raft_consensus.cc:359] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.634137  5973 raft_consensus.cc:385] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.634393  5973 raft_consensus.cc:740] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.635102  6011 leader_election.cc:290] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:56.637898  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:56.638877  5848 raft_consensus.cc:2468] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:56.640715  5667 leader_election.cc:304] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:56.638015  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:56.642175  5776 raft_consensus.cc:2468] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:56.645012  6011 raft_consensus.cc:2804] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.650430  6073 raft_consensus.cc:515] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.651110  6011 raft_consensus.cc:493] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.651823  6011 raft_consensus.cc:3060] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.656441  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:56.657261  5701 raft_consensus.cc:3060] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.657603  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
W20260709 13:07:56.659029  5742 leader_election.cc:343] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.659608  6073 leader_election.cc:290] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:56.635118  5973 consensus_queue.cc:260] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.666826  6070 consensus_queue.cc:1048] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.670149  5984 raft_consensus.cc:493] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.670681  5984 raft_consensus.cc:515] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.672896  5984 leader_election.cc:290] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:56.677901  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:07:56.680025  5891 leader_election.cc:343] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.680958  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:56.681742  5775 raft_consensus.cc:2468] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:56.683247  5890 leader_election.cc:304] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:56.685292  5984 raft_consensus.cc:2804] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.685694  5984 raft_consensus.cc:493] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.686050  5984 raft_consensus.cc:3060] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.686789  5701 raft_consensus.cc:2468] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:07:56.690393  5973 ts_tablet_manager.cc:1434] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.073s	user 0.006s	sys 0.002s
I20260709 13:07:56.692739  5973 tablet_bootstrap.cc:492] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:56.698179  5972 tablet_bootstrap.cc:492] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:56.698772  5972 ts_tablet_manager.cc:1403] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.121s	user 0.012s	sys 0.038s
I20260709 13:07:56.694972  5739 leader_election.cc:304] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:56.700906  6073 raft_consensus.cc:2804] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.701392  6073 raft_consensus.cc:697] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:07:56.701911  5970 tablet_bootstrap.cc:492] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:56.702585  5970 ts_tablet_manager.cc:1403] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.101s	user 0.020s	sys 0.014s
I20260709 13:07:56.704368  6073 consensus_queue.cc:237] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.706319  5970 raft_consensus.cc:359] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.707089  5970 raft_consensus.cc:385] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.707376  5970 raft_consensus.cc:740] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.708192  5970 consensus_queue.cc:260] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.708596  5972 raft_consensus.cc:359] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.709326  5972 raft_consensus.cc:385] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.709597  5972 raft_consensus.cc:740] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.711894  5972 consensus_queue.cc:260] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.714363  6056 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.714884  6056 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.716544  5970 ts_tablet_manager.cc:1434] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:07:56.717476  5970 tablet_bootstrap.cc:492] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:56.721258  6056 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:56.721659  5971 tablet_bootstrap.cc:492] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:56.722303  5971 ts_tablet_manager.cc:1403] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.108s	user 0.013s	sys 0.009s
I20260709 13:07:56.725157  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:56.725555  5972 ts_tablet_manager.cc:1434] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:07:56.725819  5850 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:56.719313  6011 raft_consensus.cc:515] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.726588  5972 tablet_bootstrap.cc:492] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:56.728857  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:56.728953  5667 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:56.729480  5775 raft_consensus.cc:3060] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.730499  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:56.731161  5923 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:56.732733  5972 tablet_bootstrap.cc:654] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.733145  6056 raft_consensus.cc:2804] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.733423  6034 raft_consensus.cc:493] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.733621  6056 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.734076  6056 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.734015  6034 raft_consensus.cc:515] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.737038  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:56.737860  5775 raft_consensus.cc:2468] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
W20260709 13:07:56.738579  5667 leader_election.cc:343] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.739207  6011 leader_election.cc:290] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:56.740235  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:56.740893  5850 raft_consensus.cc:3060] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.741520  5666 leader_election.cc:304] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:56.741891  5984 raft_consensus.cc:515] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.742889  6074 consensus_queue.cc:1048] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.744747  5973 tablet_bootstrap.cc:654] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:56.745432  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:56.745594  6034 leader_election.cc:290] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:56.746212  6034 raft_consensus.cc:2804] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.746665  6034 raft_consensus.cc:697] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:56.749406  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:56.750239  5776 raft_consensus.cc:2468] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:56.751713  5666 leader_election.cc:304] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:56.749428  5850 raft_consensus.cc:2468] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:56.754616  6031 raft_consensus.cc:2804] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.755038  6031 raft_consensus.cc:493] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.755402  6031 raft_consensus.cc:3060] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.747611  6034 consensus_queue.cc:237] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.767063  5973 tablet_bootstrap.cc:492] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:56.767611  6056 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.768512  5597 catalog_manager.cc:5697] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 1 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.756716  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:56.773620  5776 raft_consensus.cc:3060] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.770948  5973 ts_tablet_manager.cc:1403] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.078s	user 0.012s	sys 0.002s
I20260709 13:07:56.783051  6056 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:56.787906  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:56.788636  5923 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.753396  5971 raft_consensus.cc:359] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.789316  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:56.779668  5973 raft_consensus.cc:359] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.801179  5973 raft_consensus.cc:385] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.801275  5851 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.794430  5972 tablet_bootstrap.cc:492] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:56.802243  5972 ts_tablet_manager.cc:1403] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.076s	user 0.017s	sys 0.001s
I20260709 13:07:56.802134  5923 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:56.797175  6031 raft_consensus.cc:515] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.805506  5972 raft_consensus.cc:359] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.806213  5972 raft_consensus.cc:385] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.806479  5972 raft_consensus.cc:740] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.807286  5972 consensus_queue.cc:260] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.809243  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:56.809494  5972 ts_tablet_manager.cc:1434] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:56.809792  5922 raft_consensus.cc:3060] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.794620  5970 tablet_bootstrap.cc:654] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.810962  5851 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:56.812472  5667 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:56.813375  6034 raft_consensus.cc:2804] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.813867  6034 raft_consensus.cc:697] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:56.814730  6034 consensus_queue.cc:237] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.815281  5972 tablet_bootstrap.cc:492] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:56.821753  5922 raft_consensus.cc:2468] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:56.801501  5973 raft_consensus.cc:740] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.836896  5922 raft_consensus.cc:3060] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.836761  5973 consensus_queue.cc:260] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.839061  5973 ts_tablet_manager.cc:1434] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.001s
I20260709 13:07:56.840431  5667 leader_election.cc:304] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:56.791272  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:56.826514  5599 catalog_manager.cc:5697] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.827884  5971 raft_consensus.cc:385] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.843039  5971 raft_consensus.cc:740] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.843811  5971 consensus_queue.cc:260] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.844393  6011 raft_consensus.cc:2804] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.844949  6011 raft_consensus.cc:697] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:56.800248  5984 leader_election.cc:290] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:56.828438  5776 raft_consensus.cc:2468] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:56.831076  5972 tablet_bootstrap.cc:654] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.833803  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:56.834306  6031 leader_election.cc:290] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:56.847518  6035 consensus_queue.cc:1048] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.849370  5850 raft_consensus.cc:3060] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.845784  6011 consensus_queue.cc:237] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:56.850131  5890 leader_election.cc:304] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:07:56.851605  5984 raft_consensus.cc:2804] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.847115  6070 raft_consensus.cc:493] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.847297  5773 raft_consensus.cc:3060] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.852624  6070 raft_consensus.cc:515] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.853823  5971 ts_tablet_manager.cc:1434] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.131s	user 0.003s	sys 0.004s
I20260709 13:07:56.854971  5971 tablet_bootstrap.cc:492] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:56.847851  5973 tablet_bootstrap.cc:492] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:56.852133  5984 raft_consensus.cc:697] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:07:56.856792  5984 consensus_queue.cc:237] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.859401  5850 raft_consensus.cc:2468] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:56.861856  5970 tablet_bootstrap.cc:492] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:56.862401  5970 ts_tablet_manager.cc:1403] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.145s	user 0.003s	sys 0.011s
I20260709 13:07:56.867136  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:56.873557  5773 raft_consensus.cc:2468] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:56.870404  6070 leader_election.cc:290] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:56.877986  5973 tablet_bootstrap.cc:654] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.882018  5970 raft_consensus.cc:359] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.883183  5970 raft_consensus.cc:385] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.883944  5970 raft_consensus.cc:740] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.888530  5970 consensus_queue.cc:260] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:56.885318  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:56.892678  5891 leader_election.cc:304] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:56.895869  5971 tablet_bootstrap.cc:654] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.897722  5970 ts_tablet_manager.cc:1434] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.001s
I20260709 13:07:56.899430  5970 tablet_bootstrap.cc:492] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:56.908000  5972 tablet_bootstrap.cc:492] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:56.909421  5972 ts_tablet_manager.cc:1403] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.094s	user 0.012s	sys 0.009s
I20260709 13:07:56.906625  6070 raft_consensus.cc:2749] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:56.926774  5973 tablet_bootstrap.cc:492] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:56.926611  5972 raft_consensus.cc:359] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.927299  5973 ts_tablet_manager.cc:1403] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.087s	user 0.007s	sys 0.006s
I20260709 13:07:56.927407  5972 raft_consensus.cc:385] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.927727  5972 raft_consensus.cc:740] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.928546  5972 consensus_queue.cc:260] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.930823  5973 raft_consensus.cc:359] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.931527  5973 raft_consensus.cc:385] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.934684  6059 consensus_queue.cc:1048] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.935418  6070 consensus_queue.cc:1048] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.938576  5599 catalog_manager.cc:5697] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.931784  5973 raft_consensus.cc:740] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.947808  5973 consensus_queue.cc:260] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:56.945214  5972 ts_tablet_manager.cc:1434] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.001s
I20260709 13:07:56.955029  5972 tablet_bootstrap.cc:492] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:56.955104  5973 ts_tablet_manager.cc:1434] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.027s	user 0.002s	sys 0.003s
I20260709 13:07:56.956118  5973 tablet_bootstrap.cc:492] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:56.960543  5971 tablet_bootstrap.cc:492] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:56.961117  5971 ts_tablet_manager.cc:1403] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.106s	user 0.022s	sys 0.001s
I20260709 13:07:56.962262  5972 tablet_bootstrap.cc:654] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.964109  5971 raft_consensus.cc:359] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.969540  5971 raft_consensus.cc:385] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.953392  5970 tablet_bootstrap.cc:654] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.940752  5599 catalog_manager.cc:5697] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:56.969900  5971 raft_consensus.cc:740] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.973598  5971 consensus_queue.cc:260] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.974680  5973 tablet_bootstrap.cc:654] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.980376  6011 raft_consensus.cc:493] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.981691  6011 raft_consensus.cc:515] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:56.983871  6011 leader_election.cc:290] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:56.986999  5971 ts_tablet_manager.cc:1434] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:07:56.989825  5970 tablet_bootstrap.cc:492] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:56.990513  5970 ts_tablet_manager.cc:1403] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.091s	user 0.011s	sys 0.002s
I20260709 13:07:56.990985  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:56.992395  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:56.993096  5922 raft_consensus.cc:2468] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:56.994498  5667 leader_election.cc:304] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:56.995483  6011 raft_consensus.cc:2804] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.995860  6011 raft_consensus.cc:493] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.996212  6011 raft_consensus.cc:3060] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.991760  5850 raft_consensus.cc:2468] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:56.999042  5595 catalog_manager.cc:5697] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 1 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:57.001353  5984 consensus_queue.cc:1048] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.001662  5970 raft_consensus.cc:359] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.007741  5970 raft_consensus.cc:385] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.008106  5970 raft_consensus.cc:740] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.008961  5970 consensus_queue.cc:260] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.011232  5970 ts_tablet_manager.cc:1434] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:07:57.013108  5970 tablet_bootstrap.cc:492] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:57.006938  6011 raft_consensus.cc:515] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.022691  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:57.023546  5851 raft_consensus.cc:3060] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.024302  6059 consensus_queue.cc:1048] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.987897  5971 tablet_bootstrap.cc:492] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:57.042368  5971 tablet_bootstrap.cc:654] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.053311  5984 consensus_queue.cc:1048] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:57.063162  5971 tablet_bootstrap.cc:492] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:57.063764  5971 ts_tablet_manager.cc:1403] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.076s	user 0.004s	sys 0.006s
I20260709 13:07:57.066962  5971 raft_consensus.cc:359] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.067728  5971 raft_consensus.cc:385] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.068044  5971 raft_consensus.cc:740] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.068753  5971 consensus_queue.cc:260] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.070717  6000 raft_consensus.cc:493] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.071444  6000 raft_consensus.cc:515] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.074178  5973 tablet_bootstrap.cc:492] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:57.074273  6000 leader_election.cc:290] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:57.074781  5973 ts_tablet_manager.cc:1403] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.119s	user 0.014s	sys 0.003s
I20260709 13:07:57.076962  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:57.077530  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:07:57.078624  5742 leader_election.cc:343] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.078859  5973 raft_consensus.cc:359] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:07:57.079416  5742 leader_election.cc:343] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.079555  5973 raft_consensus.cc:385] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.079985  5973 raft_consensus.cc:740] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.079950  5742 leader_election.cc:304] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:57.080981  5973 consensus_queue.cc:260] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.081463  6073 raft_consensus.cc:2749] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.082255  6011 leader_election.cc:290] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:57.084132  5973 ts_tablet_manager.cc:1434] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:07:57.084911  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:57.085214  5973 tablet_bootstrap.cc:492] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:57.085330  5851 raft_consensus.cc:2468] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:57.089021  5984 consensus_queue.cc:1048] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.090006  5922 raft_consensus.cc:3060] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.070953  5971 ts_tablet_manager.cc:1434] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:07:57.099078  5972 tablet_bootstrap.cc:492] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:57.117143  5972 ts_tablet_manager.cc:1403] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.162s	user 0.024s	sys 0.026s
I20260709 13:07:57.092655  5973 tablet_bootstrap.cc:654] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.118633  6034 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.119251  6034 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.120085  5972 raft_consensus.cc:359] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.120822  5972 raft_consensus.cc:385] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.120895  5922 raft_consensus.cc:2468] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:57.123711  5667 leader_election.cc:304] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:57.125499  6124 raft_consensus.cc:2804] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.121516  6034 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:57.126109  6124 raft_consensus.cc:697] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:57.127058  5972 raft_consensus.cc:740] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.127058  6124 consensus_queue.cc:237] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.124303  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:57.122833  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:57.134248  5972 consensus_queue.cc:260] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.136449  5971 tablet_bootstrap.cc:492] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:57.139392  5972 ts_tablet_manager.cc:1434] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.007s
W20260709 13:07:57.140668  5667 leader_election.cc:343] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:57.148331  5666 leader_election.cc:343] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.156687  5971 tablet_bootstrap.cc:654] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.164038  5666 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:57.165738  5970 tablet_bootstrap.cc:654] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.165228  6073 consensus_queue.cc:1048] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:57.170048  5972 tablet_bootstrap.cc:492] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:57.170181  6034 raft_consensus.cc:2749] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.181473  5973 tablet_bootstrap.cc:492] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:57.182193  5973 ts_tablet_manager.cc:1403] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.097s	user 0.010s	sys 0.004s
I20260709 13:07:57.197021  5970 tablet_bootstrap.cc:492] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:57.197782  5970 ts_tablet_manager.cc:1403] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.185s	user 0.016s	sys 0.007s
I20260709 13:07:57.202333  5973 raft_consensus.cc:359] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.203642  5973 raft_consensus.cc:385] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.203972  5973 raft_consensus.cc:740] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.204787  5973 consensus_queue.cc:260] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.207715  5972 tablet_bootstrap.cc:654] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.211128  5973 ts_tablet_manager.cc:1434] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 13:07:57.212235  5973 tablet_bootstrap.cc:492] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:57.212294  5971 tablet_bootstrap.cc:492] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:57.193504  6073 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.192935  6065 raft_consensus.cc:493] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.213104  5971 ts_tablet_manager.cc:1403] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.080s	user 0.015s	sys 0.004s
I20260709 13:07:57.214262  6065 raft_consensus.cc:515] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.229269  5970 raft_consensus.cc:359] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.231289  5970 raft_consensus.cc:385] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.231665  5970 raft_consensus.cc:740] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.231829  6070 raft_consensus.cc:493] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.232493  6070 raft_consensus.cc:515] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.232535  5970 consensus_queue.cc:260] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.234706  6065 leader_election.cc:290] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:57.235025  6070 leader_election.cc:290] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:57.215564  6073 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.237387  6073 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:57.237468  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:57.238270  5773 raft_consensus.cc:2468] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:57.240160  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:57.242684  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:57.244057  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:57.244290  5890 leader_election.cc:304] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:07:57.245860  5970 ts_tablet_manager.cc:1434] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.005s
W20260709 13:07:57.245796  5742 leader_election.cc:343] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.246588  5815 leader_election.cc:304] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:57.247483  5970 tablet_bootstrap.cc:492] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:57.247689  6065 raft_consensus.cc:2749] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.248835  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:07:57.250458  5742 leader_election.cc:343] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.250977  5742 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:57.252770  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:57.253630  6073 raft_consensus.cc:2749] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.258248  5970 tablet_bootstrap.cc:654] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.263406  5984 raft_consensus.cc:2804] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:07:57.269771  5891 leader_election.cc:343] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.292558  5984 raft_consensus.cc:493] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.272691  5971 raft_consensus.cc:359] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.280372  5850 raft_consensus.cc:3060] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.285025  6124 consensus_queue.cc:1048] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.286417  6011 consensus_queue.cc:1048] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:57.322536  6011 consensus_queue.cc:1048] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:57.328415  6034 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.329218  6034 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.335636  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:07:57.339139  5667 leader_election.cc:343] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.340050  5971 raft_consensus.cc:385] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.340523  5971 raft_consensus.cc:740] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.340646  6034 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:57.293025  5984 raft_consensus.cc:3060] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.342490  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:57.344339  5776 raft_consensus.cc:2468] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:57.345871  5666 leader_election.cc:304] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:57.347043  6124 raft_consensus.cc:2804] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.347492  6124 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.347863  6124 raft_consensus.cc:3060] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.358644  5971 consensus_queue.cc:260] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.288601  5972 tablet_bootstrap.cc:492] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:57.362037  5972 ts_tablet_manager.cc:1403] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.192s	user 0.016s	sys 0.006s
I20260709 13:07:57.361128  6011 consensus_queue.cc:1048] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:57.366348  5973 tablet_bootstrap.cc:654] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.355012  5597 catalog_manager.cc:5697] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:57.394491  6073 consensus_queue.cc:1048] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:57.373497  5971 ts_tablet_manager.cc:1434] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.158s	user 0.007s	sys 0.000s
I20260709 13:07:57.397112  5971 tablet_bootstrap.cc:492] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:57.385953  5972 raft_consensus.cc:359] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.400265  5972 raft_consensus.cc:385] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.400774  5972 raft_consensus.cc:740] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.401867  5972 consensus_queue.cc:260] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.403154  6150 raft_consensus.cc:493] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.376932  6034 consensus_queue.cc:1048] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:07:57.380306  5984 raft_consensus.cc:515] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.411779  5972 ts_tablet_manager.cc:1434] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.004s
I20260709 13:07:57.413329  5972 tablet_bootstrap.cc:492] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:57.403898  6150 raft_consensus.cc:515] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.419904  6073 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.434830  6073 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.437186  6073 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:57.442361  6150 leader_election.cc:290] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:57.444710  5984 leader_election.cc:290] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:57.449635  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:57.450425  5850 raft_consensus.cc:3060] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.421147  6011 consensus_queue.cc:1048] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:07:57.450662  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:57.456851  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:57.457697  5776 raft_consensus.cc:3060] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.461078  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:57.463245  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:57.464234  5924 raft_consensus.cc:2393] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:07:57.466178  5850 raft_consensus.cc:2468] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:57.469385  5891 leader_election.cc:304] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:07:57.422626  6124 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:07:57.471750  5742 leader_election.cc:343] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.473496  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:57.474112  6124 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:07:57.476104  5739 leader_election.cc:343] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.476737  5739 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:57.485913  6059 raft_consensus.cc:2804] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.499815  6011 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.527697  5972 tablet_bootstrap.cc:654] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.509442  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:57.517582  6073 raft_consensus.cc:2749] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.553915  5816 leader_election.cc:304] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
W20260709 13:07:57.555263  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 52673.3us
I20260709 13:07:57.555845  5973 tablet_bootstrap.cc:492] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:57.556066  6150 raft_consensus.cc:2749] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.556455  5973 ts_tablet_manager.cc:1403] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.344s	user 0.021s	sys 0.028s
I20260709 13:07:57.557565  5971 tablet_bootstrap.cc:654] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.520990  6059 raft_consensus.cc:697] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:07:57.560644  6070 consensus_queue.cc:1048] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:57.561553  6059 consensus_queue.cc:237] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.549281  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:57.520256  6011 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.583660  5973 raft_consensus.cc:359] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.585587  5973 raft_consensus.cc:385] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.585738  5970 tablet_bootstrap.cc:492] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:57.586010  5973 raft_consensus.cc:740] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.586318  5970 ts_tablet_manager.cc:1403] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.339s	user 0.126s	sys 0.124s
I20260709 13:07:57.589234  5973 consensus_queue.cc:260] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.584996  5773 raft_consensus.cc:3060] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:57.574770  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 98711.4us
I20260709 13:07:57.595306  5973 ts_tablet_manager.cc:1434] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:07:57.600382  5973 tablet_bootstrap.cc:492] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:57.603175  5984 consensus_queue.cc:1048] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:57.605353  6011 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:57.615334  5776 raft_consensus.cc:2468] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:57.636649  5970 raft_consensus.cc:359] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.637743  5970 raft_consensus.cc:385] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.638175  5970 raft_consensus.cc:740] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.639497  5970 consensus_queue.cc:260] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.641444  5972 tablet_bootstrap.cc:492] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:57.642371  5972 ts_tablet_manager.cc:1403] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.229s	user 0.019s	sys 0.020s
I20260709 13:07:57.646018  5973 tablet_bootstrap.cc:654] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.646831  5970 ts_tablet_manager.cc:1434] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.000s
W20260709 13:07:57.647603  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 97860.3us
I20260709 13:07:57.650429  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:07:57.651381  5667 leader_election.cc:343] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.651998  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:57.651961  5970 tablet_bootstrap.cc:492] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:57.661228  5970 tablet_bootstrap.cc:654] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:57.665462  5667 leader_election.cc:343] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.669083  6162 consensus_queue.cc:1048] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:07:57.684329  5972 raft_consensus.cc:359] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.685271  5972 raft_consensus.cc:385] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.691395  5972 raft_consensus.cc:740] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.692276  5972 consensus_queue.cc:260] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.693078  5773 raft_consensus.cc:2468] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:57.708302  5971 tablet_bootstrap.cc:492] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:57.708926  5971 ts_tablet_manager.cc:1403] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.312s	user 0.017s	sys 0.005s
I20260709 13:07:57.709122  5972 ts_tablet_manager.cc:1434] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 13:07:57.709872  5666 leader_election.cc:304] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:57.717715  5972 tablet_bootstrap.cc:492] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:57.717828  6011 raft_consensus.cc:2804] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.718493  6011 raft_consensus.cc:697] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:57.694832  5923 raft_consensus.cc:2468] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:57.723104  5971 raft_consensus.cc:359] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.723367  5667 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:57.719404  6011 consensus_queue.cc:237] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.724772  5972 tablet_bootstrap.cc:654] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.726820  5971 raft_consensus.cc:385] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.728807  5971 raft_consensus.cc:740] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.737864  5971 consensus_queue.cc:260] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.736944  6162 raft_consensus.cc:2804] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.739463  6162 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.739864  6162 raft_consensus.cc:3060] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.740918  5971 ts_tablet_manager.cc:1434] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.003s
I20260709 13:07:57.742127  5971 tablet_bootstrap.cc:492] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:57.748827  5971 tablet_bootstrap.cc:654] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.768932  6165 consensus_queue.cc:1048] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:57.771440  5973 tablet_bootstrap.cc:492] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:57.772015  5973 ts_tablet_manager.cc:1403] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.176s	user 0.017s	sys 0.003s
I20260709 13:07:57.774893  5973 raft_consensus.cc:359] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.775593  5973 raft_consensus.cc:385] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.775849  5973 raft_consensus.cc:740] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.776845  5973 consensus_queue.cc:260] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.781581  5973 ts_tablet_manager.cc:1434] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:07:57.781765  5599 catalog_manager.cc:5697] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 1 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:57.782888  5973 tablet_bootstrap.cc:492] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:57.784272  5984 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.771796  6162 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.784825  5984 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.786551  6162 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:57.788668  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:57.789134  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:57.789412  5776 raft_consensus.cc:2468] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:57.789777  5849 raft_consensus.cc:2468] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:57.790520  5984 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:57.790938  5890 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:07:57.792189  5984 raft_consensus.cc:2804] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.792615  5984 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.793022  5984 raft_consensus.cc:3060] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.795847  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:57.796630  5925 raft_consensus.cc:3060] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.804777  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:57.808307  5970 tablet_bootstrap.cc:492] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:57.808843  5970 ts_tablet_manager.cc:1403] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.158s	user 0.020s	sys 0.021s
I20260709 13:07:57.812317  5970 raft_consensus.cc:359] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.813112  5970 raft_consensus.cc:385] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:07:57.806186  5667 leader_election.cc:343] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.813315  5970 raft_consensus.cc:740] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.806519  6162 raft_consensus.cc:493] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.816352  6162 raft_consensus.cc:515] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.816705  5970 consensus_queue.cc:260] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.827773  5970 ts_tablet_manager.cc:1434] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 13:07:57.828815  5970 tablet_bootstrap.cc:492] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:57.833395  5971 tablet_bootstrap.cc:492] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:57.832170  5973 tablet_bootstrap.cc:654] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.836599  5971 ts_tablet_manager.cc:1403] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.095s	user 0.015s	sys 0.002s
I20260709 13:07:57.839608  5972 tablet_bootstrap.cc:492] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:57.839380  5984 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.840236  5972 ts_tablet_manager.cc:1403] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.123s	user 0.015s	sys 0.007s
I20260709 13:07:57.842800  6162 leader_election.cc:290] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:57.843012  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:57.845463  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:57.846235  5776 raft_consensus.cc:2468] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:57.847704  5666 leader_election.cc:304] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
W20260709 13:07:57.849232  5667 leader_election.cc:343] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.846315  5925 raft_consensus.cc:2468] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:57.850895  5667 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:57.851276  5971 raft_consensus.cc:359] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.852380  6165 raft_consensus.cc:2804] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.857391  5971 raft_consensus.cc:385] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.857570  6165 raft_consensus.cc:697] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:57.858968  6165 consensus_queue.cc:237] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.857771  5971 raft_consensus.cc:740] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.878851  5971 consensus_queue.cc:260] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.881796  5971 ts_tablet_manager.cc:1434] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
W20260709 13:07:57.882354  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:57.856873  5970 tablet_bootstrap.cc:654] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.853690  6162 raft_consensus.cc:2804] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.862622  5599 catalog_manager.cc:5697] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.884979  6162 raft_consensus.cc:493] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.885464  6162 raft_consensus.cc:3060] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.882613  5971 tablet_bootstrap.cc:492] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:57.893772  6162 raft_consensus.cc:515] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.896807  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:57.897934  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:57.898710  5776 raft_consensus.cc:3060] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:57.898729  5667 leader_election.cc:343] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.904804  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:57.905563  5773 raft_consensus.cc:3060] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.904970  6150 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.908154  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:57.908350  6150 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.909027  5850 raft_consensus.cc:3060] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.912817  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:57.913889  5700 raft_consensus.cc:2468] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:07:57.915992  5813 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:57.918183  5971 tablet_bootstrap.cc:654] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.919277  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:57.856714  5972 raft_consensus.cc:359] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.920508  5972 raft_consensus.cc:385] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.920954  5972 raft_consensus.cc:740] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.921128  6161 raft_consensus.cc:493] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.922122  5984 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:07:57.923064  5815 leader_election.cc:343] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.921844  6161 raft_consensus.cc:515] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.926520  6161 leader_election.cc:290] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:57.928951  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:57.930588  5850 raft_consensus.cc:2468] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
W20260709 13:07:57.931074  5742 leader_election.cc:343] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.931876  5972 consensus_queue.cc:260] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.933068  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
W20260709 13:07:57.934809  5739 leader_election.cc:343] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.935317  5739 leader_election.cc:304] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:57.937047  6150 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:57.937630  6150 raft_consensus.cc:2804] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:57.937963  6150 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.938074  5973 tablet_bootstrap.cc:492] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:57.938338  6150 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.938580  5973 ts_tablet_manager.cc:1403] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.156s	user 0.013s	sys 0.008s
I20260709 13:07:57.934441  5972 ts_tablet_manager.cc:1434] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.000s
I20260709 13:07:57.940742  5972 tablet_bootstrap.cc:492] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:57.940882  6073 raft_consensus.cc:2749] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.941583  5973 raft_consensus.cc:359] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.942337  5973 raft_consensus.cc:385] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.942612  5973 raft_consensus.cc:740] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.943343  5973 consensus_queue.cc:260] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.948182  5973 ts_tablet_manager.cc:1434] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:07:57.949265  5973 tablet_bootstrap.cc:492] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:57.951190  5599 catalog_manager.cc:5697] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:57.954244  5984 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.954955  5984 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.956125  6162 leader_election.cc:290] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:57.959585  5984 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:57.961256  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:57.961696  5972 tablet_bootstrap.cc:654] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.961573  5776 raft_consensus.cc:2468] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:57.959251  5773 raft_consensus.cc:2468] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:57.955798  5973 tablet_bootstrap.cc:654] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:57.966756  5890 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:07:57.968357  5984 raft_consensus.cc:2804] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:57.968875  5984 raft_consensus.cc:697] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:07:57.970075  5984 consensus_queue.cc:237] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:57.971331  5666 leader_election.cc:304] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:57.972517  6162 raft_consensus.cc:2804] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:07:57.972828  5888 leader_election.cc:343] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.973569  6162 raft_consensus.cc:697] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:57.974463  6162 consensus_queue.cc:237] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:57.976639  5973 tablet_bootstrap.cc:492] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:57.977245  5973 ts_tablet_manager.cc:1403] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.003s
I20260709 13:07:57.980589  5973 raft_consensus.cc:359] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.981344  5973 raft_consensus.cc:385] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.981657  5973 raft_consensus.cc:740] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
W20260709 13:07:57.983640  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 67780.6us
I20260709 13:07:57.985217  5970 tablet_bootstrap.cc:492] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:57.985822  5970 ts_tablet_manager.cc:1403] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.157s	user 0.020s	sys 0.017s
I20260709 13:07:57.986084  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:57.966747  6150 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:07:57.987663  5891 leader_election.cc:343] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:57.988185  5891 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:57.988924  6150 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:57.989398  6070 raft_consensus.cc:2749] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:57.990231  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:57.989405  5970 raft_consensus.cc:359] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.990708  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:57.991108  5970 raft_consensus.cc:385] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:57.991427  5700 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:57.991680  5973 consensus_queue.cc:260] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:57.991673  5972 tablet_bootstrap.cc:492] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:57.991436  5970 raft_consensus.cc:740] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:57.993343  5972 ts_tablet_manager.cc:1403] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.003s
I20260709 13:07:57.993786  5970 consensus_queue.cc:260] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:57.997792  5971 tablet_bootstrap.cc:492] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:57.998363  5971 ts_tablet_manager.cc:1403] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.116s	user 0.009s	sys 0.007s
I20260709 13:07:57.998903  5773 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.003340  5970 ts_tablet_manager.cc:1434] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 13:07:58.005692  5970 tablet_bootstrap.cc:492] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:58.004551  5973 ts_tablet_manager.cc:1434] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:07:58.014285  5973 tablet_bootstrap.cc:492] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:58.019025  5700 raft_consensus.cc:2468] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:07:58.022032  5970 tablet_bootstrap.cc:654] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.022557  5813 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:58.021214  5971 raft_consensus.cc:359] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.023465  5971 raft_consensus.cc:385] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.023684  6150 raft_consensus.cc:2804] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:58.016877  5972 raft_consensus.cc:359] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.024510  6150 raft_consensus.cc:697] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:07:58.025197  5972 raft_consensus.cc:385] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.024652  5598 catalog_manager.cc:5697] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.025985  5972 raft_consensus.cc:740] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.026171  5973 tablet_bootstrap.cc:654] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.025849  6150 consensus_queue.cc:237] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.026891  5972 consensus_queue.cc:260] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.029385  5972 ts_tablet_manager.cc:1434] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.001s
I20260709 13:07:58.030416  5972 tablet_bootstrap.cc:492] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:58.032002  5599 catalog_manager.cc:5697] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 1 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:58.023787  5971 raft_consensus.cc:740] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.035535  5971 consensus_queue.cc:260] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.036701  5972 tablet_bootstrap.cc:654] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.042433  5973 tablet_bootstrap.cc:492] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:58.042953  5973 ts_tablet_manager.cc:1403] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.000s
I20260709 13:07:58.049714  5973 raft_consensus.cc:359] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.050747  5973 raft_consensus.cc:385] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.051210  5973 raft_consensus.cc:740] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.052048  5973 consensus_queue.cc:260] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.053570  6165 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.054054  6165 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.057665  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:58.058422  5776 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:58.059762  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:58.060530  5850 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:58.061923  5667 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:58.062898  6162 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:58.063633  6165 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:58.063565  6162 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.064535  6162 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.071324  5973 ts_tablet_manager.cc:1434] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 13:07:58.075836  5984 consensus_queue.cc:1048] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:58.050452  5971 ts_tablet_manager.cc:1434] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 13:07:58.079185  5971 tablet_bootstrap.cc:492] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:58.079504  5597 catalog_manager.cc:5697] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 1 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:58.084221  5773 raft_consensus.cc:2468] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:07:58.084475  5984 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.092337  5984 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.094026  5970 tablet_bootstrap.cc:492] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:58.094661  5970 ts_tablet_manager.cc:1403] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.009s
I20260709 13:07:58.094694  5984 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:58.072690  5973 tablet_bootstrap.cc:492] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:58.096815  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:58.097714  5775 raft_consensus.cc:2393] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:07:58.098413  5970 raft_consensus.cc:359] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:58.099392  5970 raft_consensus.cc:385] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.099776  5970 raft_consensus.cc:740] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.100684  5970 consensus_queue.cc:260] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:58.102844  5970 ts_tablet_manager.cc:1434] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:07:58.104113  5970 tablet_bootstrap.cc:492] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:58.104411  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:58.106041  5973 tablet_bootstrap.cc:654] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.106666  5888 leader_election.cc:304] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:58.109061  6070 raft_consensus.cc:3060] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.112303  5972 tablet_bootstrap.cc:492] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:58.112848  5972 ts_tablet_manager.cc:1403] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.083s	user 0.031s	sys 0.010s
I20260709 13:07:58.116369  5972 raft_consensus.cc:359] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.117074  5972 raft_consensus.cc:385] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.117344  5972 raft_consensus.cc:740] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.118587  5970 tablet_bootstrap.cc:654] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.118779  5972 consensus_queue.cc:260] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.121775  5972 ts_tablet_manager.cc:1434] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:07:58.123736  5972 tablet_bootstrap.cc:492] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:58.137881  5972 tablet_bootstrap.cc:654] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.143998  5973 tablet_bootstrap.cc:492] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:58.144588  5973 ts_tablet_manager.cc:1403] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.072s	user 0.016s	sys 0.002s
I20260709 13:07:58.147588  5973 raft_consensus.cc:359] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:58.148769  5973 raft_consensus.cc:385] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.149055  5973 raft_consensus.cc:740] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.149902  5973 consensus_queue.cc:260] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:58.154346  5973 ts_tablet_manager.cc:1434] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:07:58.156819  5973 tablet_bootstrap.cc:492] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:58.162999  6075 consensus_queue.cc:1048] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:58.170516  5973 tablet_bootstrap.cc:654] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.138646  6162 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.176237  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:58.177143  5776 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.089085  5971 tablet_bootstrap.cc:654] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.185606  5973 tablet_bootstrap.cc:492] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:58.186378  5973 ts_tablet_manager.cc:1403] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.005s
I20260709 13:07:58.185516  6070 raft_consensus.cc:2749] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:58.194263  5973 raft_consensus.cc:359] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.217300  5973 raft_consensus.cc:385] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.217653  5973 raft_consensus.cc:740] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.218374  5973 consensus_queue.cc:260] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.195749  5776 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:58.220959  5666 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:58.219640  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:58.222589  5849 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.223745  5973 ts_tablet_manager.cc:1434] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.007s
I20260709 13:07:58.228888  5973 tablet_bootstrap.cc:492] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:58.228073  6124 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:58.227674  6011 raft_consensus.cc:493] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.229094  6162 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:58.240068  6124 raft_consensus.cc:697] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:58.241843  5971 tablet_bootstrap.cc:492] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:58.248945  5971 ts_tablet_manager.cc:1403] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.170s	user 0.015s	sys 0.000s
I20260709 13:07:58.248060  6011 raft_consensus.cc:515] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.270757  6011 leader_election.cc:290] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:58.272692  6124 consensus_queue.cc:237] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.282011  5970 tablet_bootstrap.cc:492] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:58.282516  5970 ts_tablet_manager.cc:1403] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.179s	user 0.015s	sys 0.041s
I20260709 13:07:58.285979  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:58.286876  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:07:58.294806  5666 leader_election.cc:343] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:58.297058  5667 leader_election.cc:343] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.299029  5667 leader_election.cc:304] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:58.299109  5970 raft_consensus.cc:359] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.299906  5970 raft_consensus.cc:385] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.303836  5970 raft_consensus.cc:740] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.304764  5970 consensus_queue.cc:260] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.301283  6158 raft_consensus.cc:2749] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:58.305645  6162 consensus_queue.cc:1048] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:58.307461  5971 raft_consensus.cc:359] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:58.320569  5971 raft_consensus.cc:385] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.322194  5971 raft_consensus.cc:740] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.323735  5973 tablet_bootstrap.cc:654] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.310310  5972 tablet_bootstrap.cc:492] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:58.320758  5849 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:58.321740  5970 ts_tablet_manager.cc:1434] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:07:58.325626  5972 ts_tablet_manager.cc:1403] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.202s	user 0.035s	sys 0.031s
I20260709 13:07:58.323633  5971 consensus_queue.cc:260] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:58.329433  5970 tablet_bootstrap.cc:492] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
W20260709 13:07:58.333652  5667 consensus_peers.cc:597] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:58.334375  6073 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.334975  5971 ts_tablet_manager.cc:1434] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.086s	user 0.011s	sys 0.000s
I20260709 13:07:58.336258  6073 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:58.337203  5972 raft_consensus.cc:359] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:58.338625  6073 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:58.342931  5971 tablet_bootstrap.cc:492] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:58.342773  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:58.344513  5972 raft_consensus.cc:385] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.344936  5972 raft_consensus.cc:740] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.345849  5972 consensus_queue.cc:260] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:58.347445  5970 tablet_bootstrap.cc:654] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.347476  5925 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:07:58.348285  5972 ts_tablet_manager.cc:1434] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:07:58.350735  5742 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:07:58.351774  6073 raft_consensus.cc:2804] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:58.352191  6073 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.352545  6073 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.353788  5971 tablet_bootstrap.cc:654] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.358173  5597 catalog_manager.cc:5697] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.361811  6201 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.363202  6201 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:58.365554  6201 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:58.369853  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:58.371177  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:07:58.372687  5742 leader_election.cc:343] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.375325  5973 tablet_bootstrap.cc:492] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:58.375939  5973 ts_tablet_manager.cc:1403] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.147s	user 0.015s	sys 0.002s
I20260709 13:07:58.379302  6182 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.379880  6182 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:58.381868  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:58.382264  5973 raft_consensus.cc:359] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.383031  5973 raft_consensus.cc:385] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.382963  6182 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:58.387348  5973 raft_consensus.cc:740] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.388732  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:58.389618  5924 raft_consensus.cc:2393] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:07:58.389766  5973 consensus_queue.cc:260] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:07:58.390933  5667 leader_election.cc:343] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.392071  5973 ts_tablet_manager.cc:1434] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:07:58.392155  6011 raft_consensus.cc:493] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.392719  6011 raft_consensus.cc:515] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.382658  5923 raft_consensus.cc:2468] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:58.393224  5973 tablet_bootstrap.cc:492] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:58.397328  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:58.394735  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
W20260709 13:07:58.403419  5666 leader_election.cc:343] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.404811  5813 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:58.405246  6011 leader_election.cc:290] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:58.406204  6182 raft_consensus.cc:3060] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.410567  6075 consensus_queue.cc:1048] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:58.411006  5972 tablet_bootstrap.cc:492] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:58.417495  6073 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:58.421581  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:58.422722  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:58.457319  5923 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.433089  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:58.436874  5970 tablet_bootstrap.cc:492] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:58.460907  5970 ts_tablet_manager.cc:1403] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.136s	user 0.014s	sys 0.016s
W20260709 13:07:58.460807  5742 leader_election.cc:343] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.437019  5973 tablet_bootstrap.cc:654] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.445160  6075 raft_consensus.cc:493] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.465112  6075 raft_consensus.cc:515] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.465526  5923 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:07:58.466746  5742 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:58.467363  6075 leader_election.cc:290] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:58.419883  6073 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:58.452059  6011 consensus_queue.cc:1048] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:07:58.470729  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:58.471571  5775 raft_consensus.cc:2393] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:07:58.452337  6124 consensus_queue.cc:1048] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:07:58.423523  6182 raft_consensus.cc:2749] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:58.467738  6000 raft_consensus.cc:2804] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:58.452191  5971 tablet_bootstrap.cc:492] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:58.474496  6000 raft_consensus.cc:697] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:07:58.474684  5971 ts_tablet_manager.cc:1403] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.139s	user 0.025s	sys 0.015s
I20260709 13:07:58.475476  6000 consensus_queue.cc:237] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:58.475605  6059 consensus_queue.cc:1048] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:58.452497  5667 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:58.484009  6011 raft_consensus.cc:2804] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:58.484450  6011 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.448570  6201 raft_consensus.cc:493] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.484822  6124 consensus_queue.cc:1048] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:58.485332  6200 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.485649  6201 raft_consensus.cc:515] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.485862  6200 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.484827  6011 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.467053  5970 raft_consensus.cc:359] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.494724  5970 raft_consensus.cc:385] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.495052  5970 raft_consensus.cc:740] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.495190  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:58.493103  6182 consensus_queue.cc:1048] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:07:58.496066  5776 raft_consensus.cc:2468] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:58.495962  5970 consensus_queue.cc:260] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.497440  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:58.498165  5775 raft_consensus.cc:2468] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:58.489629  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:58.498726  5970 ts_tablet_manager.cc:1434] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:07:58.487739  6201 leader_election.cc:290] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:58.500905  5666 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:58.501860  6201 raft_consensus.cc:2804] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:58.502266  6201 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.488082  6200 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:58.502694  6201 raft_consensus.cc:3060] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:58.490082  5667 leader_election.cc:343] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.507854  5667 leader_election.cc:304] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:58.509169  6200 raft_consensus.cc:2749] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:58.511929  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:58.502068  5666 leader_election.cc:304] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:58.513199  6200 raft_consensus.cc:2804] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:58.513640  6200 raft_consensus.cc:493] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.514007  6200 raft_consensus.cc:3060] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.515352  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:58.518126  5972 tablet_bootstrap.cc:654] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.519469  5888 leader_election.cc:304] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:58.524860  6075 raft_consensus.cc:3060] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.499768  5970 tablet_bootstrap.cc:492] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
W20260709 13:07:58.521524  5667 leader_election.cc:343] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.533169  6201 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.477669  5971 raft_consensus.cc:359] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:58.537292  5971 raft_consensus.cc:385] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.537591  5971 raft_consensus.cc:740] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.538579  5971 consensus_queue.cc:260] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:07:58.540928  5667 leader_election.cc:343] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.541457  5971 ts_tablet_manager.cc:1434] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.001s
I20260709 13:07:58.550552  5971 tablet_bootstrap.cc:492] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:58.563398  6200 raft_consensus.cc:515] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.568058  6200 leader_election.cc:290] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:58.568878  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:58.571514  5774 raft_consensus.cc:3060] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.578099  5970 tablet_bootstrap.cc:654] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.551941  6011 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:58.592329  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:58.595938  6073 raft_consensus.cc:493] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.596822  6073 raft_consensus.cc:515] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.612496  5973 tablet_bootstrap.cc:492] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:58.613173  5973 ts_tablet_manager.cc:1403] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.220s	user 0.013s	sys 0.010s
I20260709 13:07:58.614355  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:58.615051  5776 raft_consensus.cc:3060] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.584990  6200 raft_consensus.cc:493] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.585084  6201 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:58.626323  5595 catalog_manager.cc:5697] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 1 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.631814  5776 raft_consensus.cc:2468] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:58.638744  5666 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:58.640682  6201 raft_consensus.cc:2804] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:58.641211  6201 raft_consensus.cc:697] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
W20260709 13:07:58.645090  5667 leader_election.cc:343] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.646653  6150 consensus_queue.cc:1048] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:58.648715  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:58.649108  6011 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:58.657289  5971 tablet_bootstrap.cc:654] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.629345  6200 raft_consensus.cc:515] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:58.657975  6075 raft_consensus.cc:2749] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:58.642480  6201 consensus_queue.cc:237] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.678295  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:07:58.680454  5742 leader_election.cc:343] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.682614  6073 leader_election.cc:290] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:58.684279  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
W20260709 13:07:58.690927  6215 log.cc:927] Time spent T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:07:58.701547  5972 tablet_bootstrap.cc:492] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:58.702580  5972 ts_tablet_manager.cc:1403] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.292s	user 0.016s	sys 0.018s
I20260709 13:07:58.705958  5701 raft_consensus.cc:2468] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:07:58.713596  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:58.717275  5739 leader_election.cc:304] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:58.718228  6073 raft_consensus.cc:2804] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:58.721642  5774 raft_consensus.cc:2468] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:58.723289  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:58.723984  5922 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.728215  5972 raft_consensus.cc:359] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.729002  5972 raft_consensus.cc:385] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.729249  5972 raft_consensus.cc:740] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.718912  6073 raft_consensus.cc:493] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:58.737279  6073 raft_consensus.cc:3060] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.734542  5973 raft_consensus.cc:359] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:58.748579  5973 raft_consensus.cc:385] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.748960  5973 raft_consensus.cc:740] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.750154  6124 consensus_queue.cc:1048] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:58.753960  5666 leader_election.cc:304] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:58.755713  6011 raft_consensus.cc:2804] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:58.756558  6011 raft_consensus.cc:697] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
W20260709 13:07:58.756816  5667 leader_election.cc:343] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:58.758240  5667 leader_election.cc:343] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.757589  6011 consensus_queue.cc:237] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.781162  5973 consensus_queue.cc:260] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:58.783310  5972 consensus_queue.cc:260] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.797034  5972 ts_tablet_manager.cc:1434] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.094s	user 0.007s	sys 0.001s
I20260709 13:07:58.807139  5972 tablet_bootstrap.cc:492] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:58.812713  5970 tablet_bootstrap.cc:492] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:58.813402  5970 ts_tablet_manager.cc:1403] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.314s	user 0.019s	sys 0.035s
I20260709 13:07:58.847404  5973 ts_tablet_manager.cc:1434] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.234s	user 0.006s	sys 0.009s
I20260709 13:07:58.804194  6200 leader_election.cc:290] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:58.849309  5973 tablet_bootstrap.cc:492] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:58.855204  5970 raft_consensus.cc:359] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.856982  5970 raft_consensus.cc:385] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.857352  5970 raft_consensus.cc:740] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:58.858211  5970 consensus_queue.cc:260] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:07:58.845753  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 101875us
I20260709 13:07:58.867277  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:58.873872  5970 ts_tablet_manager.cc:1434] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.003s
I20260709 13:07:58.875572  5970 tablet_bootstrap.cc:492] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:58.868754  6158 consensus_queue.cc:1048] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:07:58.892344  5849 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.892387  6073 raft_consensus.cc:515] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:58.900302  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:07:58.900532  5667 leader_election.cc:343] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.902269  6073 leader_election.cc:290] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:58.910730  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
W20260709 13:07:58.916797  5742 leader_election.cc:343] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.919832  5973 tablet_bootstrap.cc:654] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.926767  5922 raft_consensus.cc:2468] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:58.930027  6161 consensus_queue.cc:1048] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:58.936818  5972 tablet_bootstrap.cc:654] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.944622  5971 tablet_bootstrap.cc:492] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:58.945061  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:58.946693  6220 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:58.940021  6200 consensus_queue.cc:1048] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
W20260709 13:07:58.949146  5667 leader_election.cc:343] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:58.954238  5667 leader_election.cc:304] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:58.950750  5700 raft_consensus.cc:3060] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:58.952013  5598 catalog_manager.cc:5697] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.960126  6162 raft_consensus.cc:2749] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:58.953063  5971 ts_tablet_manager.cc:1403] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.404s	user 0.016s	sys 0.020s
I20260709 13:07:58.964484  5667 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:58.959621  6220 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:58.968222  5970 tablet_bootstrap.cc:654] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:58.978565  6200 raft_consensus.cc:2804] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:58.981472  6200 raft_consensus.cc:697] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:58.982793  6200 consensus_queue.cc:237] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:58.981488  5971 raft_consensus.cc:359] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:58.988622  5971 raft_consensus.cc:385] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:58.989007  5971 raft_consensus.cc:740] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.007616  5971 consensus_queue.cc:260] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.012492  5971 ts_tablet_manager.cc:1434] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.051s	user 0.010s	sys 0.000s
I20260709 13:07:59.015676  5972 tablet_bootstrap.cc:492] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:59.016285  5972 ts_tablet_manager.cc:1403] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.209s	user 0.016s	sys 0.047s
I20260709 13:07:59.023383  6221 raft_consensus.cc:493] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.024061  6221 raft_consensus.cc:515] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.026341  6221 leader_election.cc:290] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:59.031723  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:59.035585  5700 raft_consensus.cc:2468] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
W20260709 13:07:59.007968  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 43576.7us
I20260709 13:07:59.042064  5970 tablet_bootstrap.cc:492] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:59.042615  5970 ts_tablet_manager.cc:1403] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.167s	user 0.012s	sys 0.016s
I20260709 13:07:59.044255  5739 leader_election.cc:304] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:59.046141  6161 raft_consensus.cc:2804] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:59.046674  6161 raft_consensus.cc:697] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:07:59.047490  6161 consensus_queue.cc:237] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.059939  6220 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:59.069453  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:59.070443  5972 raft_consensus.cc:359] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.070907  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:59.071203  5972 raft_consensus.cc:385] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.071501  5972 raft_consensus.cc:740] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.071679  5774 raft_consensus.cc:2393] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:07:59.075223  5815 leader_election.cc:304] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:59.076634  6221 raft_consensus.cc:3060] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:59.079428  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:59.081391  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:59.082347  5922 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:07:59.060426  5971 tablet_bootstrap.cc:492] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:59.090013  6073 consensus_queue.cc:1048] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:59.092736  5970 raft_consensus.cc:359] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.093958  5970 raft_consensus.cc:385] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.094242  5970 raft_consensus.cc:740] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.094853  5970 consensus_queue.cc:260] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.100461  5816 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:07:59.102383  5973 tablet_bootstrap.cc:492] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:59.105381  5973 ts_tablet_manager.cc:1403] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.257s	user 0.042s	sys 0.012s
I20260709 13:07:59.104362  5970 ts_tablet_manager.cc:1434] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 13:07:59.109393  5970 tablet_bootstrap.cc:492] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:59.115417  5973 raft_consensus.cc:359] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.116204  5973 raft_consensus.cc:385] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.116549  5973 raft_consensus.cc:740] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.120525  5971 tablet_bootstrap.cc:654] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.112864  5972 consensus_queue.cc:260] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.122604  5970 tablet_bootstrap.cc:654] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.124349  5972 ts_tablet_manager.cc:1434] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.108s	user 0.004s	sys 0.004s
I20260709 13:07:59.124549  5973 consensus_queue.cc:260] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.125496  5972 tablet_bootstrap.cc:492] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:59.126681  5973 ts_tablet_manager.cc:1434] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:07:59.127676  5973 tablet_bootstrap.cc:492] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:59.129432  6200 raft_consensus.cc:493] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.130188  6200 raft_consensus.cc:515] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.135676  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:59.138849  6220 raft_consensus.cc:2804] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:07:59.141163  5667 leader_election.cc:343] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.142640  6221 raft_consensus.cc:2749] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:59.146602  5972 tablet_bootstrap.cc:654] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.159446  6201 consensus_queue.cc:1048] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:59.170082  5973 tablet_bootstrap.cc:654] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.171509  6220 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:07:59.172091  6220 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:59.185936  6200 leader_election.cc:290] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:59.187623  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:07:59.189415  5667 leader_election.cc:343] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.189878  5667 leader_election.cc:304] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:59.191192  6200 raft_consensus.cc:2749] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:59.197806  6220 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.204474  5599 catalog_manager.cc:5697] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 1 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:59.200647  6220 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:59.210361  5598 catalog_manager.cc:5697] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:59.212702  6218 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.215711  6218 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.216210  5598 catalog_manager.cc:5697] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:59.218250  5847 raft_consensus.cc:1240] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:59.220181  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:59.220285  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:59.207568  6000 raft_consensus.cc:493] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.222607  6000 raft_consensus.cc:515] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.223292  6218 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:59.225104  6000 leader_election.cc:290] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:59.214938  5971 tablet_bootstrap.cc:492] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:59.226931  5971 ts_tablet_manager.cc:1403] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.169s	user 0.018s	sys 0.009s
I20260709 13:07:59.229110  5970 tablet_bootstrap.cc:492] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:59.229770  5970 ts_tablet_manager.cc:1403] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.121s	user 0.014s	sys 0.003s
I20260709 13:07:59.239192  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:59.239332  6000 consensus_queue.cc:1059] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:59.241066  6000 raft_consensus.cc:3055] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:59.241413  6000 raft_consensus.cc:740] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:07:59.242578  6000 consensus_queue.cc:260] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.244361  6000 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:59.247457  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:59.248241  5700 raft_consensus.cc:2468] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:07:59.250072  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:59.251078  5774 raft_consensus.cc:2393] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:07:59.254407  5815 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:59.255692  5972 tablet_bootstrap.cc:492] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:59.256976  5890 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:59.261072  5970 raft_consensus.cc:359] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.251097  5971 raft_consensus.cc:359] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.263978  6218 raft_consensus.cc:493] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.264122  5971 raft_consensus.cc:385] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.261201  6220 raft_consensus.cc:2749] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:59.264549  6218 raft_consensus.cc:515] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.265519  5971 raft_consensus.cc:740] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.262962  5970 raft_consensus.cc:385] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.266732  5970 raft_consensus.cc:740] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.267100  6218 leader_election.cc:290] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:59.261209  5972 ts_tablet_manager.cc:1403] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.136s	user 0.013s	sys 0.005s
I20260709 13:07:59.254444  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:59.268015  5971 consensus_queue.cc:260] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.263517  6070 raft_consensus.cc:3060] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.271216  5971 ts_tablet_manager.cc:1434] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.001s
I20260709 13:07:59.270799  5972 raft_consensus.cc:359] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.271797  5972 raft_consensus.cc:385] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.271647  5970 consensus_queue.cc:260] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.272294  5972 raft_consensus.cc:740] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.272854  5971 tablet_bootstrap.cc:492] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:59.257411  5739 leader_election.cc:304] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:59.274326  5970 ts_tablet_manager.cc:1434] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:07:59.274488  6073 raft_consensus.cc:2804] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:59.274991  6073 raft_consensus.cc:493] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.275337  6073 raft_consensus.cc:3060] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.275491  5970 tablet_bootstrap.cc:492] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:59.279098  6070 raft_consensus.cc:2749] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:59.283784  5742 leader_election.cc:343] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.288089  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:59.289006  5700 raft_consensus.cc:2393] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:07:59.293848  5970 tablet_bootstrap.cc:654] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.295996  6200 consensus_queue.cc:1048] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:59.297508  5971 tablet_bootstrap.cc:654] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.309499  6000 raft_consensus.cc:493] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.314049  6000 raft_consensus.cc:515] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.314565  6073 raft_consensus.cc:515] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.316890  6000 leader_election.cc:290] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:59.310719  5973 tablet_bootstrap.cc:492] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:59.318593  5973 ts_tablet_manager.cc:1403] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.191s	user 0.023s	sys 0.004s
W20260709 13:07:59.319676  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 62360.9us
I20260709 13:07:59.321197  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:59.322315  5973 raft_consensus.cc:359] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.323321  5973 raft_consensus.cc:385] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.323663  5973 raft_consensus.cc:740] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.324589  5973 consensus_queue.cc:260] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.313005  5970 tablet_bootstrap.cc:492] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:59.325701  5970 ts_tablet_manager.cc:1403] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.000s
I20260709 13:07:59.327188  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:07:59.329293  5742 leader_election.cc:343] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.329519  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:59.330227  6200 consensus_queue.cc:1048] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:59.331122  5890 leader_election.cc:304] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:59.352576  6070 raft_consensus.cc:3060] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:59.353574  5742 leader_election.cc:343] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.354328  5742 leader_election.cc:304] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:59.358147  6000 raft_consensus.cc:2749] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:59.360333  6070 raft_consensus.cc:2749] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:59.332314  5970 raft_consensus.cc:359] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.362072  5970 raft_consensus.cc:385] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.362546  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:59.363195  5700 raft_consensus.cc:3060] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.363391  6073 leader_election.cc:290] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:59.339874  5973 ts_tablet_manager.cc:1434] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:07:59.367009  5973 tablet_bootstrap.cc:492] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:59.366902  6124 consensus_queue.cc:1048] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:59.371234  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
W20260709 13:07:59.373521  5667 consensus_peers.cc:597] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:59.273072  5972 consensus_queue.cc:260] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.362389  5970 raft_consensus.cc:740] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.378677  5970 consensus_queue.cc:260] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.380563  5972 ts_tablet_manager.cc:1434] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.113s	user 0.008s	sys 0.000s
I20260709 13:07:59.381986  5972 tablet_bootstrap.cc:492] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:59.381393  5970 ts_tablet_manager.cc:1434] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.000s
I20260709 13:07:59.384862  5970 tablet_bootstrap.cc:492] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:59.390550  5700 raft_consensus.cc:2468] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:07:59.392108  5739 leader_election.cc:304] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:59.393110  6000 raft_consensus.cc:2804] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:59.393661  6000 raft_consensus.cc:697] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:07:59.394737  6000 consensus_queue.cc:237] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.399391  5973 tablet_bootstrap.cc:654] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.402016  5970 tablet_bootstrap.cc:654] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.432906  5972 tablet_bootstrap.cc:654] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.426425  6124 consensus_queue.cc:1048] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:59.435034  5971 tablet_bootstrap.cc:492] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:59.435612  5971 ts_tablet_manager.cc:1403] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.163s	user 0.014s	sys 0.000s
W20260709 13:07:59.427413  5742 leader_election.cc:343] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.447629  5971 raft_consensus.cc:359] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.448400  5971 raft_consensus.cc:385] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.448715  5971 raft_consensus.cc:740] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.449795  5971 consensus_queue.cc:260] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.452008  5971 ts_tablet_manager.cc:1434] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:07:59.452955  5971 tablet_bootstrap.cc:492] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:59.459187  6221 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.459970  6221 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.463749  6221 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:59.467715  5595 catalog_manager.cc:5697] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 1 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:59.473806  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:07:59.475622  5816 leader_election.cc:343] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.493774  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:59.494485  5773 raft_consensus.cc:2468] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:07:59.496794  6124 raft_consensus.cc:493] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.497684  6124 raft_consensus.cc:515] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.500697  6124 leader_election.cc:290] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:59.501911  6073 consensus_queue.cc:1048] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:59.504662  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:59.505872  5776 raft_consensus.cc:2468] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:07:59.505856  5815 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:59.506422  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:59.507340  6220 raft_consensus.cc:2804] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:07:59.507853  5667 leader_election.cc:343] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.509218  6220 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.510371  6220 raft_consensus.cc:3060] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.510890  5666 leader_election.cc:304] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:59.513789  5971 tablet_bootstrap.cc:654] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.515632  6124 raft_consensus.cc:2804] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:59.517294  6124 raft_consensus.cc:493] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.517792  6124 raft_consensus.cc:3060] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.514797  5973 tablet_bootstrap.cc:492] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:59.522563  5973 ts_tablet_manager.cc:1403] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.156s	user 0.018s	sys 0.009s
I20260709 13:07:59.532565  5970 tablet_bootstrap.cc:492] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:59.533077  5970 ts_tablet_manager.cc:1403] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.148s	user 0.014s	sys 0.009s
I20260709 13:07:59.538213  5973 raft_consensus.cc:359] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.539067  5973 raft_consensus.cc:385] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.539358  5973 raft_consensus.cc:740] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.540535  5970 raft_consensus.cc:359] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.541127  5973 consensus_queue.cc:260] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.541602  5970 raft_consensus.cc:385] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.546809  5970 raft_consensus.cc:740] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.547756  5970 consensus_queue.cc:260] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.549520  5972 tablet_bootstrap.cc:492] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:59.549659  5973 ts_tablet_manager.cc:1434] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:07:59.550172  5972 ts_tablet_manager.cc:1403] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.168s	user 0.005s	sys 0.009s
I20260709 13:07:59.550858  5973 tablet_bootstrap.cc:492] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:59.552763  6221 raft_consensus.cc:493] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.553308  6221 raft_consensus.cc:515] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.556699  6221 leader_election.cc:290] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:59.555588  6124 raft_consensus.cc:515] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.558367  5972 raft_consensus.cc:359] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.559077  5972 raft_consensus.cc:385] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.559351  5972 raft_consensus.cc:740] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.560215  5972 consensus_queue.cc:260] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.561728  6124 leader_election.cc:290] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:59.561854  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:59.562017  5972 ts_tablet_manager.cc:1434] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:07:59.563350  5972 tablet_bootstrap.cc:492] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
W20260709 13:07:59.563743  5667 leader_election.cc:343] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.564157  5970 ts_tablet_manager.cc:1434] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:07:59.564893  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:59.565128  5970 tablet_bootstrap.cc:492] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:59.569628  5700 raft_consensus.cc:2468] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:07:59.571411  5971 tablet_bootstrap.cc:492] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:59.571951  5971 ts_tablet_manager.cc:1403] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.119s	user 0.011s	sys 0.016s
I20260709 13:07:59.572001  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:59.572690  5775 raft_consensus.cc:3060] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.574082  5813 leader_election.cc:304] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:59.574009  6220 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.575095  5971 raft_consensus.cc:359] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.575114  6221 raft_consensus.cc:2804] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:59.576251  6221 raft_consensus.cc:493] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.576712  6221 raft_consensus.cc:3060] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.575420  6218 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.580070  5973 tablet_bootstrap.cc:654] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.580152  6200 raft_consensus.cc:493] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.581097  6200 raft_consensus.cc:515] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.580209  6218 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.575803  5971 raft_consensus.cc:385] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.588141  5971 raft_consensus.cc:740] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.589811  5971 consensus_queue.cc:260] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.589701  6000 consensus_queue.cc:1048] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:07:59.589495  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:59.598757  5850 raft_consensus.cc:2393] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:07:59.599243  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:59.599521  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:59.600015  5773 raft_consensus.cc:2468] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:59.595072  5971 ts_tablet_manager.cc:1434] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:07:59.608992  5890 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:59.610280  6070 raft_consensus.cc:2804] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:59.610412  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:59.610715  6070 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.611353  6070 raft_consensus.cc:3060] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:59.612790  5667 leader_election.cc:343] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.589987  6218 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:59.620538  6200 leader_election.cc:290] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:59.612792  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:59.625986  6220 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:59.607118  5971 tablet_bootstrap.cc:492] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:59.625018  6070 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.626178  5925 raft_consensus.cc:2468] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:07:59.631121  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:59.632499  5972 tablet_bootstrap.cc:654] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.633054  5775 raft_consensus.cc:2468] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:07:59.649108  5922 raft_consensus.cc:2393] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:07:59.657428  5816 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:59.659629  6220 raft_consensus.cc:2749] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:59.671005  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:59.672015  6221 raft_consensus.cc:515] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.674852  6221 leader_election.cc:290] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:59.676132  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:59.676704  5973 tablet_bootstrap.cc:492] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:59.676836  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:07:59.676924  5701 raft_consensus.cc:3060] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.677524  5922 raft_consensus.cc:3060] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.678956  5970 tablet_bootstrap.cc:654] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.686025  5922 raft_consensus.cc:2468] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:07:59.686311  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:07:59.687079  5773 raft_consensus.cc:3060] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.687467  5816 leader_election.cc:304] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:07:59.690500  6220 raft_consensus.cc:2804] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:59.691042  6220 raft_consensus.cc:697] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:07:59.691874  6220 consensus_queue.cc:237] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.696234  5773 raft_consensus.cc:2468] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:59.697355  5973 ts_tablet_manager.cc:1403] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.147s	user 0.007s	sys 0.007s
I20260709 13:07:59.697193  5666 leader_election.cc:304] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:07:59.697577  5890 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:07:59.698530  6124 raft_consensus.cc:2804] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:59.699044  6124 raft_consensus.cc:697] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:07:59.699875  6124 consensus_queue.cc:237] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.700093  5971 tablet_bootstrap.cc:654] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.698557  6218 raft_consensus.cc:2804] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:59.704607  6218 raft_consensus.cc:697] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:07:59.705452  6218 consensus_queue.cc:237] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.708268  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:59.709246  5850 raft_consensus.cc:2393] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:07:59.709321  5701 raft_consensus.cc:2468] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:07:59.714923  6070 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:07:59.700845  5973 raft_consensus.cc:359] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.717850  5973 raft_consensus.cc:385] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.718097  5598 catalog_manager.cc:5697] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 1 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:59.718146  5973 raft_consensus.cc:740] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
W20260709 13:07:59.716312  5666 leader_election.cc:343] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.721750  5666 leader_election.cc:304] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:59.722599  6201 raft_consensus.cc:2749] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:59.728664  5973 consensus_queue.cc:260] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.733579  5973 ts_tablet_manager.cc:1434] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.001s
I20260709 13:07:59.735028  5973 tablet_bootstrap.cc:492] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:59.742189  5972 tablet_bootstrap.cc:492] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:59.742812  5972 ts_tablet_manager.cc:1403] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.180s	user 0.011s	sys 0.003s
I20260709 13:07:59.740477  5597 catalog_manager.cc:5697] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 1 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:59.745896  5972 raft_consensus.cc:359] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.746814  5972 raft_consensus.cc:385] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.747144  5972 raft_consensus.cc:740] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.748344  5972 consensus_queue.cc:260] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.749610  5973 tablet_bootstrap.cc:654] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.749366  5595 catalog_manager.cc:5697] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:59.752581  5970 tablet_bootstrap.cc:492] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:59.756290  5970 ts_tablet_manager.cc:1403] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.191s	user 0.011s	sys 0.002s
I20260709 13:07:59.754583  5972 ts_tablet_manager.cc:1434] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 13:07:59.762969  5971 tablet_bootstrap.cc:492] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:59.763188  5972 tablet_bootstrap.cc:492] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:59.763540  5971 ts_tablet_manager.cc:1403] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.159s	user 0.013s	sys 0.002s
I20260709 13:07:59.767472  5971 raft_consensus.cc:359] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.768393  5971 raft_consensus.cc:385] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.769225  5971 raft_consensus.cc:740] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.770028  5971 consensus_queue.cc:260] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.772271  5971 ts_tablet_manager.cc:1434] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 13:07:59.772296  6000 raft_consensus.cc:493] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.773111  6000 raft_consensus.cc:515] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.776835  5973 tablet_bootstrap.cc:492] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:59.777366  5973 ts_tablet_manager.cc:1403] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.004s
I20260709 13:07:59.781354  6201 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.781869  6201 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.782112  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:59.783973  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:07:59.784260  5742 leader_election.cc:343] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.785380  6201 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:07:59.786803  5973 raft_consensus.cc:359] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.787518  5973 raft_consensus.cc:385] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.808568  5973 raft_consensus.cc:740] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.809494  5973 consensus_queue.cc:260] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.759681  5970 raft_consensus.cc:359] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.810359  5970 raft_consensus.cc:385] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.810731  5970 raft_consensus.cc:740] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.811784  5973 ts_tablet_manager.cc:1434] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.000s
I20260709 13:07:59.811467  5970 consensus_queue.cc:260] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.813683  5970 ts_tablet_manager.cc:1434] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
I20260709 13:07:59.814699  5970 tablet_bootstrap.cc:492] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:59.795756  5972 tablet_bootstrap.cc:654] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.775373  6000 leader_election.cc:290] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:07:59.788004  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:59.817131  5971 tablet_bootstrap.cc:492] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
W20260709 13:07:59.794793  5742 leader_election.cc:343] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.817762  5742 leader_election.cc:304] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:59.820099  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:07:59.821107  6000 raft_consensus.cc:2749] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:59.822783  5667 leader_election.cc:343] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.825094  5971 tablet_bootstrap.cc:654] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:59.826771  5666 leader_election.cc:343] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.827289  5666 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:07:59.828516  6158 raft_consensus.cc:2749] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:59.833072  5973 tablet_bootstrap.cc:492] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:59.840380  5973 tablet_bootstrap.cc:654] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.847859  5970 tablet_bootstrap.cc:654] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.856907  5972 tablet_bootstrap.cc:492] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:07:59.857822  5972 ts_tablet_manager.cc:1403] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.095s	user 0.015s	sys 0.000s
I20260709 13:07:59.861807  5972 raft_consensus.cc:359] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.864658  5970 tablet_bootstrap.cc:492] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:07:59.865927  5973 tablet_bootstrap.cc:492] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:07:59.867125  5970 ts_tablet_manager.cc:1403] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.008s
I20260709 13:07:59.867225  5973 ts_tablet_manager.cc:1403] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.004s
I20260709 13:07:59.870128  5973 raft_consensus.cc:359] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.870875  5973 raft_consensus.cc:385] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.871218  5973 raft_consensus.cc:740] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.871810  5971 tablet_bootstrap.cc:492] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:07:59.872442  5971 ts_tablet_manager.cc:1403] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.056s	user 0.019s	sys 0.024s
I20260709 13:07:59.872584  5973 consensus_queue.cc:260] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.874967  5973 ts_tablet_manager.cc:1434] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:07:59.875003  6218 raft_consensus.cc:493] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.875289  5971 raft_consensus.cc:359] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.875590  6218 raft_consensus.cc:515] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.876116  5971 raft_consensus.cc:385] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.876513  5971 raft_consensus.cc:740] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.877239  5971 consensus_queue.cc:260] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:07:59.877736  6218 leader_election.cc:290] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:59.866155  5972 raft_consensus.cc:385] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.871232  5970 raft_consensus.cc:359] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.878970  5970 raft_consensus.cc:385] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:59.878793  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:07:59.879350  5970 raft_consensus.cc:740] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.879370  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:59.879639  5850 raft_consensus.cc:2468] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:59.880184  5701 raft_consensus.cc:2468] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:07:59.880262  5970 consensus_queue.cc:260] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.882682  5970 ts_tablet_manager.cc:1434] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:07:59.883417  5888 leader_election.cc:304] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:07:59.884478  6218 raft_consensus.cc:2804] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:59.884919  6218 raft_consensus.cc:493] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:59.885233  6218 raft_consensus.cc:3060] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.878824  5972 raft_consensus.cc:740] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:07:59.891659  5972 consensus_queue.cc:260] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:07:59.903129  5972 ts_tablet_manager.cc:1434] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.000s
I20260709 13:07:59.904353  5970 tablet_bootstrap.cc:492] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:07:59.907401  5972 tablet_bootstrap.cc:492] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:07:59.909271  5850 raft_consensus.cc:3060] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.910326  5970 tablet_bootstrap.cc:654] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.912693  5973 tablet_bootstrap.cc:492] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:07:59.914378  6000 consensus_queue.cc:1048] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:59.904407  5971 ts_tablet_manager.cc:1434] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 13:07:59.921546  6255 consensus_queue.cc:1048] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:59.924623  5971 tablet_bootstrap.cc:492] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:07:59.927009  6218 raft_consensus.cc:515] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.929674  6218 leader_election.cc:290] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:59.929834  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:07:59.930529  5849 raft_consensus.cc:3060] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.931535  5973 tablet_bootstrap.cc:654] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.932323  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:07:59.932931  5700 raft_consensus.cc:3060] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.935984  6220 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.936518  6220 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:07:59.937865  5972 tablet_bootstrap.cc:654] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:59.939352  6220 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:59.939785  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:07:59.941157  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:59.943186  5816 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:07:59.944696  6220 raft_consensus.cc:2749] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:59.945854  6000 raft_consensus.cc:493] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:59.946486  6000 raft_consensus.cc:515] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:07:59.949893  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:07:59.950734  6000 leader_election.cc:290] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:07:59.950976  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
W20260709 13:07:59.951370  5742 leader_election.cc:343] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:59.951680  5699 raft_consensus.cc:2468] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:07:59.965523  5739 leader_election.cc:304] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:07:59.978092  5851 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:59.985360  5849 raft_consensus.cc:2468] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:07:59.977910  5700 raft_consensus.cc:2468] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:00.104543  5971 tablet_bootstrap.cc:654] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:00.116520  5891 leader_election.cc:304] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
W20260709 13:08:00.183045  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 66715.2us
I20260709 13:08:00.121102  6073 raft_consensus.cc:2804] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:00.187621  6255 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.258237  5984 raft_consensus.cc:2804] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:00.437498  6218 raft_consensus.cc:493] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:00.312821  6073 raft_consensus.cc:493] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:00.483739  5984 raft_consensus.cc:697] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:00.486193  6073 raft_consensus.cc:3060] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:00.442169  6255 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:00.466023  6218 raft_consensus.cc:515] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:00.775741  5971 tablet_bootstrap.cc:492] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:00.776590  5971 ts_tablet_manager.cc:1403] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.852s	user 0.025s	sys 0.031s
W20260709 13:08:00.820113  5890 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.003s	sys 0.006s
I20260709 13:08:00.788553  5984 consensus_queue.cc:237] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:00.903888  5889 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.009s	sys 0.028s
I20260709 13:08:00.916847  5971 raft_consensus.cc:359] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:00.919670  5971 raft_consensus.cc:385] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:00.920133  5971 raft_consensus.cc:740] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:00.923036  5971 consensus_queue.cc:260] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:00.927757  5970 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.993s	user 0.022s	sys 0.039s
W20260709 13:08:00.928268  5970 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.022s	sys 0.039s
W20260709 13:08:00.956557  5973 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 1.021s	user 0.021s	sys 0.040s
W20260709 13:08:00.920639  5889 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.009s	sys 0.028s
W20260709 13:08:00.957002  5973 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 1.022s	user 0.021s	sys 0.040s
I20260709 13:08:00.925915  6255 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:00.963639  5971 ts_tablet_manager.cc:1434] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.187s	user 0.008s	sys 0.002s
I20260709 13:08:00.968036  5971 tablet_bootstrap.cc:492] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:00.977746  6283 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:01.110318  5970 tablet_bootstrap.cc:492] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:01.110927  5970 ts_tablet_manager.cc:1403] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 1.207s	user 0.032s	sys 0.039s
I20260709 13:08:01.033656  6283 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.117250  5973 tablet_bootstrap.cc:492] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:08:01.118008  5973 ts_tablet_manager.cc:1403] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 1.206s	user 0.029s	sys 0.044s
W20260709 13:08:01.121740  6073 consensus_meta.cc:287] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.004s
I20260709 13:08:01.122258  6073 raft_consensus.cc:515] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.124167  6073 leader_election.cc:290] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.123558  5970 raft_consensus.cc:359] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.127115  5970 raft_consensus.cc:385] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:01.127254  6283 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.127507  5970 raft_consensus.cc:740] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:08:01.128379  5970 consensus_queue.cc:260] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.130090  5970 ts_tablet_manager.cc:1434] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:08:01.131539  5970 tablet_bootstrap.cc:492] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:01.132128  6287 raft_consensus.cc:493] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:01.132746  6287 raft_consensus.cc:515] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.135571  6287 leader_election.cc:290] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.137913  5971 tablet_bootstrap.cc:654] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:01.136528  5973 raft_consensus.cc:359] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.142992  5973 raft_consensus.cc:385] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:01.143395  5973 raft_consensus.cc:740] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:01.144258  5973 consensus_queue.cc:260] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.139581  5970 tablet_bootstrap.cc:654] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:01.147528  5973 ts_tablet_manager.cc:1434] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 13:08:01.148679  5973 tablet_bootstrap.cc:492] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:01.151531  6291 raft_consensus.cc:493] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.152302  6291 raft_consensus.cc:515] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.156776  6291 leader_election.cc:290] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.157070  5973 tablet_bootstrap.cc:654] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:01.157573  5970 tablet_bootstrap.cc:492] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:01.158161  5970 ts_tablet_manager.cc:1403] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.003s
I20260709 13:08:01.161795  5970 raft_consensus.cc:359] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.164880  5971 tablet_bootstrap.cc:492] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:01.165901  5971 ts_tablet_manager.cc:1403] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.198s	user 0.013s	sys 0.002s
I20260709 13:08:01.168782  5973 tablet_bootstrap.cc:492] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:08:01.169445  5973 ts_tablet_manager.cc:1403] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.007s
I20260709 13:08:01.169744  5970 raft_consensus.cc:385] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:01.170074  5970 raft_consensus.cc:740] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:08:01.170818  5970 consensus_queue.cc:260] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.172268  5973 raft_consensus.cc:359] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.172742  5970 ts_tablet_manager.cc:1434] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:08:01.173000  5973 raft_consensus.cc:385] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:01.173310  5973 raft_consensus.cc:740] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:01.173871  5970 tablet_bootstrap.cc:492] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:01.173971  5973 consensus_queue.cc:260] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:01.174520  5701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.249s	user 0.008s	sys 0.001s
I20260709 13:08:01.175719  5973 ts_tablet_manager.cc:1434] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20260709 13:08:01.168917  5971 raft_consensus.cc:359] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.177134  5971 raft_consensus.cc:385] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:01.177627  5971 raft_consensus.cc:740] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:01.178500  5971 consensus_queue.cc:260] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.180958  5971 ts_tablet_manager.cc:1434] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.002s
I20260709 13:08:01.182504  5971 tablet_bootstrap.cc:492] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:01.184234  5970 tablet_bootstrap.cc:654] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:01.188328  5973 tablet_bootstrap.cc:492] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:01.193737  5971 tablet_bootstrap.cc:654] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:01.194648  5970 tablet_bootstrap.cc:492] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:01.195171  5970 ts_tablet_manager.cc:1403] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.003s
I20260709 13:08:01.198472  5973 tablet_bootstrap.cc:654] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:01.198092  5970 raft_consensus.cc:359] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.198956  5970 raft_consensus.cc:385] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:01.199251  5970 raft_consensus.cc:740] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:08:01.200054  5970 consensus_queue.cc:260] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.202103  5970 ts_tablet_manager.cc:1434] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:08:01.203430  5970 tablet_bootstrap.cc:492] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:01.211177  5973 tablet_bootstrap.cc:492] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:08:01.211761  5973 ts_tablet_manager.cc:1403] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.009s
I20260709 13:08:01.212877  5970 tablet_bootstrap.cc:654] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:01.219632  5971 tablet_bootstrap.cc:492] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:01.220448  5971 ts_tablet_manager.cc:1403] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.003s
I20260709 13:08:01.224440  5971 raft_consensus.cc:359] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.225453  5971 raft_consensus.cc:385] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:01.225796  5971 raft_consensus.cc:740] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:01.230289  5970 tablet_bootstrap.cc:492] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:01.231153  5970 ts_tablet_manager.cc:1403] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.003s
I20260709 13:08:01.234978  6287 raft_consensus.cc:493] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:01.235953  6287 raft_consensus.cc:515] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.240728  6287 leader_election.cc:290] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:01.225135  5814 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.013s	sys 0.023s
W20260709 13:08:01.244582  5814 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.014s	sys 0.024s
I20260709 13:08:01.245970  6227 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.246102  6285 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.247177  6283 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.246765  6285 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.247246  6282 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
W20260709 13:08:01.177642  5701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.252s	user 0.010s	sys 0.002s
I20260709 13:08:01.247790  6283 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.248131  6282 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.249068  5701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44372 (request call id 34) took 1332 ms. Trace:
I20260709 13:08:01.250319  6282 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.251643  6283 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.249529  5701 rpcz_store.cc:276] 0709 13:07:59.916916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.923801 (+  6885us) service_pool.cc:224] Handling call
0709 13:07:59.924210 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:59.924617 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:59.924657 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:59.925183 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 13:08:01.248677 (+1323494us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:08:01.248720 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.248805 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:01.248853 (+    48us) raft_consensus.cc:1685] finished
0709 13:08:01.248870 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:01.249005 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1222,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":1323055,"threads_started":1,"wal-append.queue_time_us":54398,"wal-append.run_cpu_time_us":2242,"wal-append.run_wall_time_us":1002255}
I20260709 13:08:01.253159  6287 raft_consensus.cc:493] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:01.253835  6287 raft_consensus.cc:515] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.259487  6287 leader_election.cc:290] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.321208  6287 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.319936  6282 raft_consensus.cc:493] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.323405  6287 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.325323  6282 raft_consensus.cc:515] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.328576  6287 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.337790  6301 raft_consensus.cc:493] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:01.339103  6301 raft_consensus.cc:515] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.355130  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.360110  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:01.362819  6073 raft_consensus.cc:493] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.363704  6280 raft_consensus.cc:493] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.363940  6073 raft_consensus.cc:515] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.364735  6280 raft_consensus.cc:515] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.362278  6302 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:01.368661  6281 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.368454  6073 leader_election.cc:290] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.374540  6073 raft_consensus.cc:493] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.375976  6073 raft_consensus.cc:515] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.379484  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.380462  6073 leader_election.cc:290] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.372152  6302 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.372794  6281 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.384354  6281 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.388062  6280 leader_election.cc:290] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.388386  6302 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:01.391242  5972 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 1.448s	user 0.080s	sys 0.130s
W20260709 13:08:01.391723  5972 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 1.449s	user 0.080s	sys 0.130s
I20260709 13:08:01.396838  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.397575  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:08:01.397855  5742 leader_election.cc:343] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:01.397994  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:08:01.403740  5742 leader_election.cc:343] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:01.407562  6303 raft_consensus.cc:493] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.408699  6303 raft_consensus.cc:515] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.415632  6280 raft_consensus.cc:493] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.416705  6280 raft_consensus.cc:515] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.418232  6287 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.419179  6287 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.419554  6280 leader_election.cc:290] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.420830  6283 raft_consensus.cc:493] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.421932  6283 raft_consensus.cc:515] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.422407  6287 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.426189  6283 leader_election.cc:290] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.461822  6220 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.462828  6220 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:01.466053  5816 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.981s	user 0.002s	sys 0.015s
W20260709 13:08:01.466648  5816 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.003s	sys 0.016s
W20260709 13:08:01.467053  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 1.05877e+06us
W20260709 13:08:01.480208  5813 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.020s	sys 0.040s
W20260709 13:08:01.480772  5813 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.020s	sys 0.040s
I20260709 13:08:01.485980  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.488392  6304 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.489480  6304 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.492450  6304 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.493606  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.496206  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.496958  5851 raft_consensus.cc:2468] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:01.498265  6283 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:01.499115  6283 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.501111  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:01.502187  5775 raft_consensus.cc:2468] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:01.503249  6283 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.504067  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.504820  5851 raft_consensus.cc:2468] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:01.507684  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.508782  5851 raft_consensus.cc:2468] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:01.510409  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.512058  5849 raft_consensus.cc:2468] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:08:01.521034  5742 leader_election.cc:304] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:01.522362  6255 raft_consensus.cc:2804] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:01.522912  6255 raft_consensus.cc:493] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.523394  6255 raft_consensus.cc:3060] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.528029  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:01.529978  6218 leader_election.cc:290] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.531315  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:01.532334  5774 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:01.532464  6304 raft_consensus.cc:493] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:01.532660  5742 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:01.533581  6280 raft_consensus.cc:2804] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:01.534006  6280 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.534415  6280 raft_consensus.cc:3060] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.535768  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:01.536710  5773 raft_consensus.cc:2468] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:01.533263  6304 raft_consensus.cc:515] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.544823  6304 leader_election.cc:290] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.545499  6304 raft_consensus.cc:493] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.546222  6304 raft_consensus.cc:515] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.548653  5742 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:01.549265  6280 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.549647  6302 raft_consensus.cc:2804] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:01.550144  6302 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:01.550529  6302 raft_consensus.cc:3060] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.550806  5742 leader_election.cc:304] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:01.551708  6281 raft_consensus.cc:2804] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:01.552098  6281 raft_consensus.cc:493] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.552433  6281 raft_consensus.cc:3060] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.246608  6227 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.563721  6302 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.567813  6227 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
W20260709 13:08:01.568307  5741 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.162s	user 0.025s	sys 0.023s
W20260709 13:08:01.568849  5741 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.185s	user 0.026s	sys 0.023s
I20260709 13:08:01.568711  5970 raft_consensus.cc:359] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.570940  5970 raft_consensus.cc:385] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:01.570748  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:01.571379  5970 raft_consensus.cc:740] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:08:01.572412  5970 consensus_queue.cc:260] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.573697  6218 raft_consensus.cc:493] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.574587  6218 raft_consensus.cc:515] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.577502  6303 leader_election.cc:290] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.579779  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.580585  6255 raft_consensus.cc:515] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.581467  6302 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.559736  5971 consensus_queue.cc:260] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.586977  6304 leader_election.cc:290] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.587090  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:01.589967  6304 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.588047  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.590989  6304 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.591382  5924 raft_consensus.cc:2468] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:01.585101  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.592922  5922 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:01.588527  6255 leader_election.cc:290] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.590108  5849 raft_consensus.cc:3060] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.589172  6280 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.598202  5599 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39364 (request call id 59) took 1646 ms. Trace:
I20260709 13:08:01.599542  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:01.599977  6302 raft_consensus.cc:493] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:01.600297  6282 leader_election.cc:290] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.601796  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.612569  6255 raft_consensus.cc:493] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.612982  5846 raft_consensus.cc:2468] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:08:01.613241  6255 raft_consensus.cc:515] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.603598  5971 ts_tablet_manager.cc:1434] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.383s	user 0.010s	sys 0.004s
I20260709 13:08:01.606199  6306 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.614637  6306 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.617138  6306 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.606282  5815 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:01.568810  6227 raft_consensus.cc:493] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.618357  5815 leader_election.cc:304] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:01.618624  6227 raft_consensus.cc:515] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.620453  6218 leader_election.cc:290] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.622174  5970 ts_tablet_manager.cc:1434] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.391s	user 0.012s	sys 0.002s
I20260709 13:08:01.623858  5970 tablet_bootstrap.cc:492] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:01.624570  6220 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.625036  5972 tablet_bootstrap.cc:492] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:01.626009  5972 ts_tablet_manager.cc:1403] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 1.719s	user 0.096s	sys 0.130s
I20260709 13:08:01.626555  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.630059  5846 raft_consensus.cc:2393] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:01.631816  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:01.632139  6218 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.632723  5850 raft_consensus.cc:3060] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.632843  6218 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.637673  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:08:01.610548  5847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.682s	user 0.094s	sys 0.182s
W20260709 13:08:01.640138  5847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.713s	user 0.095s	sys 0.183s
I20260709 13:08:01.640879  6282 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:01.641335  5847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57468 (request call id 38) took 1717 ms. Trace:
I20260709 13:08:01.642118  6306 raft_consensus.cc:2804] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:01.642688  6306 raft_consensus.cc:493] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:01.643260  6306 raft_consensus.cc:3060] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.641882  5847 rpcz_store.cc:276] 0709 13:07:59.924095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.925456 (+  1361us) service_pool.cc:224] Handling call
0709 13:07:59.925834 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:07:59.926227 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:07:59.926264 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:07:59.926760 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:08:01.640794 (+1714034us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:08:01.640886 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:01.641002 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:01.641062 (+    60us) raft_consensus.cc:1685] finished
0709 13:08:01.641073 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:01.641265 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1226,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":1713446,"threads_started":1,"wal-append.queue_time_us":1043790}
I20260709 13:08:01.644737  5815 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:01.646135  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.647032  5845 raft_consensus.cc:2468] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:01.611635  6302 raft_consensus.cc:515] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.650079  6302 leader_election.cc:290] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.650746  6302 raft_consensus.cc:493] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.630523  6277 raft_consensus.cc:493] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:01.652031  6218 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.652433  6277 raft_consensus.cc:515] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.652988  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.653882  5845 raft_consensus.cc:2468] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:01.653954  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.654973  6277 leader_election.cc:290] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.656734  6220 raft_consensus.cc:2804] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:01.657305  6220 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.657750  6220 raft_consensus.cc:3060] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.641390  6282 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.609846  5599 rpcz_store.cc:276] 0709 13:07:59.952024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.952189 (+   165us) service_pool.cc:224] Handling call
0709 13:08:01.597142 (+1644953us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:08:01.612022  6073 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:01.662588  6073 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.665207  5595 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39344 (request call id 65) took 1731 ms. Trace:
I20260709 13:08:01.665434  6073 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.666170  6073 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.665740  5595 rpcz_store.cc:276] 0709 13:07:59.933887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:59.934053 (+   166us) service_pool.cc:224] Handling call
0709 13:08:01.664675 (+1730622us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b240068d098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f7833bb8970 00007f7833bc5a8f 00007f783c9cae14 00007f783ae8b2a4 00007f783ae8b232 00007f783ae8b161 00007f783ae8b0dc
0709 13:08:01.665020 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26767,"reported_tablets":1,"spinlock_wait_cycles":3746446592}
I20260709 13:08:01.667593  6301 leader_election.cc:290] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.669795  5598 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39354 (request call id 53) took 1411 ms. Trace:
I20260709 13:08:01.630668  5972 raft_consensus.cc:359] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.673596  5972 raft_consensus.cc:385] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:01.673916  5972 raft_consensus.cc:740] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:01.674940  5972 consensus_queue.cc:260] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:01.675839  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 343540us
I20260709 13:08:01.676934  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:01.672866  5598 rpcz_store.cc:276] 0709 13:08:00.390471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:00.857151 (+466680us) service_pool.cc:224] Handling call
0709 13:08:01.669176 (+812025us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b240068d098. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f7833bb8970 00007f7833bc5a8f 00007f783c9cae14 00007f783ae8b2a4 00007f783ae8b232 00007f783ae8b161 00007f783ae8b0dc
0709 13:08:01.669559 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":1035030272}
I20260709 13:08:01.603399  6303 consensus_queue.cc:1048] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:08:01.666994  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.669325  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.666996  6073 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.651372  6302 raft_consensus.cc:515] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.679231  6255 leader_election.cc:290] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.667147  6282 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.682216  6281 raft_consensus.cc:515] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.675732  5973 raft_consensus.cc:359] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.691792  5973 raft_consensus.cc:385] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:01.691890  6304 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.692206  5973 raft_consensus.cc:740] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:01.693100  5973 consensus_queue.cc:260] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.693531  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.681319  5847 raft_consensus.cc:3060] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.679411  6289 raft_consensus.cc:493] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:01.720863  5972 ts_tablet_manager.cc:1434] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.094s	user 0.005s	sys 0.004s
W20260709 13:08:01.725188  5742 leader_election.cc:343] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:01.682430  5845 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:01.739603  6255 raft_consensus.cc:493] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:01.759572  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 82145.3us
I20260709 13:08:01.749456  6227 leader_election.cc:290] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.763523  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.780038  6294 raft_consensus.cc:493] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.780721  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:08:01.781395  5739 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.418s	user 0.006s	sys 0.008s
I20260709 13:08:01.762650  6255 raft_consensus.cc:515] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.776861  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.785591  6280 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.780082  6279 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
W20260709 13:08:01.788159  5739 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.482s	user 0.007s	sys 0.008s
I20260709 13:08:01.788432  6073 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.803471  6279 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.805303  6280 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.806165  6279 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.807475  6309 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.807996  6294 raft_consensus.cc:515] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.808209  6309 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.810642  6309 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.811237  6309 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:01.769636  6289 raft_consensus.cc:515] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.817523  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.811815  6309 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.796583  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.807731  6280 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.816694  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:01.816825  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:01.795975  5850 raft_consensus.cc:2468] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:01.852123  6073 raft_consensus.cc:493] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.820487  6309 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.852432  6307 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.852792  6073 raft_consensus.cc:515] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.853304  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.853364  6307 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.855103  6073 leader_election.cc:290] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.794926  5816 leader_election.cc:304] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:01.855835  6307 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.856513  6303 raft_consensus.cc:2804] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:01.857019  6303 raft_consensus.cc:493] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:01.857384  6303 raft_consensus.cc:3060] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:01.819315  6289 leader_election.cc:290] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.858228  5776 raft_consensus.cc:2393] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 2: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:01.850137  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:01.860316  5775 raft_consensus.cc:2468] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
W20260709 13:08:01.856444  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 68813.4us
I20260709 13:08:01.860730  5891 leader_election.cc:304] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:01.861485  5816 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:01.810642  6294 leader_election.cc:290] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.848853  6302 leader_election.cc:290] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.862843  5891 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:01.864558  6279 consensus_queue.cc:1048] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:01.866199  6277 raft_consensus.cc:2804] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:01.800468  5849 raft_consensus.cc:2468] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:01.822527  5972 tablet_bootstrap.cc:492] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:01.869429  5973 ts_tablet_manager.cc:1434] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.657s	user 0.027s	sys 0.042s
W20260709 13:08:01.868289  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 88476.1us
I20260709 13:08:01.880332  6284 raft_consensus.cc:2804] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:01.880927  6284 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.881392  6284 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.880199  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.822201  5922 raft_consensus.cc:2468] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:01.873421  6281 leader_election.cc:290] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.817863  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:01.878742  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:01.890367  6312 raft_consensus.cc:493] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:01.911468  6312 raft_consensus.cc:515] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:01.891258  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:01.915130  5850 raft_consensus.cc:2468] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:01.920131  6285 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.894040  6220 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.872295  6227 raft_consensus.cc:493] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:01.895390  5847 raft_consensus.cc:2468] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:01.896471  6308 raft_consensus.cc:493] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:01.897472  6305 raft_consensus.cc:493] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.900152  6255 leader_election.cc:290] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:01.841323  6282 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:01.874681  6277 raft_consensus.cc:493] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:01.876425  5973 ts_tablet_manager.cc:1467] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: Tablet startup took 664ms
I20260709 13:08:01.873593  5773 raft_consensus.cc:2468] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:01.913997  6312 leader_election.cc:290] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:01.907152  6307 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.921161  5846 raft_consensus.cc:1240] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:01.945192  6277 raft_consensus.cc:3060] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:01.940725  5973 ts_tablet_manager.cc:1469] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: Trace:
0709 13:07:55.077137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.084518 (+  7381us) service_pool.cc:224] Handling call
0709 13:07:55.124294 (+ 39776us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:07:55.124409 (+   115us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:07:55.174725 (+ 50316us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:55.316989 (+142264us) tablet_replica.cc:206] Creating consensus instance
0709 13:07:55.317422 (+   433us) inbound_call.cc:177] Queueing success response
0709 13:08:01.187946 (+5870524us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:08:01.212055 (+ 24109us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:08:01.212582 (+   527us) tablet_replica.cc:261] Starting instrumentation
0709 13:08:01.213187 (+   605us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":459942,"raft.queue_time_us":179480,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":20439,"tablet-open.queue_time_us":5870561,"thread_start_us":82262,"threads_started":1}
I20260709 13:08:01.956547  6282 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.958599  5972 tablet_bootstrap.cc:654] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:01.932539  6283 raft_consensus.cc:2804] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:08:01.929816  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 67164.3us
I20260709 13:08:01.938659  5772 raft_consensus.cc:2468] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:01.945508  6279 raft_consensus.cc:493] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.935627  6227 raft_consensus.cc:515] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:01.946704  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 57901.5us
I20260709 13:08:01.977948  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:01.978365  5973 tablet_bootstrap.cc:492] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:01.978444  5971 tablet_bootstrap.cc:492] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:01.980141  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:01.980755  6307 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:01.981245  6318 raft_consensus.cc:493] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:01.998045  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:01.970259  6308 raft_consensus.cc:515] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.998531  5970 tablet_bootstrap.cc:654] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:02.000358  6308 leader_election.cc:290] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.003187  6303 raft_consensus.cc:515] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.004885  6302 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:02.005125  6312 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.005594  6302 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.976195  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.004961  6310 raft_consensus.cc:493] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.006781  5776 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:02.006968  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:02.006341  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:01.986182  5844 raft_consensus.cc:1240] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:01.984261  6313 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:02.008412  6313 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.009788  6307 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:02.011067  6313 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:01.984948  6311 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:01.991724  6279 raft_consensus.cc:515] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.012276  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:01.968887  6283 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.013232  6283 raft_consensus.cc:3060] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.014660  6279 leader_election.cc:290] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.014953  6220 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.991830  5891 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:02.016642  5891 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:02.006982  6310 raft_consensus.cc:515] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:01.988827  5843 raft_consensus.cc:1240] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:01.995581  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.020303  6218 raft_consensus.cc:2804] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.022845  6218 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.023280  6218 raft_consensus.cc:3060] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:01.997870  6318 raft_consensus.cc:515] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.025633  6318 leader_election.cc:290] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:02.026129  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.005702  6312 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.020540  6302 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.028204  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.028491  6290 raft_consensus.cc:493] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:02.029304  6312 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.977433  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.029938  6227 leader_election.cc:290] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:02.030154  5774 raft_consensus.cc:2468] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:02.030804  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.031070  6070 raft_consensus.cc:2804] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.031383  5695 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:08:02.031553  6070 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.032037  5694 raft_consensus.cc:3060] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:02.032068  6070 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.012502  6311 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.037576  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:02.028198  6316 raft_consensus.cc:493] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:02.038715  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.038993  6311 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:02.038630  6316 raft_consensus.cc:515] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.039616  5924 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:02.011759  5775 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:02.040015  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:01.976112  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.040825  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.041393  6316 leader_election.cc:290] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:02.036422  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.042412  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.043172  6310 leader_election.cc:290] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:01.964450  6305 raft_consensus.cc:515] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.044162  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.044956  5920 raft_consensus.cc:2468] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:02.046164  6305 leader_election.cc:290] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.046499  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.029201  6290 raft_consensus.cc:515] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.048004  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.048153  6279 raft_consensus.cc:493] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.048789  5920 raft_consensus.cc:2468] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:02.048841  6279 raft_consensus.cc:515] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.049899  6290 leader_election.cc:290] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:02.051216  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.051931  5699 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:01.987010  6284 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.063796  6279 leader_election.cc:290] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:02.064765  5815 leader_election.cc:304] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:02.065097  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.067420  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.005746  6303 leader_election.cc:290] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:02.067971  5776 raft_consensus.cc:1240] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.069504  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.071326  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.073946  6218 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.069126  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.080430  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:02.084636  6073 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.084735  5850 raft_consensus.cc:3060] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.085453  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.085839  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:02.087038  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.027242  6318 raft_consensus.cc:493] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.081032  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.087749  5692 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:08:02.087935  6318 raft_consensus.cc:515] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.085493  6073 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.090358  6318 leader_election.cc:290] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:02.012298  6306 raft_consensus.cc:515] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.091483  6073 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:02.091625  5891 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:02.092429  6306 leader_election.cc:290] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:02.080421  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.076061  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.075655  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.069275  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:02.076251  6227 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:02.100531  6227 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.102448  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.103116  5773 raft_consensus.cc:2468] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:08:02.099062  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.100919  5771 raft_consensus.cc:1240] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.079376  5816 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:02.099485  5973 tablet_bootstrap.cc:654] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:02.094432  5851 raft_consensus.cc:2468] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:02.097678  6279 raft_consensus.cc:2804] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.099086  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:02.145129  5846 raft_consensus.cc:2393] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
W20260709 13:08:02.147670  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 68362.5us
I20260709 13:08:02.165761  5970 tablet_bootstrap.cc:492] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:02.166414  5970 ts_tablet_manager.cc:1403] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.543s	user 0.016s	sys 0.012s
I20260709 13:08:02.166301  5816 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.167416  6305 raft_consensus.cc:2749] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.169410  5816 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:02.170284  6305 raft_consensus.cc:2804] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.171139  6305 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.171500  6305 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.172077  6303 raft_consensus.cc:493] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.172713  6303 raft_consensus.cc:515] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.176278  6322 raft_consensus.cc:2804] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.180584  5850 raft_consensus.cc:2468] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:02.111141  6227 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:02.119745  5773 raft_consensus.cc:1240] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
W20260709 13:08:02.193673  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:02.194109  5851 raft_consensus.cc:1240] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:08:02.146998  6220 raft_consensus.cc:2804] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:02.196352  6220 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.197000  6220 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:02.162341  6073 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:02.190002  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.202636  6073 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.198084  5972 tablet_bootstrap.cc:492] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:02.204166  5972 ts_tablet_manager.cc:1403] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.410s	user 0.017s	sys 0.015s
I20260709 13:08:02.205161  6073 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:02.208693  5851 raft_consensus.cc:1240] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
W20260709 13:08:02.109742  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 45063.7us
I20260709 13:08:02.210633  5972 raft_consensus.cc:359] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.118351  5774 raft_consensus.cc:2468] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:08:02.212462  5972 raft_consensus.cc:385] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:02.212809  5972 raft_consensus.cc:740] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:02.214178  6283 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.203294  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.217545  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.216697  5599 catalog_manager.cc:5697] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 1 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:02.219096  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.220044  6218 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.194175  6327 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:02.221972  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.221925  6327 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.224220  5775 raft_consensus.cc:1240] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.224529  6284 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.225449  5971 tablet_bootstrap.cc:654] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:02.226999  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.230130  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.230929  5771 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:02.232384  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.232875  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.235474  5666 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:02.236888  5666 leader_election.cc:304] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:02.239095  6316 raft_consensus.cc:2804] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:02.239600  6316 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.239972  6316 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:02.240190  6303 leader_election.cc:290] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.186970  6277 raft_consensus.cc:515] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.241438  6310 raft_consensus.cc:2804] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:02.241966  5922 raft_consensus.cc:2393] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 1: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:02.242424  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.244797  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.245554  5772 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:02.209748  5813 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:02.248522  5872 debug-util.cc:398] Leaking SignalData structure 0x7b0800158d60 after lost signal to thread 27937
W20260709 13:08:02.249976  5872 debug-util.cc:398] Leaking SignalData structure 0x7b0800149400 after lost signal to thread 5633
I20260709 13:08:02.250401  5813 leader_election.cc:304] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:02.250515  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.251343  6285 raft_consensus.cc:2749] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:02.251542  5872 debug-util.cc:398] Leaking SignalData structure 0x7b080029d8e0 after lost signal to thread 5725
I20260709 13:08:02.215121  5972 consensus_queue.cc:260] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.252581  6285 raft_consensus.cc:2749] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.253135  5972 ts_tablet_manager.cc:1434] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.049s	user 0.010s	sys 0.000s
I20260709 13:08:02.253062  6322 raft_consensus.cc:493] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.253715  6322 raft_consensus.cc:3060] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:02.253791  5872 debug-util.cc:398] Leaking SignalData structure 0x7b08002af940 after lost signal to thread 5800
I20260709 13:08:02.254457  5972 tablet_bootstrap.cc:492] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
W20260709 13:08:02.255402  5872 debug-util.cc:398] Leaking SignalData structure 0x7b0800188cc0 after lost signal to thread 5875
I20260709 13:08:02.242009  6310 raft_consensus.cc:493] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.256599  6310 raft_consensus.cc:3060] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:02.257033  5872 debug-util.cc:398] Leaking SignalData structure 0x7b080036e8a0 after lost signal to thread 5949
W20260709 13:08:02.257637  5872 debug-util.cc:398] Leaking SignalData structure 0x7b0800132420 after lost signal to thread 6017
I20260709 13:08:02.258244  6283 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.260341  6318 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.260888  6318 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.261902  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.263090  6070 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.263478  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.263099  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.267997  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.268747  5693 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:08:02.214612  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.269210  5972 tablet_bootstrap.cc:654] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:02.201797  5816 leader_election.cc:304] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:02.133373  6279 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.266014  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.270915  5701 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:02.119271  6294 raft_consensus.cc:493] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:02.266549  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.272138  6294 raft_consensus.cc:515] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.273522  6284 raft_consensus.cc:2804] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.273975  6284 raft_consensus.cc:493] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.274353  6284 raft_consensus.cc:3060] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.266978  5767 raft_consensus.cc:2393] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 1: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:02.272876  6279 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.277433  5694 raft_consensus.cc:2468] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:02.283200  6289 consensus_queue.cc:1059] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:02.284153  6327 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:02.284667  6289 raft_consensus.cc:3055] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.285203  6289 raft_consensus.cc:740] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:02.286290  6318 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:02.286420  6289 consensus_queue.cc:260] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:02.278061  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 89175.8us
W20260709 13:08:02.293577  5816 leader_election.cc:343] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:02.272501  5970 raft_consensus.cc:359] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.296614  5970 raft_consensus.cc:385] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:02.297091  5970 raft_consensus.cc:740] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:08:02.297801  5970 consensus_queue.cc:260] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.296535  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:02.298143  6294 leader_election.cc:290] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:02.310564  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.311674  5666 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:02.312717  5666 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:02.314030  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:02.314378  6220 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.314710  6314 raft_consensus.cc:2804] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.315207  6311 raft_consensus.cc:2804] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:02.315636  6311 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.315990  6311 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:02.338282  5973 tablet_bootstrap.cc:492] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:08:02.338925  5973 ts_tablet_manager.cc:1403] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.361s	user 0.032s	sys 0.022s
I20260709 13:08:02.340524  6326 consensus_queue.cc:1059] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:02.341681  6283 raft_consensus.cc:3055] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.342284  6283 raft_consensus.cc:740] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:02.342959  6326 consensus_queue.cc:1059] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:02.343351  6283 consensus_queue.cc:260] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.344717  6283 raft_consensus.cc:3060] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.348840  6218 consensus_queue.cc:1059] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:02.350570  6218 raft_consensus.cc:3055] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.351063  6218 raft_consensus.cc:740] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:02.352550  6218 consensus_queue.cc:260] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.354738  6218 raft_consensus.cc:3060] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:02.365008  5666 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.342247  5973 raft_consensus.cc:359] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.366155  5973 raft_consensus.cc:385] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:02.366636  5973 raft_consensus.cc:740] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:02.367854  5973 consensus_queue.cc:260] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.370601  6220 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:02.373181  6284 raft_consensus.cc:515] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.388883  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:02.389364  6284 leader_election.cc:290] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.374061  6305 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.325701  6289 raft_consensus.cc:3060] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.391609  6305 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.395674  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.375649  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.397975  5843 raft_consensus.cc:2393] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 3: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:02.399272  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:02.401346  6073 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.381862  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:02.402133  6073 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.383518  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:08:02.383893  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 54765.7us
I20260709 13:08:02.404181  5742 leader_election.cc:304] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
W20260709 13:08:02.405614  5742 leader_election.cc:343] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:02.406024  5970 ts_tablet_manager.cc:1434] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.239s	user 0.009s	sys 0.001s
I20260709 13:08:02.380853  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.384956  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.385560  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
W20260709 13:08:02.385658  5667 leader_election.cc:343] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:02.389611  5849 raft_consensus.cc:3060] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.315234  6314 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:02.384268  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.406113  5742 leader_election.cc:304] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
W20260709 13:08:02.418936  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 88617.3us
I20260709 13:08:02.379163  6326 raft_consensus.cc:3055] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.418676  5696 raft_consensus.cc:2468] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:02.414001  5920 raft_consensus.cc:2393] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 2: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:02.429306  5742 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.431725  6302 raft_consensus.cc:2804] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:02.432268  6302 raft_consensus.cc:697] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:02.433247  6302 consensus_queue.cc:237] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.437436  6070 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.438513  6070 consensus_queue.cc:1059] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:02.411337  5847 raft_consensus.cc:3060] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.442716  6307 raft_consensus.cc:2749] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:02.443881  5813 leader_election.cc:304] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.431577  5742 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.448731  5742 leader_election.cc:304] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
W20260709 13:08:02.450706  5742 leader_election.cc:343] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:02.427286  6314 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.426600  6277 leader_election.cc:290] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:02.452073  5742 leader_election.cc:304] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:02.399341  5739 leader_election.cc:304] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:02.470815  6218 consensus_queue.cc:1048] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:02.471609  5970 tablet_bootstrap.cc:492] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:02.476473  5815 leader_election.cc:304] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.479954  6284 raft_consensus.cc:2804] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.480542  6284 raft_consensus.cc:493] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:02.481062  6284 raft_consensus.cc:3060] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.481720  5971 tablet_bootstrap.cc:492] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:02.482277  5971 ts_tablet_manager.cc:1403] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.504s	user 0.030s	sys 0.036s
I20260709 13:08:02.419536  6310 raft_consensus.cc:515] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.428887  6326 raft_consensus.cc:740] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:02.484098  5697 raft_consensus.cc:2393] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:02.460726  5742 leader_election.cc:304] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:02.464625  6320 raft_consensus.cc:2749] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.464901  6319 raft_consensus.cc:2804] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.465595  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.467195  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.447207  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.467954  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.469285  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.470109  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.463879  5973 ts_tablet_manager.cc:1434] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.125s	user 0.011s	sys 0.000s
I20260709 13:08:02.455360  6307 raft_consensus.cc:2804] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:02.455761  6305 raft_consensus.cc:2804] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:02.440029  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.492642  6319 raft_consensus.cc:493] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.498291  6321 raft_consensus.cc:2804] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:02.503903  6321 raft_consensus.cc:697] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:02.504328  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:02.446779  6285 consensus_queue.cc:1048] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.047s
I20260709 13:08:02.502437  6320 raft_consensus.cc:2804] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:02.506943  5971 raft_consensus.cc:359] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.490298  6280 raft_consensus.cc:2749] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.508325  5971 raft_consensus.cc:385] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:02.502642  6279 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.488742  6073 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:02.484668  6326 consensus_queue.cc:260] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.490609  5849 raft_consensus.cc:2468] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:02.508708  5971 raft_consensus.cc:740] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
W20260709 13:08:02.485301  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 72669.8us
I20260709 13:08:02.516582  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:02.505311  5922 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:02.495257  5695 raft_consensus.cc:2468] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:02.518047  6316 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.456144  6290 consensus_queue.cc:1059] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:08:02.504832  6322 raft_consensus.cc:515] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.496948  6305 raft_consensus.cc:493] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.505978  6319 raft_consensus.cc:3060] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.524257  5924 raft_consensus.cc:1240] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.533618  5971 consensus_queue.cc:260] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.520354  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:02.532541  6326 raft_consensus.cc:3060] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.526434  6320 raft_consensus.cc:697] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:02.560482  6290 raft_consensus.cc:3055] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.560879  6279 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.561051  6290 raft_consensus.cc:740] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:02.562440  6290 consensus_queue.cc:260] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.564002  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.564443  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.564682  5924 raft_consensus.cc:2468] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:02.564953  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.565887  5851 raft_consensus.cc:2393] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:02.566879  6310 leader_election.cc:290] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:02.569921  6320 consensus_queue.cc:237] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.571031  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:02.572257  5972 tablet_bootstrap.cc:492] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:02.572034  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:02.572777  5972 ts_tablet_manager.cc:1403] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.319s	user 0.016s	sys 0.047s
I20260709 13:08:02.541110  5916 raft_consensus.cc:1240] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.573194  5846 raft_consensus.cc:2393] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:02.541028  6336 raft_consensus.cc:493] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.574784  5739 leader_election.cc:304] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:02.560364  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 81662.1us
I20260709 13:08:02.505384  6321 consensus_queue.cc:237] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.538779  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.524830  6311 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.582482  5970 tablet_bootstrap.cc:654] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:02.548112  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 103575us
I20260709 13:08:02.575385  6336 raft_consensus.cc:515] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.545682  6305 raft_consensus.cc:3060] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:02.620311  5919 raft_consensus.cc:1240] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.582970  5847 raft_consensus.cc:2468] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:02.595270  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.576908  6280 raft_consensus.cc:2749] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.627379  5973 tablet_bootstrap.cc:492] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:02.575838  5972 raft_consensus.cc:359] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.628149  5971 ts_tablet_manager.cc:1434] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.145s	user 0.010s	sys 0.000s
I20260709 13:08:02.631072  5972 raft_consensus.cc:385] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:02.614070  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:08:02.631119  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 64854.5us
W20260709 13:08:02.634523  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 173866us
I20260709 13:08:02.631541  5972 raft_consensus.cc:740] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:02.577834  5739 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.621939  6307 raft_consensus.cc:697] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
W20260709 13:08:02.636363  5742 leader_election.cc:343] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:02.630327  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.638638  5849 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
W20260709 13:08:02.638187  5667 leader_election.cc:343] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:02.632200  5848 raft_consensus.cc:1240] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.632190  6220 raft_consensus.cc:493] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.649523  5742 leader_election.cc:304] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:02.652201  5667 leader_election.cc:304] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.655843  5843 raft_consensus.cc:1240] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.657385  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.657505  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.639906  5972 consensus_queue.cc:260] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.661052  6314 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.672717  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.673461  6290 raft_consensus.cc:3060] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.674125  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.662496  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:02.675504  5920 raft_consensus.cc:3060] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.663018  5890 leader_election.cc:304] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:02.678808  5971 tablet_bootstrap.cc:492] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
W20260709 13:08:02.666846  5742 leader_election.cc:343] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:02.674259  6314 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:02.672097  5848 raft_consensus.cc:2468] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:08:02.681267  5844 raft_consensus.cc:1240] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.682343  5972 ts_tablet_manager.cc:1434] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.109s	user 0.008s	sys 0.000s
I20260709 13:08:02.682801  5844 raft_consensus.cc:1240] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.683535  5972 tablet_bootstrap.cc:492] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:02.660861  5816 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:02.685832  6324 raft_consensus.cc:2749] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.686228  5890 leader_election.cc:304] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:02.666455  6073 raft_consensus.cc:2804] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.687460  6073 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:02.687501  6070 raft_consensus.cc:2749] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.687549  6338 raft_consensus.cc:2749] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.688510  5696 raft_consensus.cc:1240] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.690218  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.690817  6311 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:02.690929  5916 raft_consensus.cc:2468] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:02.666803  6220 raft_consensus.cc:515] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.694075  6220 leader_election.cc:290] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
W20260709 13:08:02.694698  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 116942us
I20260709 13:08:02.686340  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:02.710883  5742 leader_election.cc:304] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.688127  6073 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:02.712951  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 74788.9us
I20260709 13:08:02.681272  5848 raft_consensus.cc:1240] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.718266  6308 raft_consensus.cc:3060] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:02.719197  5742 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.720952  5890 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:02.721426  5843 raft_consensus.cc:1240] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.727936  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.728557  6322 leader_election.cc:290] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:02.729123  5739 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:02.729773  5813 leader_election.cc:304] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:02.732415  5891 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:02.732254  6307 consensus_queue.cc:237] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.733592  6316 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:02.712008  6302 raft_consensus.cc:2804] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.737525  5973 tablet_bootstrap.cc:654] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:02.702950  6314 raft_consensus.cc:2749] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:02.732174  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.738991  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.756992  5742 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.757532  6344 raft_consensus.cc:2749] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.756047  6311 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:02.751219  5842 raft_consensus.cc:1240] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
W20260709 13:08:02.749495  5667 leader_election.cc:343] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:02.803457  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 54010.7us
I20260709 13:08:02.751932  6336 leader_election.cc:290] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.750152  5918 raft_consensus.cc:2393] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 2: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:02.752197  6073 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.749118  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.766973  5844 raft_consensus.cc:1240] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.795626  5972 tablet_bootstrap.cc:654] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:02.800529  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 79755.6us
I20260709 13:08:02.800769  5971 tablet_bootstrap.cc:654] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:02.747570  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.812562  5667 leader_election.cc:304] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:02.779112  6302 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.828109  6302 raft_consensus.cc:3060] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.828989  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:08:02.783156  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 53478.7us
W20260709 13:08:02.803784  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 90504.7us
I20260709 13:08:02.771692  6319 raft_consensus.cc:515] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.783447  6342 raft_consensus.cc:2749] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.793859  6324 raft_consensus.cc:2804] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:02.805836  6311 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.818869  6073 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.819217  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:02.819984  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.820262  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:08:02.821177  5888 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.129:40419 timed out after 1.559s (SENT) [suppressed 194 similar messages]
I20260709 13:08:02.838760  6220 consensus_queue.cc:1059] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:08:02.822664  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:02.823097  5922 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:02.803202  6280 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:02.824949  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:02.825516  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.819530  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:02.826509  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:08:02.805781  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 76758.7us
I20260709 13:08:02.851868  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:02.852353  5739 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:02.854391  5739 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.855754  6344 raft_consensus.cc:2804] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:02.827219  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:02.856316  6344 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.856858  6344 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:02.857734  5776 raft_consensus.cc:2468] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:02.847091  5816 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:02.834497  5970 tablet_bootstrap.cc:492] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:02.861198  6220 raft_consensus.cc:2749] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.861385  5970 ts_tablet_manager.cc:1403] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.402s	user 0.018s	sys 0.001s
I20260709 13:08:02.818889  6284 raft_consensus.cc:515] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.805150  6338 raft_consensus.cc:2804] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:08:02.837981  5667 leader_election.cc:343] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:02.815835  6316 consensus_queue.cc:1059] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:08:02.862812  6336 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.845557  6322 raft_consensus.cc:3055] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.863345  6316 raft_consensus.cc:3055] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.864065  6316 raft_consensus.cc:740] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:02.864099  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:02.865182  5846 raft_consensus.cc:2393] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:02.865623  5891 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:02.865309  6316 consensus_queue.cc:260] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.866701  6280 raft_consensus.cc:2749] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:02.867323  6338 raft_consensus.cc:697] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:02.806025  6308 raft_consensus.cc:2749] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:02.838907  6324 raft_consensus.cc:493] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:02.844955  6218 raft_consensus.cc:2764] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:08:02.868336  6324 raft_consensus.cc:3060] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.868415  6338 consensus_queue.cc:237] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.852608  6311 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:02.863370  6336 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.863606  6284 leader_election.cc:290] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:02.841214  5666 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.818913  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:02.872215  6311 raft_consensus.cc:2749] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.835958  6305 raft_consensus.cc:515] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.860628  6073 raft_consensus.cc:2804] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:02.863750  6322 raft_consensus.cc:740] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:02.873006  6073 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.873417  6073 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:02.873453  5851 raft_consensus.cc:2393] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:02.841169  6314 raft_consensus.cc:2764] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
W20260709 13:08:02.882858  5666 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.890033  5667 leader_election.cc:304] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:02.894340  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:02.894758  6336 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:02.895839  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.896590  5696 raft_consensus.cc:2468] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:02.873661  6322 consensus_queue.cc:260] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.897765  6322 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.884056  6318 consensus_queue.cc:1059] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:08:02.899381  6343 consensus_queue.cc:1059] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:02.900359  6343 raft_consensus.cc:3055] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.900723  6343 raft_consensus.cc:740] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:02.901481  6343 consensus_queue.cc:260] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.902042  6302 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.902791  6343 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.904505  6318 raft_consensus.cc:3055] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.905134  6318 raft_consensus.cc:740] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:02.906095  6318 consensus_queue.cc:260] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.906487  6305 leader_election.cc:290] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.906591  6313 raft_consensus.cc:2749] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:02.907326  6318 raft_consensus.cc:3060] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.909010  6289 consensus_queue.cc:1059] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:02.843947  5920 raft_consensus.cc:2468] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:02.867053  6316 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.915337  5923 raft_consensus.cc:1240] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.912462  6326 raft_consensus.cc:2804] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:02.920485  6326 raft_consensus.cc:697] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:02.921590  6326 consensus_queue.cc:237] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.923396  5667 leader_election.cc:304] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:02.926652  6313 raft_consensus.cc:2804] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:02.927078  6313 raft_consensus.cc:493] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:02.927430  6313 raft_consensus.cc:3060] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:02.930437  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:02.924845  6314 raft_consensus.cc:3055] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.932423  6314 raft_consensus.cc:740] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:02.933527  6314 consensus_queue.cc:260] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:02.934944  6314 raft_consensus.cc:3060] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.925203  6319 leader_election.cc:290] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:02.925910  5664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.925755  5816 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:02.936609  6284 raft_consensus.cc:2804] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:02.937454  6284 raft_consensus.cc:697] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
W20260709 13:08:02.911759  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 46376.5us
I20260709 13:08:02.945115  6073 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.942930  6284 consensus_queue.cc:237] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:02.961187  6073 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.943892  6070 raft_consensus.cc:493] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:02.909214  6302 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:02.966172  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:08:02.968083  5665 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:02.969798  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.980430  5775 raft_consensus.cc:1240] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:02.980515  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:02.981453  5843 raft_consensus.cc:2393] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:02.975405  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:02.975567  5816 leader_election.cc:304] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:02.970765  6070 raft_consensus.cc:515] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:02.976315  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.976975  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:02.978127  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.982020  5769 raft_consensus.cc:1240] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
W20260709 13:08:02.969277  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 114995us
I20260709 13:08:02.995342  5739 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:02.993795  5970 raft_consensus.cc:359] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:02.996109  5970 raft_consensus.cc:385] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:02.996464  5970 raft_consensus.cc:740] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
W20260709 13:08:02.973076  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 152053us
I20260709 13:08:02.999966  5970 consensus_queue.cc:260] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.002698  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:03.003885  5772 raft_consensus.cc:2393] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 1: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:03.004278  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:02.988543  5770 raft_consensus.cc:3060] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:02.980464  5776 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:03.017782  5599 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39364 (request call id 60) took 1019 ms. Trace:
I20260709 13:08:02.991794  6305 raft_consensus.cc:2804] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:02.982576  6313 raft_consensus.cc:515] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.026008  5815 leader_election.cc:304] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.029372  6227 raft_consensus.cc:2749] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.029978  5843 raft_consensus.cc:1240] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:03.030543  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.031777  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:03.022341  6302 raft_consensus.cc:2804] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:03.033277  6302 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.028337  6305 raft_consensus.cc:697] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:03.033687  6302 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:03.034600  6305 consensus_queue.cc:237] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.037035  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.037308  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:03.037820  5697 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:03.038683  5970 ts_tablet_manager.cc:1434] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.177s	user 0.008s	sys 0.002s
I20260709 13:08:03.039074  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.039848  5970 tablet_bootstrap.cc:492] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:03.041731  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.037070  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.007429  6324 raft_consensus.cc:515] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.043790  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
W20260709 13:08:03.084862  5742 leader_election.cc:343] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.086520  6073 raft_consensus.cc:493] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.087147  6073 raft_consensus.cc:515] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.019690  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:03.023233  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:03.045527  6324 leader_election.cc:290] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:03.091679  5842 raft_consensus.cc:1240] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:03.093696  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.096398  6070 leader_election.cc:290] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:03.045586  6344 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:03.086553  5742 leader_election.cc:343] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.100773  5742 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.101190  5771 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:03.104130  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:03.105034  5851 raft_consensus.cc:2393] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:03.105469  6358 raft_consensus.cc:2749] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:03.105789  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:03.106132  5774 raft_consensus.cc:2468] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:03.107127  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.107613  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:03.108420  5772 raft_consensus.cc:2393] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 1: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:03.109153  5920 raft_consensus.cc:3060] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:03.109462  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.113049  5972 tablet_bootstrap.cc:492] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:03.113600  5972 ts_tablet_manager.cc:1403] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.430s	user 0.029s	sys 0.015s
I20260709 13:08:03.113849  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.115851  6289 consensus_queue.cc:1059] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:03.116772  5666 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:03.116914  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.117571  6318 consensus_queue.cc:1059] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.117928  6289 raft_consensus.cc:3055] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:03.118405  6289 raft_consensus.cc:740] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:03.118629  6318 raft_consensus.cc:3055] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:03.119151  6318 raft_consensus.cc:740] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:03.091857  6336 raft_consensus.cc:493] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.119158  5696 raft_consensus.cc:2468] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:03.120048  6336 raft_consensus.cc:515] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:03.120477  5666 leader_election.cc:343] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.122112  6290 raft_consensus.cc:2804] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:03.122514  6336 leader_election.cc:290] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:03.122682  6290 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.123160  6290 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:03.018404  5599 rpcz_store.cc:276] 0709 13:08:01.998739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:01.998924 (+   185us) service_pool.cc:224] Handling call
0709 13:08:03.017715 (+1018791us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:02.498826 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:02.502089 (+  3263us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:02.502172 (+    83us) write_op.cc:435] Acquired schema lock
0709 13:08:02.502207 (+    35us) tablet.cc:662] Decoding operations
0709 13:08:02.502693 (+   486us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:02.502816 (+   123us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:02.502847 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:02.503494 (+   647us) tablet.cc:701] Row locks acquired
0709 13:08:02.503626 (+   132us) write_op.cc:260] PREPARE: finished
0709 13:08:02.503824 (+   198us) write_op.cc:270] Start()
0709 13:08:02.504012 (+   188us) write_op.cc:276] Timestamp: P: 1783602482503755 usec, L: 0
0709 13:08:02.504089 (+    77us) op_driver.cc:348] REPLICATION: starting
0709 13:08:02.504869 (+   780us) log.cc:844] Serialized 540 byte log entry
0709 13:08:02.776835 (+271966us) op_driver.cc:464] REPLICATION: finished
0709 13:08:02.993763 (+216928us) write_op.cc:301] APPLY: starting
0709 13:08:02.993849 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:02.993985 (+   136us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:02.994114 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:02.994412 (+   298us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:02.994683 (+   271us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:08:02.994722 (+    39us) write_op.cc:312] APPLY: finished
0709 13:08:02.995043 (+   321us) log.cc:844] Serialized 22 byte log entry
0709 13:08:02.995216 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:02.995290 (+    74us) write_op.cc:454] Released schema lock
0709 13:08:02.995346 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:02.995731 (+   385us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:02.995758 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":25435,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":216507,"apply.run_cpu_time_us":1970,"apply.run_wall_time_us":1971,"num_ops":1,"prepare.queue_time_us":213734,"prepare.run_cpu_time_us":35194,"prepare.run_wall_time_us":140966,"raft.queue_time_us":121611,"raft.run_cpu_time_us":8393,"raft.run_wall_time_us":206021,"replication_time_us":272560,"thread_start_us":502058,"threads_started":4,"wal-append.queue_time_us":120853}]]}
I20260709 13:08:03.104063  5597 catalog_manager.cc:5697] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from 792907f253c744aa9392e8d572f88e06 (127.27.71.132) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.126271  5597 catalog_manager.cc:5697] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.117802  5694 raft_consensus.cc:2393] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:03.128199  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:03.119372  6289 consensus_queue.cc:260] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.129016  5851 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:03.129889  6289 raft_consensus.cc:3060] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.136999  5843 raft_consensus.cc:1275] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:03.102272  6344 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:03.102456  5813 leader_election.cc:304] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.141613  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:03.091869  5739 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.142540  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
W20260709 13:08:03.142969  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 96914.9us
I20260709 13:08:03.143347  6324 raft_consensus.cc:2804] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:03.143435  5920 raft_consensus.cc:2468] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:03.143945  5739 leader_election.cc:304] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.144411  5970 tablet_bootstrap.cc:654] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:03.144901  5816 leader_election.cc:304] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:03.145421  6358 raft_consensus.cc:2749] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.146348  6322 raft_consensus.cc:2804] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:03.145187  6320 raft_consensus.cc:2804] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:03.147143  5739 leader_election.cc:304] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.147035  6290 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.148247  6319 raft_consensus.cc:2804] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:03.148723  5739 leader_election.cc:304] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.148757  6319 raft_consensus.cc:697] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:03.149173  5597 catalog_manager.cc:5697] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from 792907f253c744aa9392e8d572f88e06 (127.27.71.132) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.150012  6319 consensus_queue.cc:237] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:03.116358  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.152166  5972 raft_consensus.cc:359] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.154871  6318 consensus_queue.cc:260] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:03.156186  6318 raft_consensus.cc:3060] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.156694  6313 leader_election.cc:290] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:03.146991  6322 raft_consensus.cc:697] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:03.159349  6070 consensus_queue.cc:1059] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.143958  6324 raft_consensus.cc:493] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:03.162115  6324 raft_consensus.cc:3060] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.162533  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.160423  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:03.158758  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:03.165365  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.165975  6073 leader_election.cc:290] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:03.166043  5696 raft_consensus.cc:3060] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.147732  6320 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:03.167806  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.168285  6320 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.169162  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.169414  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.170059  5694 raft_consensus.cc:2393] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:03.171057  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.171940  5694 raft_consensus.cc:2393] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:03.160274  6322 consensus_queue.cc:237] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.150954  5597 catalog_manager.cc:5697] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.152909  5972 raft_consensus.cc:385] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.173336  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.174510  5693 raft_consensus.cc:2468] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:03.174716  6326 raft_consensus.cc:3055] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:03.164695  5888 leader_election.cc:304] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.161501  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:03.177878  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
W20260709 13:08:03.179128  5667 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:03.183279  5742 leader_election.cc:343] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:03.192631  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 166743us
W20260709 13:08:03.213125  5667 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.191476  5972 raft_consensus.cc:740] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:03.178869  5700 raft_consensus.cc:2468] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:03.207581  5695 raft_consensus.cc:2393] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:03.178376  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:03.178542  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.211359  5692 raft_consensus.cc:2393] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:03.186053  5694 raft_consensus.cc:1275] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:03.189461  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:03.217542  5815 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.222422  6073 consensus_queue.cc:1059] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:03.225251  5699 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:03.193259  5770 raft_consensus.cc:2468] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:08:03.190013  5851 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:03.199115  6326 raft_consensus.cc:740] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:03.233418  5815 leader_election.cc:304] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.235864  5973 tablet_bootstrap.cc:492] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:08:03.237002  5973 ts_tablet_manager.cc:1403] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.615s	user 0.041s	sys 0.004s
I20260709 13:08:03.238093  5667 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.233927  5972 consensus_queue.cc:260] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.220207  6343 raft_consensus.cc:493] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.194927  6302 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:03.242610  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:08:03.241194  5667 leader_election.cc:343] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.243777  5815 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.256155  5666 leader_election.cc:304] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:03.256780  5667 leader_election.cc:304] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:03.221623  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:03.239101  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:03.232309  6305 raft_consensus.cc:2749] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.239573  6342 raft_consensus.cc:3055] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:03.282181  5973 raft_consensus.cc:359] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.288178  5973 raft_consensus.cc:385] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.288555  5973 raft_consensus.cc:740] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:03.285883  5742 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
W20260709 13:08:03.286803  5666 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.232728  6321 raft_consensus.cc:2749] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:03.290053  5775 raft_consensus.cc:2393] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 1: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:03.291795  6280 raft_consensus.cc:2804] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:03.292069  6324 raft_consensus.cc:515] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.292610  5696 raft_consensus.cc:2468] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:03.294179  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:03.254904  6227 raft_consensus.cc:2749] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.239642  6313 consensus_queue.cc:1059] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:08:03.284898  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:03.311019  5845 raft_consensus.cc:2393] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 1: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:03.311493  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:03.297606  5768 raft_consensus.cc:1240] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:03.232497  6358 consensus_queue.cc:1048] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.065s
I20260709 13:08:03.298019  5970 tablet_bootstrap.cc:492] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:03.323203  5970 ts_tablet_manager.cc:1403] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.284s	user 0.018s	sys 0.011s
W20260709 13:08:03.298350  5742 leader_election.cc:343] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.258843  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:03.332432  6369 raft_consensus.cc:3055] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:03.333033  6369 raft_consensus.cc:740] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:03.242177  6289 raft_consensus.cc:2749] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.337236  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
W20260709 13:08:03.321372  5891 leader_election.cc:343] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.315816  6320 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:03.343442  5774 raft_consensus.cc:1240] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:03.344424  6320 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:03.306141  5850 raft_consensus.cc:1275] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:03.319629  5813 leader_election.cc:304] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:03.283448  6311 raft_consensus.cc:2804] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:08:03.306818  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 50162.8us
I20260709 13:08:03.346421  6311 raft_consensus.cc:697] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:03.296036  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:03.347373  6220 raft_consensus.cc:2804] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:03.347790  5667 leader_election.cc:304] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.348017  6220 raft_consensus.cc:697] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:03.347479  6311 consensus_queue.cc:237] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:03.302352  6290 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:03.348984  6290 consensus_queue.cc:1059] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.349836  5667 leader_election.cc:304] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.349889  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
W20260709 13:08:03.285885  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 121311us
I20260709 13:08:03.310178  5972 ts_tablet_manager.cc:1434] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.196s	user 0.009s	sys 0.000s
I20260709 13:08:03.310400  5697 raft_consensus.cc:2468] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:03.288095  6342 raft_consensus.cc:740] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:03.310925  5971 tablet_bootstrap.cc:492] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:03.243144  6336 raft_consensus.cc:2749] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.350064  6290 consensus_queue.cc:1059] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:08:03.317261  5843 raft_consensus.cc:1240] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:03.357074  5972 tablet_bootstrap.cc:492] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:03.356822  6342 consensus_queue.cc:260] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.316525  5701 raft_consensus.cc:1240] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:03.358286  6314 raft_consensus.cc:2764] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:08:03.359236  6336 consensus_queue.cc:1059] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:03.328486  6324 leader_election.cc:290] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:03.307721  6280 raft_consensus.cc:697] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:03.305848  6326 consensus_queue.cc:260] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.319902  5770 raft_consensus.cc:2393] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:03.349237  6316 raft_consensus.cc:2749] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.329965  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:03.346402  6302 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:03.316243  6285 raft_consensus.cc:2749] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.362812  6316 raft_consensus.cc:2749] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.364169  6280 consensus_queue.cc:237] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.364975  5667 leader_election.cc:304] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:03.351338  6313 raft_consensus.cc:3055] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:03.365921  6313 raft_consensus.cc:740] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:03.342294  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:03.366750  6313 consensus_queue.cc:260] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.360189  6338 raft_consensus.cc:3055] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:03.368399  6338 raft_consensus.cc:740] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:03.356436  5971 ts_tablet_manager.cc:1403] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.678s	user 0.038s	sys 0.034s
I20260709 13:08:03.301005  5973 consensus_queue.cc:260] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.348944  6220 consensus_queue.cc:237] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.257543  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
W20260709 13:08:03.307410  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 64521.3us
I20260709 13:08:03.370366  5667 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.376777  5815 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.378729  5667 leader_election.cc:304] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.378933  6227 raft_consensus.cc:2804] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:03.382053  6227 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:03.382527  6227 raft_consensus.cc:3060] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.373214  6314 raft_consensus.cc:2749] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.384759  6338 consensus_queue.cc:260] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.342202  6369 consensus_queue.cc:260] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.386013  6338 raft_consensus.cc:3060] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.386709  6369 raft_consensus.cc:3060] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.375365  5700 raft_consensus.cc:3060] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.368058  6326 raft_consensus.cc:3060] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.389834  6314 raft_consensus.cc:2749] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.369093  6313 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.258249  6343 raft_consensus.cc:515] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.380270  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:03.381119  6289 raft_consensus.cc:2749] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.374747  6321 consensus_queue.cc:1048] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:08:03.360250  6342 raft_consensus.cc:3060] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.405102  5845 raft_consensus.cc:1275] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:03.378746  5815 leader_election.cc:304] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.369376  6290 raft_consensus.cc:3055] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:03.383814  5972 tablet_bootstrap.cc:654] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:03.372372  5742 leader_election.cc:343] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.373306  6336 raft_consensus.cc:493] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:03.409538  5742 leader_election.cc:343] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.402127  6343 leader_election.cc:290] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:03.409796  6324 raft_consensus.cc:2749] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.419564  5920 raft_consensus.cc:1275] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:03.412807  6218 consensus_queue.cc:1048] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.430119  5700 raft_consensus.cc:2468] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:03.414444  5888 leader_election.cc:304] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.436685  5813 leader_election.cc:304] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.424590  6336 raft_consensus.cc:515] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.438735  5970 raft_consensus.cc:359] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:03.439507  5970 raft_consensus.cc:385] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.441349  5739 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.442706  5739 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.443378  5698 raft_consensus.cc:2468] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:03.419816  5815 leader_election.cc:304] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.420357  5742 leader_election.cc:304] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.446902  6073 raft_consensus.cc:2749] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.421768  5973 ts_tablet_manager.cc:1434] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.184s	user 0.008s	sys 0.001s
I20260709 13:08:03.447844  6073 raft_consensus.cc:2749] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:03.447844  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f9347b11ec44d1b07c475379a51881" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:03.448750  6073 raft_consensus.cc:2749] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:03.448717  5770 raft_consensus.cc:2468] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:03.451169  5666 leader_election.cc:304] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:03.451998  6313 raft_consensus.cc:2804] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:03.452378  6313 raft_consensus.cc:493] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.424576  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.452719  5971 raft_consensus.cc:359] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.452859  6313 raft_consensus.cc:3060] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:03.453485  5971 raft_consensus.cc:385] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.427398  6324 consensus_queue.cc:1048] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.452283  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.455426  5696 raft_consensus.cc:2393] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:03.456743  6218 raft_consensus.cc:2749] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.457341  5888 leader_election.cc:304] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.458144  6218 raft_consensus.cc:2749] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.460664  5850 raft_consensus.cc:1275] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:03.453860  5971 raft_consensus.cc:740] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:03.422792  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
W20260709 13:08:03.460340  5664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.462081  5971 consensus_queue.cc:260] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.420176  6290 raft_consensus.cc:740] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:03.407729  5844 raft_consensus.cc:2468] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:03.448855  5973 tablet_bootstrap.cc:492] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:03.439790  5970 raft_consensus.cc:740] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:08:03.458653  5888 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.471202  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.472419  5970 consensus_queue.cc:260] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:03.465858  6290 consensus_queue.cc:260] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.472627  6336 leader_election.cc:290] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:03.475488  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.471796  5888 leader_election.cc:304] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.477558  5697 raft_consensus.cc:2468] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:08:03.468397  6324 raft_consensus.cc:2804] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:03.454666  6227 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.481690  5971 ts_tablet_manager.cc:1434] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.112s	user 0.007s	sys 0.000s
W20260709 13:08:03.479463  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 60793.6us
I20260709 13:08:03.473242  6218 raft_consensus.cc:2749] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.482463  5815 leader_election.cc:304] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.474412  6313 raft_consensus.cc:515] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.516808  5970 ts_tablet_manager.cc:1434] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.193s	user 0.008s	sys 0.006s
I20260709 13:08:03.525548  5739 leader_election.cc:304] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:03.526805  5971 tablet_bootstrap.cc:492] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:03.515544  6321 consensus_queue.cc:1048] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:08:03.528982  6344 raft_consensus.cc:2804] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:03.529738  6344 raft_consensus.cc:697] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:03.530854  6344 consensus_queue.cc:237] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.534116  5772 raft_consensus.cc:2468] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:03.536098  6227 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:03.536404  5970 tablet_bootstrap.cc:492] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:03.537052  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:03.538522  5850 raft_consensus.cc:2468] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:08:03.478433  6280 consensus_queue.cc:1048] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:08:03.537930  5739 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.542207  6280 raft_consensus.cc:2749] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:03.543083  6318 raft_consensus.cc:493] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.543639  6318 raft_consensus.cc:515] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.545800  6318 leader_election.cc:290] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:03.545563  5599 catalog_manager.cc:5697] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:03.551482  5666 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.488245  6336 raft_consensus.cc:2749] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.576613  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:03.581483  6218 consensus_queue.cc:1059] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:03.585430  6218 raft_consensus.cc:3055] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:03.586056  6218 raft_consensus.cc:740] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:03.557258  6290 raft_consensus.cc:3060] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.587387  6218 consensus_queue.cc:260] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:03.589223  6218 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:03.617224  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 134835us
I20260709 13:08:03.618938  5667 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:03.597128  5971 tablet_bootstrap.cc:654] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:03.579058  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 107356us
I20260709 13:08:03.598644  5972 tablet_bootstrap.cc:492] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:03.628756  5667 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:03.603797  5890 leader_election.cc:304] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:03.570088  6326 consensus_queue.cc:1059] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:08:03.630815  5972 ts_tablet_manager.cc:1403] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.274s	user 0.010s	sys 0.006s
I20260709 13:08:03.599970  6285 raft_consensus.cc:2749] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.612404  6324 raft_consensus.cc:697] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:03.638655  6342 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:03.638895  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:03.639250  6342 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.641500  6342 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:03.650250  5667 leader_election.cc:343] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.650880  5667 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:03.653946  5739 leader_election.cc:304] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.654347  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:03.629441  6322 raft_consensus.cc:2804] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:03.660503  6070 raft_consensus.cc:2749] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:03.660712  6326 raft_consensus.cc:3055] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:03.637620  6311 raft_consensus.cc:2804] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:03.654632  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:03.654706  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f9347b11ec44d1b07c475379a51881" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:03.633961  6318 raft_consensus.cc:2749] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:03.654350  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:03.652338  6343 raft_consensus.cc:2749] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:03.673467  5739 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.674803  5920 raft_consensus.cc:1275] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:03.662143  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:03.675020  5696 raft_consensus.cc:1275] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:03.659108  6342 raft_consensus.cc:2804] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:03.658648  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f9347b11ec44d1b07c475379a51881" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:03.691066  5774 raft_consensus.cc:3060] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:03.693169  5771 raft_consensus.cc:1275] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:03.693639  5924 raft_consensus.cc:1275] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:08:03.694090  5924 pending_rounds.cc:85] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06: Aborting all ops after (but not including) 0
I20260709 13:08:03.694314  5924 pending_rounds.cc:107] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:08:03.694617  5924 raft_consensus.cc:2889] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:03.698194  6313 leader_election.cc:290] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:03.699133  6324 consensus_queue.cc:237] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.699985  6284 consensus_queue.cc:1048] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.703586  5891 leader_election.cc:304] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:03.704592  6377 raft_consensus.cc:2804] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:03.682148  5769 raft_consensus.cc:2468] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:08:03.676294  6378 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:03.685616  5667 leader_election.cc:343] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.676692  6322 raft_consensus.cc:697] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:03.661551  6326 raft_consensus.cc:740] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:03.705019  5970 tablet_bootstrap.cc:654] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:03.709996  5667 leader_election.cc:343] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.721693  6324 consensus_queue.cc:1048] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:03.705902  6342 raft_consensus.cc:697] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:03.729045  6342 consensus_queue.cc:237] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:03.705197  6377 raft_consensus.cc:493] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:03.706211  6378 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.732369  6377 raft_consensus.cc:3060] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:03.680572  6311 raft_consensus.cc:697] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 2 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:03.734277  5599 catalog_manager.cc:5697] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 1 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.688347  6320 raft_consensus.cc:2749] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.754689  5599 catalog_manager.cc:5697] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 3, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 3 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.708443  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:03.761320  5851 raft_consensus.cc:2393] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:03.760321  6322 consensus_queue.cc:237] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.761528  5971 tablet_bootstrap.cc:492] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:03.707754  5816 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:03.762056  5971 ts_tablet_manager.cc:1403] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.236s	user 0.019s	sys 0.007s
I20260709 13:08:03.725835  6302 consensus_queue.cc:1048] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
W20260709 13:08:03.720187  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 55874us
I20260709 13:08:03.765444  6326 consensus_queue.cc:260] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:03.766866  6326 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.767967  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:03.708108  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f9347b11ec44d1b07c475379a51881" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:03.771612  5599 catalog_manager.cc:5697] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 2 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.725572  6070 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.788491  6382 raft_consensus.cc:2749] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.792577  5971 raft_consensus.cc:359] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.689073  5888 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.793462  5971 raft_consensus.cc:385] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.793788  5971 raft_consensus.cc:740] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
W20260709 13:08:03.794111  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 113707us
I20260709 13:08:03.794691  6320 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.790571  5916 raft_consensus.cc:3060] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:03.796097  6336 raft_consensus.cc:2749] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:03.797927  5923 raft_consensus.cc:1275] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:03.792759  5972 raft_consensus.cc:359] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.799153  5972 raft_consensus.cc:385] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:03.799453  5972 raft_consensus.cc:740] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:03.752785  6378 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:03.795435  6320 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:03.800915  5972 consensus_queue.cc:260] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.806998  5888 leader_election.cc:304] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.813107  5888 leader_election.cc:304] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.802770  6378 raft_consensus.cc:493] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:03.803277  6070 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.825354  5696 raft_consensus.cc:1275] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
W20260709 13:08:03.805814  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 98142.8us
I20260709 13:08:03.851061  5972 ts_tablet_manager.cc:1434] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.216s	user 0.007s	sys 0.001s
I20260709 13:08:03.859020  5971 consensus_queue.cc:260] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:03.839844  6336 raft_consensus.cc:2749] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:03.850209  5696 pending_rounds.cc:85] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Aborting all ops after (but not including) 0
I20260709 13:08:03.841671  6338 raft_consensus.cc:2749] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:03.847180  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:03.876124  5698 raft_consensus.cc:2468] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:03.870807  6070 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:03.877228  6320 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:03.871536  5774 raft_consensus.cc:2468] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:08:03.873068  5848 raft_consensus.cc:1240] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:03.856235  6311 consensus_queue.cc:237] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.882793  5666 leader_election.cc:304] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:03.868435  5696 pending_rounds.cc:107] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:08:03.849359  6378 raft_consensus.cc:515] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:03.899300  5667 leader_election.cc:343] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.905625  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:03.895617  5696 raft_consensus.cc:2889] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:03.907173  5739 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:03.911654  5843 raft_consensus.cc:1275] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:03.887279  6313 raft_consensus.cc:2804] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:03.931869  5971 ts_tablet_manager.cc:1434] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.169s	user 0.011s	sys 0.003s
W20260709 13:08:03.963750  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 49545.6us
I20260709 13:08:03.957573  5598 catalog_manager.cc:5697] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.957070  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:08:03.961555  5664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:03.969017  5598 catalog_manager.cc:5697] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.974303  5972 tablet_bootstrap.cc:492] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
W20260709 13:08:03.974323  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 161334us
I20260709 13:08:03.975561  5888 leader_election.cc:304] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.959440  6313 raft_consensus.cc:697] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
W20260709 13:08:03.978190  5742 leader_election.cc:343] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:03.966324  6321 raft_consensus.cc:2804] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:03.981395  5888 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:03.985644  5888 leader_election.cc:304] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:03.967864  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 60794.3us
I20260709 13:08:03.997311  5888 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:03.998777  5971 tablet_bootstrap.cc:492] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:03.967810  6280 consensus_queue.cc:1048] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:08:03.966101  6377 raft_consensus.cc:515] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.978148  5845 raft_consensus.cc:1275] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:03.982924  6321 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:03.977613  6342 consensus_queue.cc:1048] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:08:04.024086  5973 tablet_bootstrap.cc:654] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.024407  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:04.025720  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:04.026515  5772 raft_consensus.cc:3060] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:03.986496  6395 raft_consensus.cc:2804] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:08:04.030618  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 43859.9us
I20260709 13:08:04.000576  6218 raft_consensus.cc:2749] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:04.042601  5970 tablet_bootstrap.cc:492] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:04.043429  5970 ts_tablet_manager.cc:1403] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.507s	user 0.032s	sys 0.041s
W20260709 13:08:04.045018  6390 log.cc:927] Time spent T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.187s	user 0.000s	sys 0.002s
I20260709 13:08:04.018757  6313 consensus_queue.cc:237] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:03.989135  6290 consensus_queue.cc:1048] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 13:08:04.023716  6342 raft_consensus.cc:493] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:04.048259  6342 raft_consensus.cc:515] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:03.984917  6324 consensus_queue.cc:1048] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.043s
W20260709 13:08:04.048861  5891 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.015012  6377 leader_election.cc:290] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:04.050001  5769 raft_consensus.cc:2468] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:03.984721  6336 raft_consensus.cc:2749] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:04.050760  6395 raft_consensus.cc:697] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:04.004613  6070 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.051551  6070 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:04.051756  6395 consensus_queue.cc:237] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:04.023348  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:04.057507  6070 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:04.058717  6342 leader_election.cc:290] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:04.037403  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:04.059553  5697 raft_consensus.cc:2468] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:08:04.028551  6321 raft_consensus.cc:3060] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:03.997541  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:04.065433  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:04.066289  5774 raft_consensus.cc:2468] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 0.
I20260709 13:08:04.066429  5701 raft_consensus.cc:1275] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.041507  6338 raft_consensus.cc:2749] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:04.042423  6285 consensus_queue.cc:1048] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:04.032282  6320 consensus_queue.cc:1048] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:08:04.074913  5700 raft_consensus.cc:1275] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.075703  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:04.077255  5923 raft_consensus.cc:2468] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:04.078186  5850 raft_consensus.cc:1275] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.078857  6378 leader_election.cc:290] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:04.079934  5971 tablet_bootstrap.cc:654] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.080579  6394 consensus_queue.cc:1048] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:04.085866  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:04.086851  5843 raft_consensus.cc:3060] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.032029  6073 consensus_queue.cc:1048] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:04.006438  5888 leader_election.cc:304] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:04.085881  6378 consensus_queue.cc:1059] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:04.088374  5890 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:04.090204  5970 raft_consensus.cc:359] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:04.090909  5970 raft_consensus.cc:385] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.091167  5970 raft_consensus.cc:740] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.087594  6322 consensus_queue.cc:1048] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:08:04.093608  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:04.097014  6320 consensus_queue.cc:1048] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:04.097349  6396 raft_consensus.cc:2804] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:04.097822  6396 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.098147  6396 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.104017  6402 consensus_queue.cc:1048] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260709 13:08:04.106930  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 100592us
I20260709 13:08:04.107007  5742 leader_election.cc:304] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:04.108369  5888 leader_election.cc:304] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:04.110387  5667 leader_election.cc:343] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.110989  5667 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
W20260709 13:08:04.112937  5667 leader_election.cc:343] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.113344  5667 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:04.095265  5970 consensus_queue.cc:260] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:04.119074  5667 leader_election.cc:343] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.122150  5970 ts_tablet_manager.cc:1434] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.078s	user 0.005s	sys 0.003s
I20260709 13:08:04.117290  6318 raft_consensus.cc:2749] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:04.115993  6336 raft_consensus.cc:2749] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:04.124841  5843 raft_consensus.cc:2468] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:04.115947  6070 raft_consensus.cc:2804] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:04.121290  6302 raft_consensus.cc:2804] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:04.096350  5891 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:04.121253  6289 raft_consensus.cc:2749] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:04.143323  5696 raft_consensus.cc:1275] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.130940  6070 raft_consensus.cc:493] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.167074  6070 raft_consensus.cc:3060] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.138924  6302 raft_consensus.cc:493] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
W20260709 13:08:04.172422  5666 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.185672  6302 raft_consensus.cc:3060] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:04.188678  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 100346us
I20260709 13:08:04.243171  5891 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
W20260709 13:08:04.244572  5891 leader_election.cc:343] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.245409  5973 tablet_bootstrap.cc:492] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
W20260709 13:08:04.187156  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 65563.8us
I20260709 13:08:04.246131  5973 ts_tablet_manager.cc:1403] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.798s	user 0.016s	sys 0.017s
I20260709 13:08:04.245425  5971 tablet_bootstrap.cc:492] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:04.245518  5891 leader_election.cc:304] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:04.248029  5971 ts_tablet_manager.cc:1403] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.262s	user 0.015s	sys 0.002s
I20260709 13:08:04.250018  6395 consensus_queue.cc:1048] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:04.251534  6395 raft_consensus.cc:2749] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:04.252561  5972 tablet_bootstrap.cc:654] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:04.252990  6407 log.cc:927] Time spent T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
W20260709 13:08:04.254595  6398 log.cc:927] Time spent T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.202s	user 0.002s	sys 0.000s
I20260709 13:08:04.255458  6396 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:04.256922  5970 tablet_bootstrap.cc:492] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:04.257725  6396 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:04.271773  5971 raft_consensus.cc:359] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:04.278851  5971 raft_consensus.cc:385] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.279407  5971 raft_consensus.cc:740] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.280638  5971 consensus_queue.cc:260] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:04.266970  6326 raft_consensus.cc:2749] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:04.291842  5970 tablet_bootstrap.cc:654] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.274385  6413 consensus_queue.cc:1048] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:08:04.265327  5772 raft_consensus.cc:2468] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:04.301105  5971 ts_tablet_manager.cc:1434] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.000s
I20260709 13:08:04.302825  5971 tablet_bootstrap.cc:492] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:04.277691  6338 raft_consensus.cc:2804] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:04.317363  5739 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:04.315681  5923 raft_consensus.cc:1275] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.333603  5924 raft_consensus.cc:3060] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:04.333979  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:04.336388  6319 raft_consensus.cc:2804] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:04.336823  6319 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.337165  6319 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.337298  5973 raft_consensus.cc:359] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:04.338160  5973 raft_consensus.cc:385] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.338423  5973 raft_consensus.cc:740] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
W20260709 13:08:04.339568  6404 log.cc:927] Time spent T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:08:04.339495  6358 consensus_queue.cc:1048] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:04.340938  5700 raft_consensus.cc:1275] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.341789  5921 raft_consensus.cc:1240] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:04.344049  6227 consensus_queue.cc:1048] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:04.336354  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:04.347126  6321 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:04.324142  6338 raft_consensus.cc:697] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:04.350229  5973 consensus_queue.cc:260] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:04.356212  6321 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:04.361670  5667 leader_election.cc:343] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.362187  5667 leader_election.cc:304] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:04.363651  6289 raft_consensus.cc:2749] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:04.368718  5667 leader_election.cc:304] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:04.376507  6318 raft_consensus.cc:2749] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:04.379947  6344 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:04.380733  6344 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:04.317162  6285 consensus_queue.cc:1048] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
I20260709 13:08:04.361370  6343 raft_consensus.cc:493] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.397482  5973 ts_tablet_manager.cc:1434] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.151s	user 0.007s	sys 0.001s
I20260709 13:08:04.369505  6284 consensus_queue.cc:1048] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:08:04.401239  5972 tablet_bootstrap.cc:492] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:04.401734  5972 ts_tablet_manager.cc:1403] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.436s	user 0.014s	sys 0.016s
I20260709 13:08:04.396181  5924 raft_consensus.cc:1275] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.371137  6336 consensus_queue.cc:1048] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:08:04.384717  6344 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:04.410640  6377 raft_consensus.cc:3055] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:04.410849  6227 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.435456  5973 tablet_bootstrap.cc:492] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:04.395622  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:04.393452  6396 consensus_queue.cc:1048] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:08:04.402962  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:04.424643  6338 consensus_queue.cc:237] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:04.437299  5918 raft_consensus.cc:1240] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:04.437427  6227 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:04.438207  5692 raft_consensus.cc:1275] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.399312  6343 raft_consensus.cc:515] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:04.452279  5842 raft_consensus.cc:2393] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:04.453506  5772 raft_consensus.cc:1275] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.454243  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:04.435562  6377 raft_consensus.cc:740] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:04.454928  5850 raft_consensus.cc:2468] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:04.455356  5970 tablet_bootstrap.cc:492] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:04.414685  6302 raft_consensus.cc:515] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:04.461179  5970 ts_tablet_manager.cc:1403] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.204s	user 0.013s	sys 0.018s
I20260709 13:08:04.459576  6320 consensus_queue.cc:1048] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:08:04.464776  5972 raft_consensus.cc:359] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:04.465483  5843 raft_consensus.cc:2393] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:04.467487  5971 tablet_bootstrap.cc:654] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.454991  6423 raft_consensus.cc:493] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.469241  5972 raft_consensus.cc:385] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.463706  5700 raft_consensus.cc:1275] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.471621  5972 raft_consensus.cc:740] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:04.477608  5972 consensus_queue.cc:260] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:04.496050  5664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.481194  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:04.479506  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:04.489331  6227 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:04.496083  6284 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.532829  5972 ts_tablet_manager.cc:1434] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.131s	user 0.007s	sys 0.007s
I20260709 13:08:04.536672  5890 leader_election.cc:304] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:04.540521  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:04.541254  5769 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.497242  6324 consensus_queue.cc:1048] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:08:04.477794  6070 raft_consensus.cc:515] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:04.581043  5970 raft_consensus.cc:359] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:04.584609  5970 raft_consensus.cc:385] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:04.584972  5970 raft_consensus.cc:740] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
W20260709 13:08:04.591943  5667 consensus_peers.cc:597] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb -> Peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Couldn't send request to peer 792907f253c744aa9392e8d572f88e06. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:04.592649  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 46652.5us
I20260709 13:08:04.570863  6326 raft_consensus.cc:2804] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:04.481544  6377 consensus_queue.cc:260] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:04.479732  6423 raft_consensus.cc:515] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:04.608168  5970 consensus_queue.cc:260] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:04.535195  6302 leader_election.cc:290] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:04.609154  6070 leader_election.cc:290] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:04.620579  6425 consensus_queue.cc:1048] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:08:04.623310  5970 ts_tablet_manager.cc:1434] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.156s	user 0.006s	sys 0.001s
W20260709 13:08:04.624984  6426 log.cc:927] Time spent T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:08:04.625976  6424 log.cc:927] Time spent T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:08:04.630729  6319 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:04.580536  6284 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:04.640316  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:04.644371  6322 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.644600  5597 catalog_manager.cc:5697] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 1 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:04.648064  6427 log.cc:927] Time spent T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:08:04.650027  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:04.650269  5972 tablet_bootstrap.cc:492] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:04.652794  5973 tablet_bootstrap.cc:654] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.607379  6326 raft_consensus.cc:493] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:04.607785  6320 raft_consensus.cc:493] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.667757  5597 catalog_manager.cc:5697] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 1 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:04.673753  5597 catalog_manager.cc:5697] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:04.678570  5970 tablet_bootstrap.cc:492] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:04.682771  6218 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:04.706975  5888 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.666563  5699 raft_consensus.cc:3060] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.683239  5692 raft_consensus.cc:2468] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:04.717769  6319 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:04.724229  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:04.725692  5739 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:04.679639  6326 raft_consensus.cc:3060] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:04.731312  5742 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:04.725208  5844 raft_consensus.cc:2393] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:04.735972  6073 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:04.738664  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 105256us
I20260709 13:08:04.704272  6218 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:04.669860  6322 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:04.725198  6423 leader_election.cc:290] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:04.688287  6320 raft_consensus.cc:515] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:04.732666  6319 raft_consensus.cc:2804] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:04.741211  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:04.752185  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:04.772508  5768 raft_consensus.cc:2393] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:04.780375  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:04.781401  6218 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:04.783133  5925 raft_consensus.cc:2393] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:04.786644  6284 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:04.788833  6320 leader_election.cc:290] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:04.789259  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:04.789705  5851 raft_consensus.cc:1275] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.781581  5845 raft_consensus.cc:2393] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:04.796124  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:04.796933  5701 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.798448  6343 leader_election.cc:290] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:04.807678  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:04.811161  5699 raft_consensus.cc:2468] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:04.812489  5844 raft_consensus.cc:1240] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:04.813701  5842 raft_consensus.cc:1275] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:04.814664  5769 raft_consensus.cc:2468] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:04.776089  6290 consensus_queue.cc:1048] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:08:04.817083  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:04.817778  5848 raft_consensus.cc:3060] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:04.824958  6073 raft_consensus.cc:2749] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:04.825850  5970 tablet_bootstrap.cc:654] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:04.800933  6322 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:04.857373  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4f21d87306453b81a417014e41e4a4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:04.860268  6313 consensus_queue.cc:1048] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:04.859283  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4f21d87306453b81a417014e41e4a4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:04.857568  5888 leader_election.cc:304] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
W20260709 13:08:04.862650  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 133993us
W20260709 13:08:04.871608  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 50352.8us
W20260709 13:08:04.873087  5742 leader_election.cc:343] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:04.880723  5667 leader_election.cc:343] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:04.882948  5667 consensus_peers.cc:597] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:04.877460  6395 raft_consensus.cc:2804] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:04.891017  6395 raft_consensus.cc:697] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:04.891098  6377 raft_consensus.cc:3060] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:04.901991  6073 raft_consensus.cc:493] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.902354  5971 tablet_bootstrap.cc:492] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:04.902549  6073 raft_consensus.cc:515] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:04.902861  5971 ts_tablet_manager.cc:1403] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.600s	user 0.029s	sys 0.054s
W20260709 13:08:04.902698  5667 leader_election.cc:343] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:04.903507  5667 leader_election.cc:304] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:04.905259  5598 catalog_manager.cc:5697] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 1 to 2, leader changed from 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 2 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:04.920025  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:08:04.929389  5890 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:04.913363  6321 raft_consensus.cc:493] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:04.913178  6319 raft_consensus.cc:697] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:04.956112  6073 leader_election.cc:290] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:04.972568  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:04.956926  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:04.979050  6438 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:04.979739  6438 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:04.947479  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:04.957261  6358 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:04.973676  5695 raft_consensus.cc:2393] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
W20260709 13:08:04.990800  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 222629us
I20260709 13:08:04.981228  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:04.968825  6321 raft_consensus.cc:515] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:05.008471  6443 raft_consensus.cc:2749] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:04.946054  6395 consensus_queue.cc:237] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:05.020884  5890 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
W20260709 13:08:05.054888  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 61689.6us
I20260709 13:08:05.027196  6321 leader_election.cc:290] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:05.030419  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:04.993213  6358 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.032541  5973 tablet_bootstrap.cc:492] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:08:05.060796  5973 ts_tablet_manager.cc:1403] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.655s	user 0.015s	sys 0.010s
I20260709 13:08:05.037264  6438 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:05.037707  6396 raft_consensus.cc:493] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.038084  6452 raft_consensus.cc:493] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.040151  6444 raft_consensus.cc:493] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.042186  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:05.045130  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:08:05.046258  5742 leader_election.cc:343] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.047330  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:05.048668  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:05.009083  6434 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.056062  5972 tablet_bootstrap.cc:654] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.060433  6378 raft_consensus.cc:2804] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:05.080057  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:05.081724  6434 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:05.082629  5776 raft_consensus.cc:2393] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:05.096403  5695 raft_consensus.cc:2393] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:05.114692  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:05.075830  5920 raft_consensus.cc:2468] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:05.120347  5850 raft_consensus.cc:2468] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:05.122359  6413 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.126044  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:05.132524  6382 consensus_queue.cc:1048] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:05.069934  6396 raft_consensus.cc:515] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:05.117648  5848 raft_consensus.cc:2468] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:05.068668  6452 raft_consensus.cc:515] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:05.154526  5816 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:05.147974  5973 raft_consensus.cc:359] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.155601  5973 raft_consensus.cc:385] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.115362  5697 raft_consensus.cc:1275] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:05.156388  5973 raft_consensus.cc:740] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.068390  6444 raft_consensus.cc:515] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:05.159792  5667 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:05.075641  6319 consensus_queue.cc:237] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.331945  5973 consensus_queue.cc:260] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:05.259330  6326 consensus_meta.cc:287] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.002s
I20260709 13:08:05.163395  5925 raft_consensus.cc:2468] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:05.144425  6413 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:05.355355  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 200942us
W20260709 13:08:05.257678  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 100532us
I20260709 13:08:05.325747  6318 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.373485  5971 raft_consensus.cc:359] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:05.374270  5971 raft_consensus.cc:385] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.374581  5971 raft_consensus.cc:740] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.376973  5973 ts_tablet_manager.cc:1434] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.316s	user 0.010s	sys 0.004s
I20260709 13:08:05.342613  6289 raft_consensus.cc:2804] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:05.342263  6443 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.378342  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:05.380051  5971 consensus_queue.cc:260] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:05.379941  6289 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.389386  6289 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.390148  5971 ts_tablet_manager.cc:1434] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.487s	user 0.010s	sys 0.008s
I20260709 13:08:05.394012  6444 leader_election.cc:290] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:05.399293  6377 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.405916  6396 leader_election.cc:290] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:05.407402  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:05.380398  6318 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:05.408764  5842 raft_consensus.cc:2393] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:05.410840  6318 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:05.417030  5701 consensus_meta.cc:287] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Time spent flushing consensus metadata: real 0.620s	user 0.004s	sys 0.009s
I20260709 13:08:05.417574  5701 raft_consensus.cc:2468] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:05.418468  6443 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.421505  6443 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:05.357637  6326 raft_consensus.cc:515] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:05.426384  6434 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:05.385818  6218 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.381708  6431 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.427846  6326 leader_election.cc:290] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:05.445392  6326 consensus_queue.cc:1048] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:05.429145  6413 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:05.429593  6358 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:05.429920  5970 tablet_bootstrap.cc:492] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:05.448913  5970 ts_tablet_manager.cc:1403] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.777s	user 0.024s	sys 0.007s
I20260709 13:08:05.432418  6218 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:05.433168  6431 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.454775  6218 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:05.441319  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:05.459251  5850 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:05.466305  5667 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
W20260709 13:08:05.467775  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 95840.2us
I20260709 13:08:05.440898  5918 raft_consensus.cc:2368] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3.
I20260709 13:08:05.475248  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:05.475742  6369 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.476281  5701 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:05.476464  6369 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.477911  5970 raft_consensus.cc:359] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:05.478837  5970 raft_consensus.cc:385] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.479131  5970 raft_consensus.cc:740] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.440482  6452 leader_election.cc:290] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:05.467094  6377 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:05.466773  5739 leader_election.cc:343] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.481104  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:05.485898  5850 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:05.486505  5739 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:05.489704  5667 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:05.479882  5970 consensus_queue.cc:260] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:05.479619  6369 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:05.498404  5739 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:05.503027  5972 tablet_bootstrap.cc:492] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:05.503875  5972 ts_tablet_manager.cc:1403] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.898s	user 0.043s	sys 0.026s
I20260709 13:08:05.483584  6444 raft_consensus.cc:2804] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:05.482283  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:05.494505  6318 raft_consensus.cc:2804] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:05.498762  6377 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:05.509763  6434 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:05.538686  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 405030us
I20260709 13:08:05.533000  6444 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.550261  6431 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:05.559307  6441 log.cc:927] Time spent T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.590s	user 0.002s	sys 0.000s
I20260709 13:08:05.536566  6290 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:05.563581  6290 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:05.566857  5970 ts_tablet_manager.cc:1434] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.118s	user 0.008s	sys 0.003s
I20260709 13:08:05.534482  6318 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.548373  6320 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:05.578526  5972 raft_consensus.cc:359] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:05.538615  5848 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:05.584702  5970 tablet_bootstrap.cc:492] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:05.557813  6435 raft_consensus.cc:2749] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:05.582554  5972 raft_consensus.cc:385] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:05.579015  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:05.558177  6457 raft_consensus.cc:2804] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:05.568408  6444 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.556545  6432 consensus_queue.cc:1048] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.116s
I20260709 13:08:05.592643  5972 raft_consensus.cc:740] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:05.583703  6318 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:05.583087  6320 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:05.571957  6434 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.593340  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:05.593387  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:05.627238  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:05.619793  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:05.599251  5851 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.633648  5742 leader_election.cc:304] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:05.602771  5972 consensus_queue.cc:260] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:05.638609  6378 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.008s	sys 0.016s
I20260709 13:08:05.624624  6472 raft_consensus.cc:493] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.643709  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:05.641788  6290 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:05.635294  6302 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.628554  6457 raft_consensus.cc:697] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:05.644589  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:05.645361  5972 ts_tablet_manager.cc:1434] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.141s	user 0.008s	sys 0.004s
I20260709 13:08:05.645956  6289 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:05.646454  6458 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.667912  5919 raft_consensus.cc:2468] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:05.668897  6458 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.670102  6289 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:05.670066  5772 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:08:05.671377  6458 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:05.633355  6320 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:05.648773  6473 raft_consensus.cc:2804] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:08:05.649525  5742 leader_election.cc:343] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.686021  5742 leader_election.cc:304] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:05.639885  5890 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:05.672353  6290 raft_consensus.cc:493] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.671438  6302 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:05.673130  6434 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:05.674264  6457 consensus_queue.cc:237] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:05.691640  6344 raft_consensus.cc:2749] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:05.642232  6396 consensus_queue.cc:1048] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
W20260709 13:08:05.655666  6378 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.008s	sys 0.016s
I20260709 13:08:05.696939  6378 raft_consensus.cc:697] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:05.697714  6473 raft_consensus.cc:493] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:05.697840  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:05.698287  6473 raft_consensus.cc:3060] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:05.698031  6378 consensus_queue.cc:237] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:05.688580  5742 leader_election.cc:343] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:05.700340  5742 leader_election.cc:304] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:05.701406  6302 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:05.702047  6302 raft_consensus.cc:2749] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:08:05.689697  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 50163.5us
I20260709 13:08:05.704571  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
W20260709 13:08:05.706311  6459 log.cc:927] Time spent T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.368s	user 0.000s	sys 0.001s
I20260709 13:08:05.662861  6472 raft_consensus.cc:515] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:05.696321  6474 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:05.708608  6472 leader_election.cc:290] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:05.710489  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:05.711020  5970 tablet_bootstrap.cc:654] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.719938  5770 raft_consensus.cc:1275] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:05.725283  5971 tablet_bootstrap.cc:492] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:05.728188  5699 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:05.732177  5694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44372 (request call id 83) took 1193 ms. Trace:
I20260709 13:08:05.691444  6290 raft_consensus.cc:515] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.732679  5694 rpcz_store.cc:276] 0709 13:08:04.539088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.541152 (+  2064us) service_pool.cc:224] Handling call
0709 13:08:05.138738 (+597586us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b58009a9e98. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:08:05.153307 (+ 14569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:05.285414 (+132107us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:05.328580 (+ 43166us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:05.457126 (+128546us) log.cc:844] Serialized 26 byte log entry
0709 13:08:05.659595 (+202469us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:08:05.660128 (+   533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:05.684091 (+ 23963us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:05.731781 (+ 47690us) raft_consensus.cc:1685] finished
0709 13:08:05.731809 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:05.732083 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3174,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":14267,"spinlock_wait_cycles":5653376,"thread_start_us":181292,"threads_started":1,"wal-append.queue_time_us":233283}
I20260709 13:08:05.735061  6290 leader_election.cc:290] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:05.736361  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:05.738906  6473 raft_consensus.cc:515] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:05.740454  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:05.741253  6473 leader_election.cc:290] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:05.741794  6465 log.cc:927] Time spent T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 13:08:05.744207  5770 raft_consensus.cc:2393] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 2: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:05.745141  6474 raft_consensus.cc:2749] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:05.748522  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:05.761696  5971 tablet_bootstrap.cc:654] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:05.768056  5739 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
W20260709 13:08:05.790231  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 86765.6us
I20260709 13:08:05.788369  6432 raft_consensus.cc:493] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.790410  6458 raft_consensus.cc:2804] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:05.771904  6290 raft_consensus.cc:493] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:05.813479  6438 consensus_queue.cc:1048] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:05.801693  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:05.817111  6444 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:05.793874  6394 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.799634  6070 consensus_queue.cc:1048] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 13:08:05.821811  6458 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.821409  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:05.879545  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:05.821738  6432 raft_consensus.cc:515] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:05.891623  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:05.892248  6432 leader_election.cc:290] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:05.892380  5701 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
W20260709 13:08:05.884121  6455 log.cc:927] Time spent T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:08:05.892511  5916 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:05.821681  6320 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.887789  6458 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:05.899812  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:05.900552  5768 raft_consensus.cc:2468] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:05.901252  5918 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59976 (request call id 91) took 1116 ms. Trace:
I20260709 13:08:05.894547  5770 raft_consensus.cc:2468] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:05.901719  5918 rpcz_store.cc:276] 0709 13:08:04.784421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.806119 (+ 21698us) service_pool.cc:224] Handling call
0709 13:08:05.901023 (+1094904us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54006036d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:05.901153 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453827968}
I20260709 13:08:05.903092  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:05.880849  6394 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:05.905686  6394 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:05.907109  5773 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:05.909673  5739 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:05.911054  6321 raft_consensus.cc:2804] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:05.911433  6321 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.913712  5851 raft_consensus.cc:2468] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:05.914394  5851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57474 (request call id 98) took 1103 ms. Trace:
I20260709 13:08:05.833766  6290 raft_consensus.cc:515] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.911736  6321 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.921191  5815 leader_election.cc:304] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:05.944623  5971 tablet_bootstrap.cc:492] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:05.942564  5742 leader_election.cc:400] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Vote denied by peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241) with higher term. Message: Invalid argument: T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3.
I20260709 13:08:05.945815  5971 ts_tablet_manager.cc:1403] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.221s	user 0.021s	sys 0.022s
I20260709 13:08:05.945820  5742 leader_election.cc:403] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:08:05.953359  6444 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:05.955852  6320 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:05.958046  6290 leader_election.cc:290] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:05.930995  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:05.958880  5918 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:05.932160  5775 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:05.958209  5970 tablet_bootstrap.cc:492] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:05.961890  5970 ts_tablet_manager.cc:1403] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.377s	user 0.018s	sys 0.025s
I20260709 13:08:05.934397  6434 raft_consensus.cc:2804] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:05.929638  6070 raft_consensus.cc:493] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:05.962366  6280 raft_consensus.cc:3060] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:05.965838  6318 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:05.914733  5851 rpcz_store.cc:276] 0709 13:08:04.810931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.823652 (+ 12721us) service_pool.cc:224] Handling call
0709 13:08:05.578916 (+755264us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b58001e6398. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8f8c 00007f78339e8ef6 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:08:05.914312 (+335396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33254784}
I20260709 13:08:05.968211  6434 raft_consensus.cc:493] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:05.968458  6285 raft_consensus.cc:493] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:05.984628  6218 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:06.012677  6218 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.014984  6218 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:06.016619  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:06.002388  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:06.020316  5813 leader_election.cc:304] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:06.021353  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:06.023839  5925 raft_consensus.cc:2468] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:06.025065  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:06.025830  5918 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:05.997395  6434 raft_consensus.cc:3060] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.030861  6435 raft_consensus.cc:493] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.032128  6432 raft_consensus.cc:2749] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:06.034693  5972 tablet_bootstrap.cc:492] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:06.008987  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:05.994989  6443 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:06.035995  6285 raft_consensus.cc:515] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:06.036633  6443 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.038911  6443 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:06.039225  6285 leader_election.cc:290] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:06.041326  6318 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:06.042881  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.043673  5695 raft_consensus.cc:2468] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:06.043807  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.042968  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.045413  5694 raft_consensus.cc:2393] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:06.046662  5696 raft_consensus.cc:2393] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:06.047643  6322 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.048099  5972 tablet_bootstrap.cc:654] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.044559  5699 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:06.048484  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:06.049350  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.050318  5848 raft_consensus.cc:2393] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:06.050215  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:06.051966  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.056401  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:06.057781  5971 raft_consensus.cc:359] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.058624  5971 raft_consensus.cc:385] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.058917  5971 raft_consensus.cc:740] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.060282  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:06.061419  5890 leader_election.cc:304] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:06.062317  6338 raft_consensus.cc:2804] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:06.062865  6338 raft_consensus.cc:493] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.063206  6338 raft_consensus.cc:3060] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.063295  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.009548  6458 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.064028  5695 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:06.064875  5813 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:06.065892  6432 raft_consensus.cc:2749] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:06.066349  5973 tablet_bootstrap.cc:492] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:06.031471  6435 raft_consensus.cc:515] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.068950  6435 leader_election.cc:290] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:06.069362  6458 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:06.072152  5971 consensus_queue.cc:260] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.073964  5890 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:06.048236  6322 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.075443  6338 raft_consensus.cc:515] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.083348  5666 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:05.985019  6070 raft_consensus.cc:515] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.089795  5742 leader_election.cc:304] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:06.094671  5971 ts_tablet_manager.cc:1434] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.148s	user 0.006s	sys 0.000s
I20260709 13:08:06.096601  5742 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:06.097973  6458 raft_consensus.cc:2804] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:06.098886  5971 tablet_bootstrap.cc:492] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:06.100204  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:06.100864  5773 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.102010  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.102697  5845 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:08:06.080961  6218 raft_consensus.cc:2804] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:08:06.113839  5742 leader_election.cc:343] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.111948  6218 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.130299  5970 raft_consensus.cc:359] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.136976  5970 raft_consensus.cc:385] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.137300  5970 raft_consensus.cc:740] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.108757  6458 raft_consensus.cc:697] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:06.139998  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:06.116365  6344 raft_consensus.cc:2804] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:06.153105  5970 consensus_queue.cc:260] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.155856  5850 raft_consensus.cc:1275] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:06.140630  6218 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.149745  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:06.130101  5813 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:06.176596  5970 ts_tablet_manager.cc:1434] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.214s	user 0.008s	sys 0.006s
I20260709 13:08:06.158342  6344 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.165766  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:06.184008  6344 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.229243  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:06.229451  6322 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:06.203058  5773 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
W20260709 13:08:06.233112  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 90599.7us
I20260709 13:08:06.184535  5924 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.257236  6434 raft_consensus.cc:515] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.245451  5970 tablet_bootstrap.cc:492] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:06.267431  5666 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:06.248947  6280 raft_consensus.cc:2749] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241) with higher term. Message: Invalid argument: T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3.
I20260709 13:08:06.263675  5844 raft_consensus.cc:2393] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:06.258478  6322 raft_consensus.cc:493] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.254287  5692 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:06.283939  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 128743us
I20260709 13:08:06.252588  6474 consensus_queue.cc:1048] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 13:08:06.304368  5971 tablet_bootstrap.cc:654] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.302407  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:06.284235  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:06.302784  6322 raft_consensus.cc:515] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.287616  6413 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:06.309993  6413 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.311240  6322 leader_election.cc:290] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:06.299830  6280 raft_consensus.cc:493] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.312335  6280 raft_consensus.cc:515] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.287217  6378 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.313553  6378 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.314254  5973 tablet_bootstrap.cc:654] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:06.316907  6491 log.cc:927] Time spent T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:08:06.242776  6458 consensus_queue.cc:237] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:06.317673  6494 log.cc:927] Time spent T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:08:06.309311  6503 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.328363  6338 leader_election.cc:290] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:06.333442  6070 leader_election.cc:290] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:06.289440  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 174069us
W20260709 13:08:06.371376  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 46694.3us
I20260709 13:08:06.291551  6318 raft_consensus.cc:2804] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:08:06.467406  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 167017us
W20260709 13:08:06.377417  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 127387us
I20260709 13:08:06.375252  5769 raft_consensus.cc:2468] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:06.442466  6070 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:06.501370  6319 raft_consensus.cc:493] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:06.498505  6498 log.cc:927] Time spent T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:08:06.377652  6321 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.479799  5696 raft_consensus.cc:1275] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:08:06.557134  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 88622.4us
I20260709 13:08:06.557956  5816 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:06.522964  6413 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:06.520510  6318 raft_consensus.cc:697] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 2 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:06.543686  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:06.571424  5972 tablet_bootstrap.cc:492] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:06.571978  5972 ts_tablet_manager.cc:1403] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.537s	user 0.023s	sys 0.009s
I20260709 13:08:06.545863  6070 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:06.577080  6497 log.cc:927] Time spent T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:08:06.577250  6070 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:06.547978  6319 raft_consensus.cc:515] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.578596  5693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44372 (request call id 85) took 2038 ms. Trace:
I20260709 13:08:06.579530  5970 tablet_bootstrap.cc:654] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.563347  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:06.584443  6227 raft_consensus.cc:2804] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:06.585120  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:06.585538  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.586261  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:06.587411  6444 raft_consensus.cc:493] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
W20260709 13:08:06.561637  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 62858.4us
I20260709 13:08:06.561645  6423 raft_consensus.cc:493] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:06.568909  6290 raft_consensus.cc:493] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:06.588090  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:06.589880  6218 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.592032  5771 raft_consensus.cc:2393] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 3: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:06.592383  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:06.574568  6434 leader_election.cc:290] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:06.592692  6456 raft_consensus.cc:493] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.632169  6318 consensus_queue.cc:237] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:06.632418  6456 raft_consensus.cc:515] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.635504  6456 leader_election.cc:290] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:06.593765  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.579033  5693 rpcz_store.cc:276] 0709 13:08:04.539852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.541620 (+  1768us) service_pool.cc:224] Handling call
0709 13:08:05.379580 (+837960us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b58009a9e98. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:08:05.913179 (+533599us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b58009a9e98. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8e9c 00007f78339e8a5b 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:08:05.913376 (+   197us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:05.913801 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:05.913838 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:05.914466 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 13:08:06.104251 (+189785us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:08:06.108979 (+  4728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:06.112389 (+  3410us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:06.578370 (+465981us) raft_consensus.cc:1685] finished
0709 13:08:06.578399 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:06.578531 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1362,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":6084,"spinlock_wait_cycles":15715840,"thread_start_us":174403,"threads_started":1,"wal-append.queue_time_us":254912}
I20260709 13:08:06.594206  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.574688  5918 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.594475  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:06.618348  6444 raft_consensus.cc:515] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.644297  5842 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.645964  6444 leader_election.cc:290] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:06.646492  6444 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:06.646941  6444 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.650158  5971 tablet_bootstrap.cc:492] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:06.650856  5971 ts_tablet_manager.cc:1403] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.552s	user 0.022s	sys 0.025s
I20260709 13:08:06.579860  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:06.652379  5773 raft_consensus.cc:2468] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:06.654940  5972 raft_consensus.cc:359] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.656037  5972 raft_consensus.cc:385] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.656291  5972 raft_consensus.cc:740] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.605861  5597 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39344 (request call id 71) took 2690 ms. Trace:
I20260709 13:08:06.657248  6321 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:06.656790  6378 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:06.579547  6319 leader_election.cc:290] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
W20260709 13:08:06.595584  5700 consensus_meta.cc:287] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: Time spent flushing consensus metadata: real 0.535s	user 0.013s	sys 0.000s
I20260709 13:08:06.622385  5813 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:06.621017  6423 raft_consensus.cc:515] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:06.621214  6290 raft_consensus.cc:515] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.601697  5692 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:06.613369  5767 raft_consensus.cc:2468] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:06.609104  5774 raft_consensus.cc:2468] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:06.636214  6506 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:06.593142  6474 raft_consensus.cc:493] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:06.601073  6358 consensus_queue.cc:1048] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:08:06.608336  6302 consensus_queue.cc:1048] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
W20260709 13:08:06.638325  6499 log.cc:927] Time spent T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:08:06.622648  6425 raft_consensus.cc:493] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:06.615403  6344 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.656769  5924 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:06.669301  6434 raft_consensus.cc:2804] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:06.663393  5813 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
W20260709 13:08:06.709020  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 45724.9us
I20260709 13:08:06.711004  6444 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:06.664731  5972 consensus_queue.cc:260] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.712889  5972 ts_tablet_manager.cc:1434] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.141s	user 0.007s	sys 0.000s
I20260709 13:08:06.600715  6227 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.714859  6227 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.665829  5700 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:06.715996  6509 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:06.716461  6509 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.716642  6443 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:06.716838  6509 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.717656  6344 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:06.717793  6423 leader_election.cc:290] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:06.657063  5597 rpcz_store.cc:276] 0709 13:08:03.910721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:03.910909 (+   188us) service_pool.cc:224] Handling call
0709 13:08:06.601379 (+2690470us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:04.914944 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:05.038280 (+123336us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:05.038316 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:08:05.038335 (+    19us) tablet.cc:662] Decoding operations
0709 13:08:05.038898 (+   563us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:05.038962 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:05.038988 (+    26us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:08:05.039332 (+   344us) tablet.cc:701] Row locks acquired
0709 13:08:05.039352 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:08:05.039488 (+   136us) write_op.cc:270] Start()
0709 13:08:05.039600 (+   112us) write_op.cc:276] Timestamp: P: 1783602485039445 usec, L: 0
0709 13:08:05.039619 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:08:05.040380 (+   761us) log.cc:844] Serialized 1316 byte log entry
0709 13:08:05.692948 (+652568us) op_driver.cc:464] REPLICATION: finished
0709 13:08:05.907201 (+214253us) write_op.cc:301] APPLY: starting
0709 13:08:05.907309 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:05.907447 (+   138us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:05.907583 (+   136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:05.908106 (+   523us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:05.915062 (+  6956us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:08:05.923909 (+  8847us) write_op.cc:312] APPLY: finished
0709 13:08:05.947553 (+ 23644us) log.cc:844] Serialized 34 byte log entry
0709 13:08:05.966786 (+ 19233us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:05.969794 (+  3008us) write_op.cc:454] Released schema lock
0709 13:08:05.986331 (+ 16537us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:06.003814 (+ 17483us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:06.003854 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":585167,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":202945,"apply.run_cpu_time_us":3082,"apply.run_wall_time_us":101619,"num_ops":3,"prepare.queue_time_us":171130,"prepare.run_cpu_time_us":31421,"prepare.run_wall_time_us":362687,"raft.queue_time_us":449865,"raft.run_cpu_time_us":55341,"raft.run_wall_time_us":285123,"replication_time_us":636526,"thread_start_us":643448,"threads_started":4,"wal-append.queue_time_us":407040}]]}
I20260709 13:08:06.717676  5971 raft_consensus.cc:359] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.721290  5971 raft_consensus.cc:385] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.721639  5971 raft_consensus.cc:740] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:06.724308  5971 consensus_queue.cc:260] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.712973  6434 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.713474  6425 raft_consensus.cc:515] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.688968  6474 raft_consensus.cc:515] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:06.731770  5701 raft_consensus.cc:1240] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:06.727191  6434 raft_consensus.cc:3060] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.744899  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.749408  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.750311  5693 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:06.750229  5918 raft_consensus.cc:2468] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:06.755434  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.756505  5692 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
W20260709 13:08:06.755532  5667 leader_election.cc:343] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.759642  5667 leader_election.cc:304] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:06.688769  6506 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.764261  5971 ts_tablet_manager.cc:1434] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.113s	user 0.006s	sys 0.002s
I20260709 13:08:06.767204  5667 leader_election.cc:304] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:06.769781  6444 raft_consensus.cc:2749] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:06.729359  6511 raft_consensus.cc:493] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:06.774967  5971 tablet_bootstrap.cc:492] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:06.775818  5850 raft_consensus.cc:2468] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:06.765929  6394 raft_consensus.cc:493] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:06.785918  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.793634  5973 tablet_bootstrap.cc:492] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:08:06.794262  5973 ts_tablet_manager.cc:1403] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.728s	user 0.017s	sys 0.016s
I20260709 13:08:06.794521  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:06.795253  6218 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:06.795807  6395 consensus_queue.cc:1048] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:06.796651  5888 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:06.784695  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:06.797861  5843 raft_consensus.cc:3060] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.798382  6425 leader_election.cc:290] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:06.799189  6474 leader_election.cc:290] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:06.777674  6519 raft_consensus.cc:2749] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:06.800729  6506 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:06.795290  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:06.802065  6520 consensus_queue.cc:1059] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:06.802532  6290 leader_election.cc:290] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:06.791528  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.804239  6326 raft_consensus.cc:2804] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:06.804762  6326 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.805190  6326 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.797724  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:06.787904  6394 raft_consensus.cc:515] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:06.813027  5842 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:06.814436  5849 raft_consensus.cc:1240] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:06.814745  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.813647  5973 raft_consensus.cc:359] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.818305  5667 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:06.818567  5973 raft_consensus.cc:385] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:06.818991  5973 raft_consensus.cc:740] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
W20260709 13:08:06.819391  5667 leader_election.cc:343] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.782079  6511 raft_consensus.cc:515] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.821018  6516 raft_consensus.cc:2804] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:06.815392  6394 leader_election.cc:290] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:06.823282  5842 raft_consensus.cc:2468] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:08:06.828013  5919 raft_consensus.cc:3060] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.840462  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.840960  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:06.822540  5667 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:06.841449  5697 raft_consensus.cc:2468] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:06.841652  5693 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.842675  6519 raft_consensus.cc:2804] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:06.843300  6519 raft_consensus.cc:697] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:06.828821  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.844897  5845 raft_consensus.cc:1240] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:06.829842  6280 leader_election.cc:290] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:06.845454  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:06.845948  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.846293  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.846589  5849 raft_consensus.cc:2468] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:06.832563  6474 consensus_queue.cc:1059] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
W20260709 13:08:06.832374  6510 log.cc:927] Time spent T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.234s	user 0.000s	sys 0.004s
I20260709 13:08:06.822741  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:06.824934  6511 leader_election.cc:290] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:06.828794  6516 raft_consensus.cc:697] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:06.825416  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:06.852550  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:06.853209  5848 raft_consensus.cc:2393] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:06.852962  6516 consensus_queue.cc:237] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.853762  5693 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:06.854269  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.855669  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.855760  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.830415  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:06.857197  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:06.857858  5848 raft_consensus.cc:3060] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.859719  5971 tablet_bootstrap.cc:654] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:06.860438  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.844293  6519 consensus_queue.cc:237] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.857933  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.807593  6434 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:06.874665  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 52205.9us
I20260709 13:08:06.877650  5973 consensus_queue.cc:260] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.817350  6503 raft_consensus.cc:2749] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:06.883908  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.834939  6443 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.884514  5739 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:06.888024  5739 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:06.888962  6280 raft_consensus.cc:2804] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:06.889381  6394 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:06.889508  6280 raft_consensus.cc:697] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
W20260709 13:08:06.889858  5739 leader_election.cc:343] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.885778  5970 tablet_bootstrap.cc:492] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:06.890532  6280 consensus_queue.cc:237] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.891507  5970 ts_tablet_manager.cc:1403] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.665s	user 0.015s	sys 0.018s
I20260709 13:08:06.889997  6394 raft_consensus.cc:697] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:06.847647  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:06.891865  6443 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:06.898131  5846 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.847676  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.900137  5813 leader_election.cc:304] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:06.903010  5973 ts_tablet_manager.cc:1434] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.108s	user 0.007s	sys 0.002s
I20260709 13:08:06.903820  5739 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:06.811861  5843 raft_consensus.cc:2468] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:06.834873  6326 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:06.910251  5973 tablet_bootstrap.cc:492] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:06.908277  5667 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:06.894158  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:06.937028  5667 leader_election.cc:304] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:06.899298  6227 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.907239  6369 raft_consensus.cc:493] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.941200  5667 leader_election.cc:304] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:06.908942  6524 raft_consensus.cc:2804] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:06.903265  6520 consensus_queue.cc:1059] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 13:08:06.915485  6321 raft_consensus.cc:2804] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:06.957748  5598 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39338 (request call id 62) took 2308 ms. Trace:
W20260709 13:08:06.954489  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:06.907060  5845 raft_consensus.cc:2376] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1. Re-sending same reply.
I20260709 13:08:06.950101  6524 raft_consensus.cc:493] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.961663  6524 raft_consensus.cc:3060] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.940265  6517 raft_consensus.cc:2749] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:06.952500  5599 catalog_manager.cc:5697] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 2, leader changed from 792907f253c744aa9392e8d572f88e06 (127.27.71.132) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 2 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:06.971563  5815 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:06.944746  6444 raft_consensus.cc:2804] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:06.975291  6444 raft_consensus.cc:493] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:06.975741  6444 raft_consensus.cc:3060] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:06.972817  5666 leader_election.cc:304] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:06.966579  6394 consensus_queue.cc:237] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:06.935561  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:06.974601  5815 leader_election.cc:304] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:06.972005  5599 catalog_manager.cc:5697] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 2, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 2 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:06.989506  5742 leader_election.cc:343] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:06.989982  5742 leader_election.cc:304] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:06.970219  6474 raft_consensus.cc:3055] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:06.991693  6073 raft_consensus.cc:2749] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:06.992089  5742 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:06.990185  6284 raft_consensus.cc:2749] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:06.996248  6073 raft_consensus.cc:2764] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:08:06.974793  6321 raft_consensus.cc:697] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:06.950488  6509 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:06.952960  6369 raft_consensus.cc:515] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:06.986371  6434 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:06.997453  5599 catalog_manager.cc:5697] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 1 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:06.985677  6285 raft_consensus.cc:2804] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:06.987058  6318 raft_consensus.cc:2804] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:06.988734  6320 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.041481  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:06.991803  6474 raft_consensus.cc:740] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:06.990065  5891 leader_election.cc:304] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:06.958294  5598 rpcz_store.cc:276] 0709 13:08:04.649225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.657193 (+  7968us) service_pool.cc:224] Handling call
0709 13:08:06.957663 (+2300470us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:05.376792 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:05.400916 (+ 24124us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:05.401029 (+   113us) write_op.cc:435] Acquired schema lock
0709 13:08:05.401066 (+    37us) tablet.cc:662] Decoding operations
0709 13:08:05.401555 (+   489us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:05.401625 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:05.401678 (+    53us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:05.401968 (+   290us) tablet.cc:701] Row locks acquired
0709 13:08:05.401996 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:08:05.402206 (+   210us) write_op.cc:270] Start()
0709 13:08:05.402398 (+   192us) write_op.cc:276] Timestamp: P: 1783602485402133 usec, L: 0
0709 13:08:05.402434 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:08:05.403338 (+   904us) log.cc:844] Serialized 540 byte log entry
0709 13:08:05.877352 (+474014us) op_driver.cc:464] REPLICATION: finished
0709 13:08:05.964429 (+ 87077us) write_op.cc:301] APPLY: starting
0709 13:08:06.059698 (+ 95269us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b60000785c8. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783a477161 00007f783a5e314b 00007f783a5e5774 00007f783a55a6db 00007f783a457b93 00007f783a459de8 00007f783a558e34 00007f783a54497c 00007f783a548d61 00007f783a548d19 00007f783a548ca9
0709 13:08:06.059785 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:06.059880 (+    95us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:06.059898 (+    18us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:06.103726 (+ 43828us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:06.130683 (+ 26957us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:08:06.143274 (+ 12591us) write_op.cc:312] APPLY: finished
0709 13:08:06.161364 (+ 18090us) log.cc:844] Serialized 22 byte log entry
0709 13:08:06.169969 (+  8605us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:06.175511 (+  5542us) write_op.cc:454] Released schema lock
0709 13:08:06.178005 (+  2494us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:06.233802 (+ 55797us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:06.243129 (+  9327us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":417514,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":86761,"apply.run_cpu_time_us":5069,"apply.run_wall_time_us":230078,"num_ops":1,"prepare.queue_time_us":429521,"prepare.run_cpu_time_us":3727,"prepare.run_wall_time_us":38435,"replication_time_us":474789,"spinlock_wait_cycles":6013312,"thread_start_us":62691,"threads_started":1}]]}
I20260709 13:08:07.056545  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:06.998730  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:06.991498  6520 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:06.999408  5851 raft_consensus.cc:2393] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 3: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:07.034965  6285 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.048233  6507 raft_consensus.cc:493] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.028891  6302 consensus_queue.cc:1048] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:08:07.047075  6456 raft_consensus.cc:493] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.056675  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:07.050549  6318 raft_consensus.cc:493] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.141541  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:07.144850  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:07.146694  6321 consensus_queue.cc:237] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:07.168978  5916 raft_consensus.cc:2393] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 3: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:07.169644  5774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60934 (request call id 101) took 1377 ms. Trace:
W20260709 13:08:07.170748  6528 log.cc:927] Time spent T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 13:08:07.147639  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:07.148891  6378 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:07.149235  6218 raft_consensus.cc:3055] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:07.069247  6320 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:07.155614  6285 raft_consensus.cc:3060] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:07.140401  5924 raft_consensus.cc:2468] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
W20260709 13:08:07.178048  6532 log.cc:927] Time spent T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:08:07.163067  6456 raft_consensus.cc:515] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:07.179857  6320 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:07.180482  6514 raft_consensus.cc:493] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:07.180913  6378 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:07.182782  6326 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:07.162704  6318 raft_consensus.cc:3060] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:07.138702  6520 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:07.161551  6474 consensus_queue.cc:260] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:07.175843  6218 raft_consensus.cc:740] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
W20260709 13:08:07.190450  6529 log.cc:927] Time spent T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:08:07.199906  5973 tablet_bootstrap.cc:654] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.185161  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:07.189476  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1e618980cb143c69eb4b401ba701662" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:07.208302  5922 raft_consensus.cc:2368] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 2. Current term is 3.
I20260709 13:08:07.213810  5970 raft_consensus.cc:359] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:07.160346  6507 raft_consensus.cc:515] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:07.170212  5774 rpcz_store.cc:276] 0709 13:08:05.792529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.797839 (+  5310us) service_pool.cc:224] Handling call
0709 13:08:06.008825 (+210986us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b5800081398. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8e9c 00007f78339e8a5b 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:08:07.169444 (+1160619us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54004bfd54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:07.169557 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618625152}
I20260709 13:08:07.361367  6318 raft_consensus.cc:515] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:07.369292  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:08:07.343585  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 356736us
I20260709 13:08:07.381088  5970 raft_consensus.cc:385] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.416095  5970 raft_consensus.cc:740] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.416827  5970 consensus_queue.cc:260] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:07.418391  5970 ts_tablet_manager.cc:1434] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.526s	user 0.007s	sys 0.000s
W20260709 13:08:07.418651  5970 ts_tablet_manager.cc:1467] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a: Tablet startup took 526ms
W20260709 13:08:07.392331  5889 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:07.421027  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:08:07.425840  6526 log.cc:927] Time spent T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.383s	user 0.032s	sys 0.032s
I20260709 13:08:07.426023  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:07.427048  5699 raft_consensus.cc:2393] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 3: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:07.367612  6515 raft_consensus.cc:2749] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:07.406608  6378 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:07.388370  5925 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:07.411060  6443 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:07.411746  6456 leader_election.cc:290] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:07.405483  6369 leader_election.cc:290] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:07.366079  6326 raft_consensus.cc:2804] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:07.421103  6507 leader_election.cc:290] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:07.418888  5970 ts_tablet_manager.cc:1469] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a: Trace:
0709 13:07:55.977036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:07:55.977234 (+   198us) service_pool.cc:224] Handling call
0709 13:07:55.998981 (+ 21747us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:07:55.999128 (+   147us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:07:56.085032 (+ 85904us) tablet_metadata.cc:665] Metadata flushed
0709 13:07:56.152599 (+ 67567us) tablet_replica.cc:206] Creating consensus instance
0709 13:07:56.153036 (+   437us) inbound_call.cc:177] Queueing success response
0709 13:08:06.194626 (+10041590us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:08:06.891908 (+697282us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:08:07.149394 (+257486us) tablet_replica.cc:261] Starting instrumentation
0709 13:08:07.149916 (+   522us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":5522,"tablet-open.queue_time_us":10026761}
I20260709 13:08:07.389514  6474 raft_consensus.cc:3060] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.410226  6520 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:07.437595  5971 tablet_bootstrap.cc:492] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:07.455008  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4f21d87306453b81a417014e41e4a4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:07.455508  5971 ts_tablet_manager.cc:1403] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.681s	user 0.038s	sys 0.056s
I20260709 13:08:07.455022  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:07.343614  6514 raft_consensus.cc:515] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:07.415659  6318 leader_election.cc:290] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:07.429160  5846 consensus_meta.cc:287] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent flushing consensus metadata: real 0.515s	user 0.002s	sys 0.008s
I20260709 13:08:07.450369  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:07.453972  5667 leader_election.cc:400] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Vote denied by peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241) with higher term. Message: Invalid argument: T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 2. Current term is 3.
I20260709 13:08:07.434787  6395 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:07.463301  5667 leader_election.cc:403] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:07.444078  6378 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
W20260709 13:08:07.502719  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 50777.7us
I20260709 13:08:07.402822  6218 consensus_queue.cc:260] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:07.515367  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 61536.8us
I20260709 13:08:07.518946  5970 tablet_bootstrap.cc:492] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a: Bootstrap starting.
I20260709 13:08:07.459151  6509 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:07.528126  5971 raft_consensus.cc:359] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:07.529323  5971 raft_consensus.cc:385] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.529695  5971 raft_consensus.cc:740] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.462809  6539 raft_consensus.cc:493] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:07.530587  5971 consensus_queue.cc:260] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:07.465543  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:07.465938  5846 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:07.511768  6326 raft_consensus.cc:697] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:07.508302  6520 raft_consensus.cc:493] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:07.553898  5971 ts_tablet_manager.cc:1434] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.098s	user 0.009s	sys 0.000s
W20260709 13:08:07.591723  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 82243.4us
I20260709 13:08:07.508771  6395 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:07.598330  5667 leader_election.cc:343] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.516296  6378 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:07.604265  5667 leader_election.cc:343] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.608464  6541 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:07.547202  6539 raft_consensus.cc:515] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:07.612285  6444 consensus_meta.cc:287] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: Time spent flushing consensus metadata: real 0.628s	user 0.000s	sys 0.010s
I20260709 13:08:07.612747  6444 raft_consensus.cc:515] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:07.613996  6539 leader_election.cc:290] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:07.614969  6444 leader_election.cc:290] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:07.602350  5848 consensus_meta.cc:287] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent flushing consensus metadata: real 0.722s	user 0.007s	sys 0.006s
I20260709 13:08:07.610160  6369 raft_consensus.cc:493] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:07.615094  6539 raft_consensus.cc:493] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:07.615620  6444 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:07.592917  6520 raft_consensus.cc:515] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:07.615818  6519 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:07.620455  6369 raft_consensus.cc:515] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:07.618597  5848 raft_consensus.cc:2468] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:07.637499  6378 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:07.638106  5923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59976 (request call id 114) took 1091 ms. Trace:
I20260709 13:08:07.638545  5923 rpcz_store.cc:276] 0709 13:08:06.547113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.553520 (+  6407us) service_pool.cc:224] Handling call
0709 13:08:07.637426 (+1083906us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b5800001b98. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8f8c 00007f78339e8ef6 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:08:07.637559 (+   133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.637750 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:07.637778 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:07.637819 (+    41us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:07.637851 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:07.637916 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:07.638022 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6597376}
I20260709 13:08:07.639672  6520 leader_election.cc:290] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:07.640539  5850 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57472 (request call id 139) took 1049 ms. Trace:
I20260709 13:08:07.660735  5850 rpcz_store.cc:276] 0709 13:08:06.590809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.593972 (+  3163us) service_pool.cc:224] Handling call
0709 13:08:07.640372 (+1046400us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400a690d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:07.640489 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310915968}
I20260709 13:08:07.662050  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:07.663053  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4f21d87306453b81a417014e41e4a4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:07.663195  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:07.621708  6539 raft_consensus.cc:515] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:07.665294  5851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57472 (request call id 134) took 1083 ms. Trace:
I20260709 13:08:07.665874  6539 leader_election.cc:290] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:07.672237  5846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57472 (request call id 140) took 1081 ms. Trace:
I20260709 13:08:07.672703  5846 rpcz_store.cc:276] 0709 13:08:06.591107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.592105 (+   998us) service_pool.cc:224] Handling call
0709 13:08:07.672041 (+1079936us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54004e98d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:07.672173 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629635328}
W20260709 13:08:07.674008  6524 consensus_meta.cc:287] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.000s
I20260709 13:08:07.674594  6524 raft_consensus.cc:515] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:07.640745  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:07.624969  6444 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:07.681641  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:07.682363  5844 raft_consensus.cc:2376] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3. Re-sending same reply.
I20260709 13:08:07.683703  6444 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:07.686528  6550 raft_consensus.cc:493] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
W20260709 13:08:07.686684  5771 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60934 (request call id 102) took 1893 ms (client timeout 1788 ms). Trace:
W20260709 13:08:07.687165  5771 rpcz_store.cc:269] 0709 13:08:05.792896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:05.797760 (+  4864us) service_pool.cc:224] Handling call
0709 13:08:06.056232 (+258472us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b5800081398. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8e9c 00007f78339e8a5b 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:08:07.686460 (+1630228us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b5400ac8ad4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:07.686615 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1174930304}
I20260709 13:08:07.688386  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:07.642731  6443 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:07.643601  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1e618980cb143c69eb4b401ba701662" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:07.645103  6285 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:07.670466  6227 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:07.702667  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:07.665696  5851 rpcz_store.cc:276] 0709 13:08:06.581537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.582348 (+   811us) service_pool.cc:224] Handling call
0709 13:08:07.665096 (+1082748us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b540025dfd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:07.665234 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832983808}
I20260709 13:08:07.704555  5699 raft_consensus.cc:2468] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:07.704996  5843 raft_consensus.cc:2393] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 3: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:07.647853  5973 tablet_bootstrap.cc:492] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:08:07.703495  5918 raft_consensus.cc:1275] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:07.705696  5973 ts_tablet_manager.cc:1403] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.796s	user 0.017s	sys 0.025s
I20260709 13:08:07.706646  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:07.707351  5923 raft_consensus.cc:2468] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:07.705603  6285 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:07.700517  6520 raft_consensus.cc:493] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:07.708501  6520 raft_consensus.cc:515] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:07.693217  6509 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:07.710572  6520 leader_election.cc:290] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:07.652793  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:07.656396  5972 tablet_bootstrap.cc:492] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:07.657511  6369 leader_election.cc:290] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:07.628211  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:07.624708  6326 consensus_queue.cc:237] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:07.658679  6507 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.629365  6519 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:07.668358  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:07.710389  6509 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:07.697705  5890 leader_election.cc:304] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:07.649960  6395 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:07.694908  5667 leader_election.cc:343] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.671046  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:07.670543  5970 tablet_bootstrap.cc:654] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.734016  5850 raft_consensus.cc:2393] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 3: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:07.710668  5973 raft_consensus.cc:359] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:07.734409  5973 raft_consensus.cc:385] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:07.734705  5973 raft_consensus.cc:740] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:07.733162  5890 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:07.715104  5920 raft_consensus.cc:2393] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 3: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:07.734807  5888 leader_election.cc:304] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:07.734705  5667 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
W20260709 13:08:07.743626  5667 leader_election.cc:343] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.744765  5667 leader_election.cc:304] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:07.740995  5973 consensus_queue.cc:260] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:07.741678  5850 raft_consensus.cc:1275] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:07.746827  5850 raft_consensus.cc:1275] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:07.747891  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:07.748322  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:07.748595  6443 raft_consensus.cc:2804] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:07.749042  6443 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:07.749567  6443 raft_consensus.cc:3060] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:07.749737  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:07.700644  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:07.752157  5971 tablet_bootstrap.cc:492] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:07.751653  6395 raft_consensus.cc:2804] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:07.753057  6395 raft_consensus.cc:493] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:07.753409  6395 raft_consensus.cc:3060] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:07.757596  5667 leader_election.cc:343] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.758318  5850 raft_consensus.cc:3060] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:07.759168  6539 raft_consensus.cc:493] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:07.760787  5667 leader_election.cc:304] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:07.731631  6458 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:07.768796  6369 raft_consensus.cc:2804] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:07.774940  5973 ts_tablet_manager.cc:1434] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.000s
I20260709 13:08:07.740691  6520 raft_consensus.cc:2749] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:07.741384  6541 raft_consensus.cc:2749] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241) with higher term. Message: Invalid argument: T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 2. Current term is 3.
I20260709 13:08:07.687188  6550 raft_consensus.cc:515] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:07.777508  5774 raft_consensus.cc:2393] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 3: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:07.777945  6318 consensus_queue.cc:1048] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:07.772992  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:07.778690  6285 raft_consensus.cc:493] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:07.779487  6524 leader_election.cc:290] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:07.780861  5973 tablet_bootstrap.cc:492] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:07.774279  6369 raft_consensus.cc:493] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:07.787263  5971 tablet_bootstrap.cc:654] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:07.756593  6378 raft_consensus.cc:2804] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:07.793356  5667 leader_election.cc:304] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:07.801245  6515 consensus_queue.cc:1048] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:07.731648  6507 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:07.787328  6509 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:07.774354  6516 raft_consensus.cc:2749] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:07.777674  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:07.809367  5923 raft_consensus.cc:2393] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:07.866442  5742 leader_election.cc:304] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:07.774083  6539 raft_consensus.cc:515] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:07.831583  6369 raft_consensus.cc:3060] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.789261  6458 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:07.826854  6452 raft_consensus.cc:493] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.832254  6378 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:07.803416  6290 consensus_queue.cc:1048] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
W20260709 13:08:07.886718  5742 leader_election.cc:343] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.887253  6073 raft_consensus.cc:2749] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:07.843997  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:08:07.896721  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 79295.3us
I20260709 13:08:07.832029  6474 raft_consensus.cc:493] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:07.896433  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:07.836692  6524 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
W20260709 13:08:07.897081  5742 leader_election.cc:343] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:07.905143  5776 raft_consensus.cc:3060] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:07.828380  6285 raft_consensus.cc:515] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:07.905224  5742 leader_election.cc:304] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:07.901391  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 87877.6us
W20260709 13:08:07.920485  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 127229us
I20260709 13:08:07.907611  5917 raft_consensus.cc:2410] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:08:07.907380  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:07.929096  6073 raft_consensus.cc:2749] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:07.907964  6402 raft_consensus.cc:493] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:07.908061  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:07.909176  6539 leader_election.cc:290] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:07.908958  6524 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:07.962544  6524 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:07.913102  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:07.915644  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:07.916674  6519 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:07.917295  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:07.877334  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:07.929608  5742 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:07.888347  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:07.942041  5776 raft_consensus.cc:2468] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:08:07.895051  6378 raft_consensus.cc:3060] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:08.021612  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:08.022337  5850 raft_consensus.cc:2468] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
W20260709 13:08:08.023591  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 94074.7us
I20260709 13:08:08.024183  6321 raft_consensus.cc:2749] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:08.024534  5742 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e
W20260709 13:08:08.025858  5742 leader_election.cc:343] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.026377  5742 leader_election.cc:304] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:07.894456  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:07.940443  6474 raft_consensus.cc:515] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:08.029822  6394 raft_consensus.cc:2804] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:07.902146  6227 raft_consensus.cc:493] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.030354  6394 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:08.030911  6394 raft_consensus.cc:3060] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.030875  6227 raft_consensus.cc:515] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:07.919793  6541 raft_consensus.cc:2804] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:08:07.938956  6548 log.cc:927] Time spent T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
W20260709 13:08:07.977406  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 52686.2us
I20260709 13:08:07.908061  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:07.945223  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:08.012917  5695 raft_consensus.cc:2393] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:08.013774  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:08.015650  6458 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:08.017644  6507 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:07.899842  6452 raft_consensus.cc:515] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:08.023419  6285 leader_election.cc:290] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:08.023519  5850 raft_consensus.cc:1275] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:07.948549  6402 raft_consensus.cc:515] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:07.967123  6423 raft_consensus.cc:493] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:07.967382  6509 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
W20260709 13:08:08.064400  5891 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.131:37585 timed out after 1.942s (SENT) [suppressed 1 similar messages]
I20260709 13:08:08.038096  6319 raft_consensus.cc:2749] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:08:08.094197  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 172955us
I20260709 13:08:08.034631  5742 leader_election.cc:304] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:08.043306  6541 raft_consensus.cc:493] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:08.097811  6541 raft_consensus.cc:3060] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:08.093978  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:08.099972  5971 tablet_bootstrap.cc:492] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:08.100554  5971 ts_tablet_manager.cc:1403] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.355s	user 0.016s	sys 0.000s
I20260709 13:08:08.100445  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.101217  5924 raft_consensus.cc:2468] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:08.075948  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:07.996448  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.102692  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:08.104337  5923 raft_consensus.cc:2468] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:08.105376  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.106222  6452 leader_election.cc:290] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:08.095793  6509 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:08.108211  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:08.109774  6509 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:08.041749  5972 tablet_bootstrap.cc:654] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.070695  6227 leader_election.cc:290] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:08.095556  6423 raft_consensus.cc:515] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:08.109375  5924 raft_consensus.cc:2393] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:08.113304  6423 leader_election.cc:290] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:08.114133  6541 raft_consensus.cc:515] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.115527  6443 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.118235  6402 leader_election.cc:290] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:08.086056  5850 raft_consensus.cc:1240] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:08.119485  5701 raft_consensus.cc:1240] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:08.119779  5973 tablet_bootstrap.cc:654] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:08.124513  6552 log.cc:927] Time spent T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:08:08.126703  6541 leader_election.cc:290] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:08.130445  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 95477.2us
I20260709 13:08:08.154393  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
W20260709 13:08:08.178239  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 62792.4us
I20260709 13:08:08.196293  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:08.196796  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.126868  5692 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:08.199796  6443 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:08.200266  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.201263  5768 raft_consensus.cc:2468] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:08.201758  5924 raft_consensus.cc:1240] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:08.203265  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.203792  5846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57472 (request call id 145) took 1413 ms. Trace:
I20260709 13:08:08.204255  5846 rpcz_store.cc:276] 0709 13:08:06.790310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.674926 (+884616us) service_pool.cc:224] Handling call
0709 13:08:07.675255 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:07.675507 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:08:07.844819 (+169312us) log.cc:844] Serialized 14 byte log entry
0709 13:08:08.203373 (+358554us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:08.203427 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:08.203475 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.203617 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.203734 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85802,"prepare.queue_time_us":293860,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"thread_start_us":330452,"threads_started":2}
I20260709 13:08:08.188362  5971 raft_consensus.cc:359] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.211483  5971 raft_consensus.cc:385] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.211963  5813 leader_election.cc:304] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:08.145198  6402 raft_consensus.cc:2749] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:08.216816  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:08.211956  5971 raft_consensus.cc:740] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.228637  5971 consensus_queue.cc:260] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.219277  5742 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:08.210407  6227 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
W20260709 13:08:08.259967  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 48136.9us
I20260709 13:08:08.266933  5971 ts_tablet_manager.cc:1434] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.166s	user 0.009s	sys 0.000s
I20260709 13:08:08.267928  5970 tablet_bootstrap.cc:492] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a: No bootstrap required, opened a new log
I20260709 13:08:08.268630  5970 ts_tablet_manager.cc:1403] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent bootstrapping tablet: real 0.805s	user 0.021s	sys 0.007s
I20260709 13:08:08.214666  6560 raft_consensus.cc:493] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:08.280210  5815 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:08.282655  5815 leader_election.cc:304] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:08.213523  5924 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:08.259114  6473 raft_consensus.cc:2804] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:08.267230  6524 raft_consensus.cc:2749] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.258798  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:08.312080  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:08.284384  6507 raft_consensus.cc:2749] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:08.274425  6285 raft_consensus.cc:493] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:08.271797  6321 consensus_queue.cc:1059] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
W20260709 13:08:08.286635  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 222262us
I20260709 13:08:08.326411  6285 raft_consensus.cc:515] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:08.290695  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.330020  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.313067  6564 raft_consensus.cc:2804] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.364820  6564 raft_consensus.cc:493] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.365253  6564 raft_consensus.cc:3060] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.365720  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.366359  6285 leader_election.cc:290] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:08.367192  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.367355  5891 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:08.368119  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.368878  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.369565  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.289779  6473 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:08.370206  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.371276  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "759acc6b90aa41a1b6c9053bd20846c1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:08.287981  6560 raft_consensus.cc:515] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:08.273169  6227 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:08.372746  6563 log.cc:927] Time spent T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:08:08.315362  5696 raft_consensus.cc:2393] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:08.384799  5890 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:08.374037  6319 raft_consensus.cc:493] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:08.391616  6473 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.416636  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:08.418284  6378 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:08.420367  6369 consensus_meta.cc:287] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: Time spent flushing consensus metadata: real 0.504s	user 0.001s	sys 0.011s
I20260709 13:08:08.420915  6369 raft_consensus.cc:515] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.423661  5918 raft_consensus.cc:2393] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:08.407142  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:08.426841  6227 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:08.429265  6395 consensus_meta.cc:287] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06: Time spent flushing consensus metadata: real 0.675s	user 0.014s	sys 0.000s
I20260709 13:08:08.429623  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:08.429836  6395 raft_consensus.cc:515] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.430348  5696 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:08.430954  5923 raft_consensus.cc:1240] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:08.431831  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:08.433635  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:08.435509  5970 raft_consensus.cc:359] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:08.435993  6319 raft_consensus.cc:515] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:08.436571  5970 raft_consensus.cc:385] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.436872  5970 raft_consensus.cc:740] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.437877  5970 consensus_queue.cc:260] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:08.438374  6319 leader_election.cc:290] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:08.424513  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.440086  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.441679  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.443357  6560 leader_election.cc:290] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:08.445770  6394 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:08.447160  5776 raft_consensus.cc:3060] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:08.448499  6394 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:08.526855  6473 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:08.456604  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:08.456907  5776 raft_consensus.cc:2468] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:08.569468  5769 raft_consensus.cc:1275] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:08.457842  6564 raft_consensus.cc:515] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:08.462880  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:08.471539  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:08.481452  6378 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:08.482641  5973 tablet_bootstrap.cc:492] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:08:08.490777  5813 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:08.498521  5972 tablet_bootstrap.cc:492] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:08.435596  6395 leader_election.cc:290] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:08.426836  6369 leader_election.cc:290] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:08.511314  5924 raft_consensus.cc:1275] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:08.493876  5970 ts_tablet_manager.cc:1434] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent starting tablet: real 0.225s	user 0.007s	sys 0.000s
I20260709 13:08:08.452828  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.536106  6394 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:08.538611  6319 raft_consensus.cc:493] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
W20260709 13:08:08.545598  6457 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.007s	sys 0.022s
I20260709 13:08:08.549068  5844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57474 (request call id 114) took 1725 ms. Trace:
I20260709 13:08:08.554301  5920 raft_consensus.cc:1275] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:08.571197  5697 raft_consensus.cc:2468] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:08.557389  6321 raft_consensus.cc:493] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.576892  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:08.577096  6321 raft_consensus.cc:515] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:08.578933  5844 rpcz_store.cc:276] 0709 13:08:06.823800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.683061 (+859261us) service_pool.cc:224] Handling call
0709 13:08:08.547865 (+864804us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b58001e6398. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8eee 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:08:08.548148 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.548689 (+   541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.548964 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16463104}
W20260709 13:08:08.559403  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 192186us
I20260709 13:08:08.560431  6326 raft_consensus.cc:2749] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:08.561827  6520 raft_consensus.cc:2749] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:08:08.562624  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 177964us
I20260709 13:08:08.580924  5891 leader_election.cc:304] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:08.581174  5890 leader_election.cc:304] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
W20260709 13:08:08.581710  5692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44362 (request call id 83) took 1759 ms (client timeout 1794 ms). Trace:
I20260709 13:08:08.581852  6326 raft_consensus.cc:2804] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:08.582557  6326 raft_consensus.cc:697] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:08.582779  5891 leader_election.cc:304] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:08.583232  6520 raft_consensus.cc:2749] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.583509  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.583583  6326 consensus_queue.cc:237] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:08.565438  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:08.579694  6321 leader_election.cc:290] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:08.567191  6280 raft_consensus.cc:493] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
W20260709 13:08:08.567610  6555 log.cc:927] Time spent T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.651s	user 0.008s	sys 0.017s
I20260709 13:08:08.587164  6538 consensus_queue.cc:1048] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:08.588564  6280 raft_consensus.cc:515] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:08.589463  5843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57472 (request call id 147) took 1223 ms. Trace:
I20260709 13:08:08.589898  5843 rpcz_store.cc:276] 0709 13:08:07.366165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.706593 (+340428us) service_pool.cc:224] Handling call
0709 13:08:08.575617 (+869024us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b58001e6398. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8e9c 00007f78339e8a5b 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:08:08.575883 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.589083 (+ 13200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:08.589116 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:08.589164 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:08.589204 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.589274 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.589386 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20932352}
I20260709 13:08:08.590988  6280 leader_election.cc:290] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:08.584638  6378 raft_consensus.cc:2749] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.573252  5849 raft_consensus.cc:2393] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:08.587373  5891 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:08.573880  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 83195.3us
I20260709 13:08:08.574107  6560 raft_consensus.cc:2804] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:08.597374  6560 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:08.597821  6560 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.572710  6564 leader_election.cc:290] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:08.574898  5770 raft_consensus.cc:2468] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:08.575558  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.575255  6319 raft_consensus.cc:515] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:08.601495  5924 raft_consensus.cc:2468] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:08.582583  5890 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:08.603080  6538 raft_consensus.cc:2749] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:08.575210  6394 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:08.607120  6394 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:08.607352  5698 raft_consensus.cc:1275] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260709 13:08:08.575768  6457 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.007s	sys 0.022s
I20260709 13:08:08.570327  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:08.574012  5972 ts_tablet_manager.cc:1403] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.918s	user 0.019s	sys 0.003s
W20260709 13:08:08.619170  6534 log.cc:927] Time spent T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 1.568s	user 0.012s	sys 0.074s
I20260709 13:08:08.623735  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:08.574441  6218 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:08.626088  6402 raft_consensus.cc:3055] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:08.626829  6402 raft_consensus.cc:740] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:08.573663  5973 ts_tablet_manager.cc:1403] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.793s	user 0.018s	sys 0.005s
I20260709 13:08:08.628312  6402 consensus_queue.cc:260] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:08.672068  5971 tablet_bootstrap.cc:492] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:08.672631  6402 raft_consensus.cc:3060] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:08.679418  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa162ebfa5094c25a3a311044956d167" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:08.680547  5697 raft_consensus.cc:2468] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:08:08.596876  6378 raft_consensus.cc:2749] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:08.684660  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 402084us
I20260709 13:08:08.607510  6319 leader_election.cc:290] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:08.631510  6473 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:08.272662  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 144546us
I20260709 13:08:08.699380  5815 leader_election.cc:304] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:08.646816  6320 raft_consensus.cc:493] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:08.646943  6474 leader_election.cc:290] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:08.650403  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.707147  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.707588  6227 raft_consensus.cc:2749] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:08.652810  5891 leader_election.cc:304] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:08.710493  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 57862.9us
I20260709 13:08:08.710980  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.655045  6514 leader_election.cc:290] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:08.655164  5666 leader_election.cc:304] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:08.660458  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.663703  5848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57474 (request call id 112) took 1840 ms. Trace:
I20260709 13:08:08.666788  5698 raft_consensus.cc:1240] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:08.713356  6378 raft_consensus.cc:2749] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.599629  5693 raft_consensus.cc:1240] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:08.715188  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:08.716056  5920 raft_consensus.cc:2393] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 3: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:08.670204  5847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57468 (request call id 115) took 1700 ms. Trace:
I20260709 13:08:08.721046  5816 leader_election.cc:304] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
W20260709 13:08:08.722653  5816 leader_election.cc:343] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.679764  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.697047  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:08.692389  5813 leader_election.cc:304] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:08.633399  5972 raft_consensus.cc:359] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.729705  5972 raft_consensus.cc:385] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.730568  5972 raft_consensus.cc:740] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.630978  5973 raft_consensus.cc:359] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.731433  5973 raft_consensus.cc:385] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:08.731796  5973 raft_consensus.cc:740] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:08.705265  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:08.705090  6320 raft_consensus.cc:515] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:08.650393  5890 leader_election.cc:304] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:08.737283  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 87002.5us
I20260709 13:08:08.718600  5815 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:08.737334  5742 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:08.738811  5972 consensus_queue.cc:260] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.740187  5973 consensus_queue.cc:260] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.723877  5816 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:08.743739  5972 ts_tablet_manager.cc:1434] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.126s	user 0.007s	sys 0.000s
I20260709 13:08:08.743849  5890 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:08.746598  5973 ts_tablet_manager.cc:1434] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.118s	user 0.005s	sys 0.001s
I20260709 13:08:08.748219  5973 tablet_bootstrap.cc:492] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:08.748068  6539 raft_consensus.cc:2749] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.748955  5890 leader_election.cc:304] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:08.750742  5890 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:08.751168  6539 raft_consensus.cc:2804] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:08.713171  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e83357d1cc74a0292e36bd8947af234" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:08:08.754696  5742 leader_election.cc:343] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:08.754789  5739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:08.726766  6509 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:08.582160  5692 rpcz_store.cc:269] 0709 13:08:06.821821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:06.830101 (+  8280us) service_pool.cc:224] Handling call
0709 13:08:07.662274 (+832173us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b5400808828. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8ea924 00007f783bad3227 00007f783bad2ab8 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6
0709 13:08:08.581508 (+919234us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b5400734454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:08.581618 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351787648}
I20260709 13:08:08.766244  6073 consensus_queue.cc:1048] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:08.767603  6073 consensus_queue.cc:1048] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:08.770006  6586 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.770619  6586 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:08.772766  6586 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:08.742193  6564 raft_consensus.cc:2749] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:08.752969  6378 raft_consensus.cc:2749] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.754582  6581 raft_consensus.cc:2804] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:08.751806  6539 raft_consensus.cc:493] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:08.738009  5775 raft_consensus.cc:2468] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:08.744329  6524 raft_consensus.cc:2804] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:08.791631  6473 consensus_queue.cc:1048] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:08.792994  5667 leader_election.cc:304] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:08.758103  6520 raft_consensus.cc:2749] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:08.764122  6394 raft_consensus.cc:2804] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:08.756135  5916 raft_consensus.cc:2468] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:08.743271  6514 consensus_queue.cc:1048] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 13:08:08.763100  6321 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:08.752616  6458 consensus_queue.cc:1059] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:08:08.813858  5742 leader_election.cc:304] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:08.779831  6573 raft_consensus.cc:2804] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:08.779146  5922 raft_consensus.cc:1275] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260709 13:08:08.812220  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 157082us
I20260709 13:08:08.778877  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.823383  5666 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:08.809937  6539 raft_consensus.cc:3060] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.775602  6509 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.813475  6592 raft_consensus.cc:493] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.811309  6524 raft_consensus.cc:493] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:08.814255  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:08.854916  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:08.719422  5847 rpcz_store.cc:276] 0709 13:08:06.969895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.706321 (+736426us) service_pool.cc:224] Handling call
0709 13:08:08.669169 (+962848us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b58001e6398. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8e9c 00007f78339e8a5b 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:08:08.669430 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.669685 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:08.669733 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:08.669793 (+    60us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:08.669844 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.669922 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.670109 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6659968}
I20260709 13:08:08.855772  5699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44350 (request call id 104) took 1027 ms. Trace:
I20260709 13:08:08.814756  6581 raft_consensus.cc:697] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:08.856196  5699 rpcz_store.cc:276] 0709 13:08:07.871680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.876309 (+  4629us) service_pool.cc:224] Handling call
0709 13:08:08.661619 (+785310us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b58009a9e98. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8eee 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:08:08.854845 (+193226us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b5c0043360c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783a477161 00007f783a618a79 00007f783bad29b6 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:08.855709 (+   864us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137844224}
I20260709 13:08:08.845329  6369 consensus_queue.cc:1048] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:08:08.858350  6550 leader_election.cc:290] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:08.828845  6458 raft_consensus.cc:3055] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:08.859828  6458 raft_consensus.cc:740] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:08.859864  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.832106  5739 leader_election.cc:304] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:08.861951  5972 tablet_bootstrap.cc:492] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
W20260709 13:08:08.826674  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 108112us
I20260709 13:08:08.830716  5973 tablet_bootstrap.cc:654] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.839372  5971 tablet_bootstrap.cc:654] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:08.713157  5848 rpcz_store.cc:276] 0709 13:08:06.822928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.681798 (+858870us) service_pool.cc:224] Handling call
0709 13:08:08.584942 (+903144us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b58001e6398. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8e9c 00007f78339e8a5b 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:08:08.585143 (+   201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.585345 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:08:08.657391 (+ 72046us) log.cc:844] Serialized 14 byte log entry
0709 13:08:08.663193 (+  5802us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:08.663276 (+    83us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:08.663349 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:08.663467 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:08.663624 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8,"prepare.queue_time_us":71278,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":126,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"spinlock_wait_cycles":4647808,"thread_start_us":76868,"threads_started":2,"wal-append.queue_time_us":980}
I20260709 13:08:08.843436  6586 raft_consensus.cc:2749] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:08.850402  6594 raft_consensus.cc:2804] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:08.826716  6516 raft_consensus.cc:2749] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:08.845439  6573 raft_consensus.cc:697] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:08.842548  6560 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:08.853379  6592 raft_consensus.cc:515] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.854172  6524 raft_consensus.cc:3060] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:08.824843  5922 pending_rounds.cc:85] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06: Aborting all ops after (but not including) 1
I20260709 13:08:08.856954  6581 consensus_queue.cc:237] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:08.815011  6394 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:08.866024  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:08.823688  6285 raft_consensus.cc:2749] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:08.795706  6590 consensus_queue.cc:1048] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:08:08.824697  5774 raft_consensus.cc:1275] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:08:08.872044  5742 leader_election.cc:343] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:08.875422  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 151630us
I20260709 13:08:08.876772  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:08.869474  6457 raft_consensus.cc:2804] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:08.823874  6321 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:08.870028  6587 raft_consensus.cc:493] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:08.867019  6594 raft_consensus.cc:493] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:08.860855  6458 consensus_queue.cc:260] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:08.886593  5922 pending_rounds.cc:107] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:08:08.915071  5816 leader_election.cc:304] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:08.839483  5891 leader_election.cc:304] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:08.924739  5922 raft_consensus.cc:2889] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:08.917268  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.887439  5920 raft_consensus.cc:2393] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:08.890900  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:08.950369  5816 leader_election.cc:304] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:08.919358  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 91512.6us
I20260709 13:08:08.917860  6457 raft_consensus.cc:493] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:08.890203  6394 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:08.923528  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 130690us
I20260709 13:08:08.922457  6594 raft_consensus.cc:3060] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.940021  6320 leader_election.cc:290] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:08.968685  5667 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
W20260709 13:08:08.952620  5816 leader_election.cc:343] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:08.962704  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:08.948469  6516 raft_consensus.cc:493] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
W20260709 13:08:08.963397  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 124053us
I20260709 13:08:08.963569  6592 leader_election.cc:290] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:08.964103  6573 consensus_queue.cc:237] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:08.956039  6550 raft_consensus.cc:2749] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.947165  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:08.958218  6458 raft_consensus.cc:3060] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:08.972692  5816 leader_election.cc:304] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:08.966925  5666 leader_election.cc:304] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:08.955705  6457 raft_consensus.cc:3060] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.963593  6560 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:08.954649  6564 raft_consensus.cc:2749] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.957041  6520 consensus_queue.cc:1048] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:08:08.925168  6539 raft_consensus.cc:515] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:08.956739  6599 raft_consensus.cc:2749] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:08.923413  6587 raft_consensus.cc:515] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:08.996449  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:09.006731  5916 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59964 (request call id 100) took 1263 ms. Trace:
I20260709 13:08:09.004779  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:09.003319  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:09.031114  5922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 156) took 1341 ms. Trace:
W20260709 13:08:09.032546  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 79957.3us
I20260709 13:08:09.035684  6564 raft_consensus.cc:2749] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.039526  5816 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:09.000504  6509 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:09.048292  5973 tablet_bootstrap.cc:492] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:08:09.049017  5973 ts_tablet_manager.cc:1403] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.301s	user 0.012s	sys 0.017s
W20260709 13:08:09.043992  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 75380.4us
I20260709 13:08:09.037845  5922 rpcz_store.cc:276] 0709 13:08:07.689887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.690001 (+   114us) service_pool.cc:224] Handling call
0709 13:08:07.690275 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:08.946165 (+1255890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.030823 (+ 84658us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54006031d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:08:09.031043 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2397392640}
I20260709 13:08:09.005506  6321 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:09.054613  5667 leader_election.cc:304] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:09.057005  5816 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:08.996542  5775 raft_consensus.cc:2468] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:08.986308  6516 raft_consensus.cc:515] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:09.061790  5816 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:08.998919  6550 consensus_queue.cc:1048] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.048s
I20260709 13:08:09.066519  5972 tablet_bootstrap.cc:654] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.068918  5816 leader_election.cc:304] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
W20260709 13:08:09.069285  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 102431us
I20260709 13:08:09.045799  5771 raft_consensus.cc:3060] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:08.999188  5891 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:09.073486  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:09.058441  6599 raft_consensus.cc:2804] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:08:09.076133  5740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.076795  6596 raft_consensus.cc:2804] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:09.037704  5916 rpcz_store.cc:276] 0709 13:08:07.743174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.743547 (+   373us) service_pool.cc:224] Handling call
0709 13:08:08.706980 (+963433us) spinlock_profiling.cc:243] Waited 5.32 ms on lock 0x7b5800001b98. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:08:09.006463 (+299483us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b5400495cd4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:09.006640 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93347456}
W20260709 13:08:09.080044  5741 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.080549  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:09.045935  6585 raft_consensus.cc:2749] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:09.087261  5816 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:09.096369  5815 leader_election.cc:304] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:09.067246  6509 raft_consensus.cc:2749] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:09.075053  6586 raft_consensus.cc:2804] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.083505  6524 raft_consensus.cc:515] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:09.075171  6564 raft_consensus.cc:2749] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:09.106549  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:09.114466  5816 leader_election.cc:304] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:09.116976  5776 raft_consensus.cc:3060] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.092903  6285 raft_consensus.cc:2749] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:09.093602  6599 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:09.106209  6326 raft_consensus.cc:2749] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:09.099112  6596 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:09.099460  6597 raft_consensus.cc:493] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:09.113225  6586 raft_consensus.cc:493] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.152158  5700 raft_consensus.cc:1275] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:09.114701  6507 raft_consensus.cc:2804] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:09.149514  6509 consensus_queue.cc:1059] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:08:09.146492  6423 raft_consensus.cc:2749] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:09.163480  6586 raft_consensus.cc:3060] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.194633  5973 raft_consensus.cc:359] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:09.199868  5973 raft_consensus.cc:385] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.200193  5973 raft_consensus.cc:740] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.201411  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.203006  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.196513  5776 raft_consensus.cc:2468] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:09.116811  6560 raft_consensus.cc:2749] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:09.224099  5973 consensus_queue.cc:260] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:09.198969  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 50520.6us
I20260709 13:08:09.124327  6599 raft_consensus.cc:3060] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:09.115227  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 116164us
W20260709 13:08:09.228848  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 78861.1us
I20260709 13:08:09.234571  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:09.229347  6073 consensus_queue.cc:1059] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:08:09.221477  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 124668us
I20260709 13:08:09.181103  5849 raft_consensus.cc:1240] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:09.177307  5846 raft_consensus.cc:1240] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:09.194690  6507 raft_consensus.cc:493] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:09.182047  6516 leader_election.cc:290] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:09.203466  5842 raft_consensus.cc:1240] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:09.253123  5923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59988 (request call id 161) took 1282 ms. Trace:
I20260709 13:08:09.253800  5923 rpcz_store.cc:276] 0709 13:08:07.970578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:08.578601 (+608023us) service_pool.cc:224] Handling call
0709 13:08:08.579020 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:08.579536 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:08.579581 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:08.611198 (+ 31617us) log.cc:844] Serialized 26 byte log entry
0709 13:08:09.060123 (+448925us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:09.063680 (+  3557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:09.071518 (+  7838us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:09.252840 (+181322us) raft_consensus.cc:1685] finished
0709 13:08:09.252869 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:09.253053 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1006,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":89,"thread_start_us":429049,"threads_started":1,"wal-append.queue_time_us":478101}
I20260709 13:08:09.248018  6507 raft_consensus.cc:3060] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.256975  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:09.258309  6589 raft_consensus.cc:3055] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:09.258788  6589 raft_consensus.cc:740] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:09.259953  6589 consensus_queue.cc:260] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:09.261160  6589 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:09.261502  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:09.262346  5767 raft_consensus.cc:2393] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 3: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:09.262583  6524 leader_election.cc:290] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:09.173570  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e83357d1cc74a0292e36bd8947af234" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:09.264346  5693 raft_consensus.cc:2468] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 0.
I20260709 13:08:09.182433  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa162ebfa5094c25a3a311044956d167" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.174904  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:09.184397  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.266705  5844 raft_consensus.cc:2393] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:09.174103  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.183566  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.176061  6590 raft_consensus.cc:493] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:09.150025  6596 raft_consensus.cc:3060] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:09.243832  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.288913  5850 raft_consensus.cc:2468] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:09.240624  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:09.245831  5973 ts_tablet_manager.cc:1434] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.196s	user 0.008s	sys 0.001s
I20260709 13:08:09.177692  6587 leader_election.cc:290] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:09.179657  5771 raft_consensus.cc:2468] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:09.207187  6539 leader_election.cc:290] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:09.290138  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 187274us
I20260709 13:08:09.268877  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:09.304924  5972 tablet_bootstrap.cc:492] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:09.307133  5972 ts_tablet_manager.cc:1403] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.445s	user 0.018s	sys 0.003s
I20260709 13:08:09.304867  5667 leader_election.cc:304] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.308522  5667 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.197017  6597 raft_consensus.cc:515] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:09.311040  5842 raft_consensus.cc:1240] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:09.312515  6597 leader_election.cc:290] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:09.313522  5973 tablet_bootstrap.cc:492] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:09.315404  5667 leader_election.cc:304] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.291111  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:09.326515  5971 tablet_bootstrap.cc:492] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:09.327296  5971 ts_tablet_manager.cc:1403] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.656s	user 0.037s	sys 0.019s
I20260709 13:08:09.285203  6594 raft_consensus.cc:515] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.322235  5847 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.335700  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:09.308594  6457 raft_consensus.cc:515] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:09.328807  6595 raft_consensus.cc:2749] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:08:09.365850  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 178231us
I20260709 13:08:09.292459  6590 raft_consensus.cc:515] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:09.331784  6600 raft_consensus.cc:2749] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:09.328590  6611 raft_consensus.cc:2749] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:09.372069  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:09.374573  5701 raft_consensus.cc:2376] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3. Re-sending same reply.
I20260709 13:08:09.379334  5888 leader_election.cc:304] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:09.380573  5888 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:09.334218  6369 consensus_queue.cc:1048] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:08:09.378691  6539 consensus_queue.cc:1059] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:08:09.384269  6540 raft_consensus.cc:2804] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:09.385000  6395 raft_consensus.cc:3055] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:09.382186  6550 raft_consensus.cc:2749] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:09.385406  5697 raft_consensus.cc:1240] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:09.388082  6540 raft_consensus.cc:697] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:09.386920  6395 raft_consensus.cc:740] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:09.396763  5845 raft_consensus.cc:1240] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:09.397465  6594 leader_election.cc:290] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:09.400369  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:09.414283  5773 raft_consensus.cc:1240] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:09.389200  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:09.418689  6507 raft_consensus.cc:515] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:09.424477  6395 consensus_queue.cc:260] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:09.410704  5742 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.404906  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:09.425819  6395 raft_consensus.cc:3060] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:09.426394  6607 log.cc:927] Time spent T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.247s	user 0.006s	sys 0.000s
I20260709 13:08:09.426514  6457 leader_election.cc:290] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:09.428717  6452 consensus_queue.cc:1048] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:09.409744  6394 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.415331  5971 raft_consensus.cc:359] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.424577  6581 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:09.432106  5971 raft_consensus.cc:385] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.432521  5971 raft_consensus.cc:740] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.432294  6581 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.421347  5739 leader_election.cc:304] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.434087  5768 raft_consensus.cc:1240] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:09.434523  6514 raft_consensus.cc:2749] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:09.435551  6540 consensus_queue.cc:237] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:09.439710  6394 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:09.443454  5739 leader_election.cc:304] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:09.392544  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1280a1f51344b42b4410fc2852da38d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:09.452759  6452 raft_consensus.cc:2749] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:09.454340  6507 leader_election.cc:290] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:09.441061  6581 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:09.441642  6594 consensus_queue.cc:1059] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:09.460134  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:08:09.458384  5742 leader_election.cc:343] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.442395  6473 raft_consensus.cc:2804] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:09.464043  6473 raft_consensus.cc:493] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:09.464429  6473 raft_consensus.cc:3060] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.442816  5973 tablet_bootstrap.cc:654] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.452656  6590 leader_election.cc:290] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:09.468637  6590 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:09.469108  6590 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:09.469667  5848 raft_consensus.cc:1275] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:09.423341  5972 raft_consensus.cc:359] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.443423  5769 raft_consensus.cc:1240] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:09.439569  5971 consensus_queue.cc:260] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.470850  5972 raft_consensus.cc:385] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.467957  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.471240  5972 raft_consensus.cc:740] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.473578  5742 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:09.455988  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.456840  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adfec13bcbce4db48b5f0dba7a8e5345" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.478466  5844 raft_consensus.cc:2376] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3. Re-sending same reply.
I20260709 13:08:09.478840  6594 raft_consensus.cc:3055] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:09.440902  6285 raft_consensus.cc:3055] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:09.479287  6594 raft_consensus.cc:740] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:09.479408  6285 raft_consensus.cc:740] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:09.458401  5890 leader_election.cc:304] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:09.480338  6594 consensus_queue.cc:260] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:09.483290  6378 raft_consensus.cc:2804] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:09.483603  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:09.483879  6378 raft_consensus.cc:697] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:09.484287  5916 raft_consensus.cc:2468] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:09.484823  6378 consensus_queue.cc:237] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.485782  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:09.485934  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:09.461313  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:09.462054  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:09.462164  6573 consensus_queue.cc:1059] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:09.462880  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:09.441537  6586 raft_consensus.cc:515] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:09.463304  5816 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.463433  5813 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:09.474128  5667 leader_election.cc:343] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.445487  5739 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:09.477404  6599 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.494380  5815 leader_election.cc:304] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:09.483311  5925 raft_consensus.cc:1275] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:08:09.502333  6605 log.cc:927] Time spent T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.274s	user 0.000s	sys 0.003s
W20260709 13:08:09.503146  5740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.504101  5971 ts_tablet_manager.cc:1434] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.176s	user 0.011s	sys 0.000s
I20260709 13:08:09.506757  5667 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.530347  5971 tablet_bootstrap.cc:492] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:09.500803  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:08:09.506703  5742 leader_election.cc:343] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.531903  6608 raft_consensus.cc:2749] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:09.532042  5742 leader_election.cc:304] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.533000  6458 raft_consensus.cc:2749] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.534778  6458 consensus_queue.cc:1059] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:09.507613  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1280a1f51344b42b4410fc2852da38d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.535941  6452 raft_consensus.cc:3055] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:09.536312  6452 raft_consensus.cc:740] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:09.536711  5667 leader_election.cc:304] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.537158  6452 consensus_queue.cc:260] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:09.510071  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 46727.6us
I20260709 13:08:09.505411  5920 raft_consensus.cc:2468] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:09.522116  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4f21d87306453b81a417014e41e4a4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:09.523029  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:08:09.523653  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 78269.6us
I20260709 13:08:09.523785  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.525439  5847 raft_consensus.cc:2468] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:09.526414  6394 raft_consensus.cc:2749] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:09.504530  5891 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:09.508314  5972 consensus_queue.cc:260] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:09.545130  5667 leader_election.cc:343] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.591224  5667 leader_election.cc:304] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.537606  6609 raft_consensus.cc:2749] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:09.504830  6589 raft_consensus.cc:2804] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:08:09.576581  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 113358us
I20260709 13:08:09.577685  5850 raft_consensus.cc:1240] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:09.577229  5739 leader_election.cc:304] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:09.621003  5816 leader_election.cc:343] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:09.585690  5742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:09.621649  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 44507us
I20260709 13:08:09.547713  6507 raft_consensus.cc:2804] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:09.541013  6566 raft_consensus.cc:2749] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:08:09.628269  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 83203.1us
I20260709 13:08:09.510143  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b37c67a7062e4372b72045052c828e2d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:09.584826  5699 raft_consensus.cc:2393] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 3: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:09.593768  5971 tablet_bootstrap.cc:654] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.582047  6597 raft_consensus.cc:2749] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:09.585819  6452 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.651141  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:09.620604  6589 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:09.674853  5972 ts_tablet_manager.cc:1434] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.367s	user 0.007s	sys 0.002s
I20260709 13:08:09.680625  5742 leader_election.cc:304] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:09.685593  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2d7f76b05b8419783f1a8f16afaec07" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:09.586364  6227 consensus_queue.cc:1059] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 13:08:09.626710  6514 raft_consensus.cc:2749] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:09.688094  5667 leader_election.cc:304] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.669412  6539 raft_consensus.cc:493] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.623865  6564 consensus_queue.cc:1059] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:08:09.668083  6590 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:09.667693  6592 raft_consensus.cc:2749] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:09.670950  6581 raft_consensus.cc:493] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:09.673276  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:09.702967  5922 raft_consensus.cc:1240] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:09.679903  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:09.683866  6589 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:09.708798  5972 tablet_bootstrap.cc:492] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:09.670636  6524 consensus_queue.cc:1048] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:08:09.696434  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.704522  6609 raft_consensus.cc:2749] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:09.709954  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 65710.8us
I20260709 13:08:09.731118  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa162ebfa5094c25a3a311044956d167" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:09.732249  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:09.712103  6631 raft_consensus.cc:2804] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:09.736409  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:09.708051  6581 raft_consensus.cc:515] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:09.701862  6539 raft_consensus.cc:515] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.753068  6539 leader_election.cc:290] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:09.755191  5848 raft_consensus.cc:1240] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:09.755653  5846 raft_consensus.cc:2393] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:09.749922  5973 tablet_bootstrap.cc:492] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb: No bootstrap required, opened a new log
I20260709 13:08:09.756537  5973 ts_tablet_manager.cc:1403] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.449s	user 0.017s	sys 0.013s
I20260709 13:08:09.737569  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:09.739974  6516 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:09.758049  6516 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:09.758543  6586 leader_election.cc:290] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:09.759332  6631 raft_consensus.cc:697] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:09.760504  6516 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:09.712919  6326 consensus_queue.cc:1048] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:08:09.760522  6631 consensus_queue.cc:237] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.763463  6581 leader_election.cc:290] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:09.765349  5973 raft_consensus.cc:359] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:09.776069  5973 raft_consensus.cc:385] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:09.776610  5973 raft_consensus.cc:740] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:09.777278  5973 consensus_queue.cc:260] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:09.769161  6524 raft_consensus.cc:493] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.779101  6524 raft_consensus.cc:515] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:09.780560  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 56437.2us
I20260709 13:08:09.772264  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:09.773138  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:09.773777  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:09.782322  5891 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:09.714354  6596 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:09.784431  6596 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:09.784873  6558 raft_consensus.cc:3055] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:09.785223  6326 raft_consensus.cc:2749] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:09.785358  6558 raft_consensus.cc:740] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:09.775255  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:09.776019  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4f21d87306453b81a417014e41e4a4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:09.787155  5698 raft_consensus.cc:2468] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:09.768586  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
W20260709 13:08:09.788496  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:09.790916  5701 raft_consensus.cc:1240] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:09.792186  6458 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.792390  5973 ts_tablet_manager.cc:1434] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.001s
I20260709 13:08:09.792578  5972 tablet_bootstrap.cc:654] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:09.745370  5739 leader_election.cc:304] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:09.783595  5891 leader_election.cc:304] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:09.797510  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 52281.6us
I20260709 13:08:09.811733  6539 raft_consensus.cc:2749] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:09.816968  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:09.818377  5739 leader_election.cc:304] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:09.792481  5847 raft_consensus.cc:1275] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:09.796770  6402 raft_consensus.cc:493] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:09.817075  5775 raft_consensus.cc:1275] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:08:09.840811  6507 raft_consensus.cc:697] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:09.820026  6326 consensus_queue.cc:1059] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:08:09.845469  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.818831  6320 raft_consensus.cc:2749] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:09.846356  5775 pending_rounds.cc:85] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a: Aborting all ops after (but not including) 1
I20260709 13:08:09.808569  6458 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:09.850705  5773 raft_consensus.cc:2393] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:09.821970  6473 raft_consensus.cc:515] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:09.848104  6593 raft_consensus.cc:2749] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:09.850841  5775 pending_rounds.cc:107] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:08:09.864220  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa162ebfa5094c25a3a311044956d167" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:09.922492  5701 raft_consensus.cc:3060] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.896297  6507 consensus_queue.cc:237] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:09.903640  5816 leader_election.cc:343] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.907866  5851 raft_consensus.cc:1240] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:08:09.908083  5742 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.848086  6402 raft_consensus.cc:515] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.854496  6302 consensus_queue.cc:1059] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:08:09.854398  6539 raft_consensus.cc:3055] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:08:09.927841  5742 leader_election.cc:343] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:09.929330  5813 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:09.933056  6326 consensus_queue.cc:1059] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:08:09.939363  6452 raft_consensus.cc:2804] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:09.956465  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:09.956826  6452 raft_consensus.cc:697] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:09.909745  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:09.958150  5923 raft_consensus.cc:2468] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:08:09.957906  6452 consensus_queue.cc:237] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.925376  5775 raft_consensus.cc:2889] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:09.958556  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e04051483df4ae3bf4c3f5ab037fda0" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:08:09.908828  6635 log.cc:927] Time spent T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:08:09.960296  6509 raft_consensus.cc:2749] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:09.933041  6285 consensus_queue.cc:260] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:09.961387  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:09.933959  6558 consensus_queue.cc:260] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.962203  5847 raft_consensus.cc:2393] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 3: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:09.962930  6558 raft_consensus.cc:3060] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.934599  6473 leader_election.cc:290] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:09.945616  5916 raft_consensus.cc:1240] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:09.946213  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:08:09.963416  6509 consensus_queue.cc:1175] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "792907f253c744aa9392e8d572f88e06" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:09.948652  6402 leader_election.cc:290] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:09.969563  6458 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:09.969758  5890 leader_election.cc:304] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:09.970786  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:09.972291  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e83357d1cc74a0292e36bd8947af234" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:09.972980  5698 raft_consensus.cc:3060] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:09.902381  6597 raft_consensus.cc:3055] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:09.973726  6597 raft_consensus.cc:740] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:09.974637  6597 consensus_queue.cc:260] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:09.975519  6326 raft_consensus.cc:3055] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:09.975970  6326 raft_consensus.cc:740] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:09.976212  6597 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:09.976852  6326 consensus_queue.cc:260] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:09.950624  5599 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39354 (request call id 56) took 5793 ms. Trace:
I20260709 13:08:09.954396  5813 leader_election.cc:304] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:09.981171  5813 leader_election.cc:304] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:09.962018  6599 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:09.979101  5599 rpcz_store.cc:276] 0709 13:08:04.142944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.143073 (+   129us) service_pool.cc:224] Handling call
0709 13:08:07.651058 (+3507985us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b54007392a8. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783ba36aa1 00007f783ba5d9b4 00007f783a5413dc 00007f783a502e0b 00007f783a502be4 00007f783ca032d1 00007f783ca03b2f 00007f783c8c2d4b 00007f783c9cac3e 00007f783ae8b2a4 00007f783ae8b232
0709 13:08:09.935831 (+2284773us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:08.291155 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:08.331611 (+ 40456us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:08.331658 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:08:08.331686 (+    28us) tablet.cc:662] Decoding operations
0709 13:08:08.332496 (+   810us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:08.332591 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:08.332635 (+    44us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:08:08.333121 (+   486us) tablet.cc:701] Row locks acquired
0709 13:08:08.333163 (+    42us) write_op.cc:260] PREPARE: finished
0709 13:08:08.333422 (+   259us) write_op.cc:270] Start()
0709 13:08:08.333572 (+   150us) write_op.cc:276] Timestamp: P: 1783602488333354 usec, L: 0
0709 13:08:08.333603 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:08:08.334528 (+   925us) log.cc:844] Serialized 1316 byte log entry
0709 13:08:08.777050 (+442522us) op_driver.cc:464] REPLICATION: finished
0709 13:08:08.843830 (+ 66780us) write_op.cc:301] APPLY: starting
0709 13:08:08.843962 (+   132us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:08.844161 (+   199us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:08.844319 (+   158us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:08.844878 (+   559us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:08.845083 (+   205us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:08:08.845149 (+    66us) write_op.cc:312] APPLY: finished
0709 13:08:08.845406 (+   257us) log.cc:844] Serialized 34 byte log entry
0709 13:08:08.845575 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:08.845667 (+    92us) write_op.cc:454] Released schema lock
0709 13:08:08.845705 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:08.846043 (+   338us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:08.846072 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":623692,"reported_tablets":11,"spinlock_wait_cycles":2750336,"child_traces":[["op",{"apply.queue_time_us":66471,"apply.run_cpu_time_us":2222,"apply.run_wall_time_us":2222,"num_ops":3,"prepare.queue_time_us":607136,"prepare.run_cpu_time_us":15563,"prepare.run_wall_time_us":350007,"raft.queue_time_us":147786,"raft.run_cpu_time_us":6895,"raft.run_wall_time_us":11779,"replication_time_us":443263,"thread_start_us":572020,"threads_started":4,"wal-append.queue_time_us":292043,"wal-append.run_cpu_time_us":103092,"wal-append.run_wall_time_us":1273596}]]}
W20260709 13:08:09.949645  6395 consensus_meta.cc:287] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.012s
I20260709 13:08:09.978116  6326 raft_consensus.cc:3060] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:10.009508  5667 consensus_peers.cc:597] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:10.007895  5701 raft_consensus.cc:2468] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:09.962098  6285 raft_consensus.cc:3060] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:10.023427  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 68837.2us
I20260709 13:08:09.952365  6539 raft_consensus.cc:740] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:09.990082  6423 raft_consensus.cc:2749] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:09.998890  6587 raft_consensus.cc:2804] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:10.046420  5742 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:10.025638  5667 leader_election.cc:304] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:10.036381  5815 leader_election.cc:304] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:09.990247  6402 raft_consensus.cc:493] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:10.009297  6550 raft_consensus.cc:3060] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.055784  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:09.998355  6524 leader_election.cc:290] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:10.008983  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:10.053828  5667 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:09.978312  6589 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.061664  5667 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:10.009037  6538 consensus_queue.cc:1048] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:08:10.069024  5739 leader_election.cc:304] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:10.069887  6073 raft_consensus.cc:2804] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:10.070384  6073 raft_consensus.cc:697] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:10.071316  5971 tablet_bootstrap.cc:492] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:10.072114  5971 ts_tablet_manager.cc:1403] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.542s	user 0.032s	sys 0.014s
I20260709 13:08:10.051585  6587 raft_consensus.cc:493] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.071630  6073 consensus_queue.cc:237] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.060497  6596 raft_consensus.cc:2749] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:10.008733  6581 consensus_queue.cc:1059] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 13:08:10.057219  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
W20260709 13:08:10.074606  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 61956.3us
I20260709 13:08:10.065176  6280 raft_consensus.cc:2749] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:10.029136  6636 consensus_queue.cc:1059] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:08:10.074323  6590 raft_consensus.cc:2804] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:08:10.094657  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 58383.2us
I20260709 13:08:10.068150  6592 raft_consensus.cc:2749] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:10.059100  6402 raft_consensus.cc:515] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:10.087692  6507 raft_consensus.cc:2749] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:10.080708  6587 raft_consensus.cc:3060] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.102073  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1e618980cb143c69eb4b401ba701662" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:10.127343  5972 tablet_bootstrap.cc:492] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:10.128121  5972 ts_tablet_manager.cc:1403] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.422s	user 0.029s	sys 0.054s
I20260709 13:08:10.129411  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:10.100873  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:10.102419  5771 raft_consensus.cc:1240] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 2. Current term is 3. Ops: []
W20260709 13:08:10.133062  5742 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.100097  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:10.095492  6539 consensus_queue.cc:260] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:10.130717  5701 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:10.090785  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
W20260709 13:08:10.140842  5742 leader_election.cc:343] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.097455  6590 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:10.142458  5772 raft_consensus.cc:3060] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.149050  5972 raft_consensus.cc:359] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:10.149829  6642 log.cc:927] Time spent T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:08:10.141280  6378 consensus_queue.cc:1059] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:08:10.141002  6524 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.152827  6524 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.154289  6594 raft_consensus.cc:3060] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:10.155042  6524 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:10.158620  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1e618980cb143c69eb4b401ba701662" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:10.159353  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:10.160313  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e83357d1cc74a0292e36bd8947af234" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:10.149838  5972 raft_consensus.cc:385] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.161792  5972 raft_consensus.cc:740] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
W20260709 13:08:10.162195  5667 leader_election.cc:343] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.163075  5667 leader_election.cc:304] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:10.163844  6402 leader_election.cc:290] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:10.164312  6516 raft_consensus.cc:2749] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:10.165536  6589 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:10.167618  5972 consensus_queue.cc:260] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:10.170614  5815 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
W20260709 13:08:10.179034  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 156615us
I20260709 13:08:10.181640  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:10.182988  5971 raft_consensus.cc:359] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.183815  5971 raft_consensus.cc:385] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.184199  5971 raft_consensus.cc:740] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.183524  6609 consensus_queue.cc:1059] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:10.185175  6609 raft_consensus.cc:3055] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:10.185506  6609 raft_consensus.cc:740] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:10.185603  5772 raft_consensus.cc:2468] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:08:10.186339  6609 consensus_queue.cc:260] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.188268  6609 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:10.189729  6394 raft_consensus.cc:3055] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:10.190179  6394 raft_consensus.cc:740] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:10.191130  5972 ts_tablet_manager.cc:1434] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.000s
I20260709 13:08:10.191013  6394 consensus_queue.cc:260] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:10.192966  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:10.099268  6540 consensus_queue.cc:1048] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 13:08:10.168004  6590 raft_consensus.cc:3060] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.174347  6596 consensus_queue.cc:1048] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:08:10.202279  6581 raft_consensus.cc:493] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:10.205191  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:10.205891  5693 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:10.203153  6550 raft_consensus.cc:2749] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:10.204806  6581 raft_consensus.cc:515] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.208837  5971 consensus_queue.cc:260] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.211298  5813 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:10.211475  5850 raft_consensus.cc:1275] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.213060  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:10.213747  5776 raft_consensus.cc:2410] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:10.214732  5813 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:10.196197  6524 raft_consensus.cc:2749] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:10.196803  5698 raft_consensus.cc:2468] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:10.220134  5971 ts_tablet_manager.cc:1434] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.147s	user 0.008s	sys 0.000s
I20260709 13:08:10.212702  6227 raft_consensus.cc:493] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.217885  6423 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:10.223579  6423 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.223944  6423 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:10.214890  6564 raft_consensus.cc:2749] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:10.219743  5739 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:10.227968  6636 raft_consensus.cc:2804] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:10.228358  6636 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.228682  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:10.228709  6636 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:10.233395  5701 raft_consensus.cc:1275] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:10.222885  6227 raft_consensus.cc:515] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:10.236037  6227 leader_election.cc:290] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:10.237108  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:08:10.238507  5816 leader_election.cc:343] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.241729  6587 raft_consensus.cc:515] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:10.220211  6550 consensus_queue.cc:1048] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:08:10.256527  5773 raft_consensus.cc:1275] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.261626  5971 tablet_bootstrap.cc:492] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:10.264719  6581 leader_election.cc:290] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:10.265843  6624 log.cc:927] Time spent T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.586s	user 0.005s	sys 0.000s
I20260709 13:08:10.218659  6394 raft_consensus.cc:3060] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:10.266218  5842 raft_consensus.cc:2468] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:10.266469  6654 consensus_queue.cc:1048] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:08:10.276924  6590 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:10.293876  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:10.321063  5701 raft_consensus.cc:1240] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:10.311803  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:10.323220  6423 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.326534  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:10.328648  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4f21d87306453b81a417014e41e4a4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:10.329512  5917 raft_consensus.cc:3060] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.330983  6423 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:10.332054  5843 raft_consensus.cc:2393] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 3: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:10.331946  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a62aede00e774cabb36368e7202cdff9" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:10.332765  5972 tablet_bootstrap.cc:492] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:10.332937  6608 raft_consensus.cc:3055] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:10.334120  5816 leader_election.cc:304] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:10.262462  6649 consensus_queue.cc:1048] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:08:10.349762  6320 raft_consensus.cc:3055] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:10.356668  6320 raft_consensus.cc:740] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:10.357571  6590 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:10.333357  6608 raft_consensus.cc:740] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:10.358016  6320 consensus_queue.cc:260] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:10.358865  6608 consensus_queue.cc:260] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.352128  6227 raft_consensus.cc:2749] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:10.360435  6608 raft_consensus.cc:3060] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.362715  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:10.372257  6073 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.373026  6073 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.384972  5692 raft_consensus.cc:1240] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:10.385982  6539 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.386255  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:10.388103  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:10.388278  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4f21d87306453b81a417014e41e4a4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:10.388924  5697 raft_consensus.cc:3060] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.388747  5692 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:10.394658  5971 tablet_bootstrap.cc:654] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.396308  5667 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:10.396579  5666 leader_election.cc:304] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:10.397156  6595 raft_consensus.cc:2749] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:10.397498  6590 raft_consensus.cc:2804] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:10.398078  6590 raft_consensus.cc:697] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:10.399008  6590 consensus_queue.cc:237] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:10.403443  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:10.403781  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:10.404243  5773 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:10.405659  5848 raft_consensus.cc:1275] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.408064  6227 consensus_queue.cc:1059] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:10.408281  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:10.410912  6423 raft_consensus.cc:3055] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:10.411218  5772 raft_consensus.cc:2468] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:10.411358  6423 raft_consensus.cc:740] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:10.412889  5773 raft_consensus.cc:2410] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:10.412361  6423 consensus_queue.cc:260] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:10.414143  6423 raft_consensus.cc:3060] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.421670  6073 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:10.427248  5891 leader_election.cc:304] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:10.365393  6587 leader_election.cc:290] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
W20260709 13:08:10.434274  5742 leader_election.cc:343] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.434969  5742 leader_election.cc:304] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:10.440558  6321 raft_consensus.cc:2749] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:08:10.440760  5742 leader_election.cc:343] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.467993  5774 raft_consensus.cc:1275] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:08:10.453154  5741 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.436902  6474 raft_consensus.cc:2804] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:10.470932  5972 tablet_bootstrap.cc:654] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:10.469182  5815 leader_election.cc:304] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:10.474581  5666 leader_election.cc:304] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:10.477396  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:10.479530  6596 raft_consensus.cc:493] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.477393  6581 raft_consensus.cc:3060] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:10.481407  6596 raft_consensus.cc:515] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:10.483935  6596 leader_election.cc:290] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:10.485123  6320 raft_consensus.cc:3060] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:10.479679  6474 raft_consensus.cc:697] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:10.486804  6474 consensus_queue.cc:237] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:10.488122  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:08:10.489564  5667 leader_election.cc:343] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.481387  5848 raft_consensus.cc:2468] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:10.479563  6587 raft_consensus.cc:2804] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:10.499719  6587 raft_consensus.cc:493] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.500227  6587 raft_consensus.cc:3060] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:10.506106  5742 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:10.509685  5698 raft_consensus.cc:2468] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:10.512311  6631 raft_consensus.cc:2804] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:10.471724  5917 raft_consensus.cc:2468] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:10.499792  5843 raft_consensus.cc:1275] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.507712  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:10.533295  5816 leader_election.cc:304] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:10.533492  6597 consensus_queue.cc:1048] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:10.513013  6452 consensus_queue.cc:1048] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:08:10.534659  6564 raft_consensus.cc:2804] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:10.535172  6564 raft_consensus.cc:697] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:10.505905  6227 raft_consensus.cc:493] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.536067  6564 consensus_queue.cc:237] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:10.536193  6227 raft_consensus.cc:515] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.539108  6227 leader_election.cc:290] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:10.540149  5917 raft_consensus.cc:1275] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.515527  6631 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.542599  6631 raft_consensus.cc:3060] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:10.545398  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:10.547358  6452 consensus_queue.cc:1048] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:10.548810  6452 consensus_queue.cc:1048] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:10.532469  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:10.530194  5697 raft_consensus.cc:2468] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:10.555796  5925 raft_consensus.cc:1275] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.569118  5597 catalog_manager.cc:5697] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:10.579001  5597 catalog_manager.cc:5697] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 2, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 2 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:10.667238  5597 catalog_manager.cc:5697] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 1 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } }
W20260709 13:08:10.669739  5667 leader_election.cc:343] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:10.670281  5667 leader_election.cc:304] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:10.555824  5769 raft_consensus.cc:2468] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:10.559800  5692 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 4.
I20260709 13:08:10.673768  5597 catalog_manager.cc:5697] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:10.682339  5597 catalog_manager.cc:5697] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 1 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } }
I20260709 13:08:10.685743  5597 catalog_manager.cc:5697] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } }
I20260709 13:08:10.688187  5597 catalog_manager.cc:5697] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } }
I20260709 13:08:10.689569  5597 catalog_manager.cc:5697] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:10.678963  6600 raft_consensus.cc:2749] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:10.694406  5813 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:10.696105  6550 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:10.696676  6550 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:10.699011  6550 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
W20260709 13:08:10.699860  6656 log.cc:927] Time spent T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:08:10.681859  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:10.687219  6631 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.700582  5697 raft_consensus.cc:2468] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
W20260709 13:08:10.689597  5741 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:10.701218  5770 raft_consensus.cc:2368] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 3. Current term is 4.
I20260709 13:08:10.576726  6652 consensus_queue.cc:1048] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.109s
I20260709 13:08:10.670951  6457 consensus_queue.cc:1048] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:08:10.713508  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:10.714835  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:10.721136  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:10.716573  6581 raft_consensus.cc:2749] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:10.721978  5701 raft_consensus.cc:2468] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:10.730034  5888 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:10.711164  6423 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:08:10.730580  6636 consensus_meta.cc:287] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.003s
I20260709 13:08:10.747792  5666 leader_election.cc:400] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Vote denied by peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429) with higher term. Message: Invalid argument: T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 3. Current term is 4.
I20260709 13:08:10.748354  5666 leader_election.cc:403] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:10.741521  6649 raft_consensus.cc:2804] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:10.757387  6649 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:10.757849  6649 raft_consensus.cc:3060] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:10.759158  6609 raft_consensus.cc:3060] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:08:10.759399  5769 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60920 (request call id 119) took 1136 ms. Trace:
I20260709 13:08:10.759907  5769 rpcz_store.cc:276] 0709 13:08:09.623423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.625678 (+  2255us) service_pool.cc:224] Handling call
0709 13:08:10.192772 (+567094us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b5800081398. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bad30a0 00007f783bad2984 00007f783504d6d4 00007f783504d662
0709 13:08:10.759219 (+566447us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54004bf354. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:10.759336 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722774144}
I20260709 13:08:10.778016  5815 leader_election.cc:304] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:10.778167  6631 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:10.778945  6285 raft_consensus.cc:2749] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:10.779554  6509 consensus_queue.cc:1048] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:10.789072  5815 leader_election.cc:304] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:10.742117  6423 raft_consensus.cc:697] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 4 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:10.797219  6652 raft_consensus.cc:2804] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:10.786090  6636 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:10.834504  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:10.851027  5739 leader_election.cc:304] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:10.819898  6652 raft_consensus.cc:697] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:10.865854  5739 leader_election.cc:304] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:10.867427  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:10.868451  5767 raft_consensus.cc:1240] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:10.871531  6320 raft_consensus.cc:2804] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:10.888110  5739 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
W20260709 13:08:10.896581  5742 consensus_peers.cc:597] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:10.874469  6321 raft_consensus.cc:2749] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:10.883674  6608 consensus_meta.cc:287] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.000s
I20260709 13:08:10.907595  5972 tablet_bootstrap.cc:492] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:10.908401  5971 tablet_bootstrap.cc:492] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:10.908567  5972 ts_tablet_manager.cc:1403] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.576s	user 0.014s	sys 0.008s
I20260709 13:08:10.908988  5971 ts_tablet_manager.cc:1403] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.648s	user 0.026s	sys 0.029s
I20260709 13:08:10.902437  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:10.872059  6423 consensus_queue.cc:237] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.931558  5851 raft_consensus.cc:3060] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:10.934060  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 44117.4us
I20260709 13:08:10.899334  6587 raft_consensus.cc:515] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:10.936697  5849 raft_consensus.cc:3060] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:10.934264  6631 raft_consensus.cc:2804] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:10.919291  6652 consensus_queue.cc:237] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:10.958436  5972 raft_consensus.cc:359] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:10.959232  5972 raft_consensus.cc:385] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.959602  5972 raft_consensus.cc:740] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.974785  5971 raft_consensus.cc:359] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:10.975749  5971 raft_consensus.cc:385] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:10.976131  5971 raft_consensus.cc:740] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:10.977028  5971 consensus_queue.cc:260] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:10.951979  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:10.953044  6636 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:10.990686  5971 ts_tablet_manager.cc:1434] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.002s
I20260709 13:08:10.971894  6631 raft_consensus.cc:697] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
W20260709 13:08:11.011338  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 52460.8us
I20260709 13:08:10.963883  6649 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:10.990015  6402 consensus_queue.cc:1048] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 13:08:11.013018  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:11.033771  5844 raft_consensus.cc:1275] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:08:11.034729  6680 log.cc:927] Time spent T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:08:11.028854  5697 raft_consensus.cc:1275] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:11.029564  5972 consensus_queue.cc:260] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:11.031212  6558 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:11.040545  5972 ts_tablet_manager.cc:1434] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.132s	user 0.010s	sys 0.000s
I20260709 13:08:11.041018  6326 raft_consensus.cc:493] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.040870  6558 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:11.016620  6320 raft_consensus.cc:697] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:11.061877  6587 leader_election.cc:290] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:11.064844  6649 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:11.046903  6524 raft_consensus.cc:493] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.056128  6631 consensus_queue.cc:237] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:11.066376  6524 raft_consensus.cc:515] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:11.051959  6378 consensus_queue.cc:1048] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:08:11.050561  6326 raft_consensus.cc:515] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:11.078702  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:11.080097  6550 consensus_queue.cc:1048] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:11.081267  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:11.081929  5697 raft_consensus.cc:3060] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:11.061914  6558 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:11.085798  5849 raft_consensus.cc:2468] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 4.
I20260709 13:08:11.087333  6320 consensus_queue.cc:237] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:11.078814  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:11.092994  5742 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
W20260709 13:08:11.094262  5816 leader_election.cc:343] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.096992  5739 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:11.101262  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:11.052677  6609 raft_consensus.cc:2749] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429) with higher term. Message: Invalid argument: T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 3. Current term is 4.
I20260709 13:08:11.120117  6685 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.120141  6540 consensus_queue.cc:1059] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:08:11.151623  6538 raft_consensus.cc:3055] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:11.103832  6524 leader_election.cc:290] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:11.114959  5844 raft_consensus.cc:3060] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:08:11.114312  6683 raft_consensus.cc:2749] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:11.158876  5971 tablet_bootstrap.cc:492] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:11.130420  6394 raft_consensus.cc:2804] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:11.121176  6654 raft_consensus.cc:493] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.131361  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:11.207094  6538 raft_consensus.cc:740] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:11.256059  5701 raft_consensus.cc:2468] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 4.
I20260709 13:08:11.220592  6423 raft_consensus.cc:493] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.250149  6558 consensus_queue.cc:1048] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.048s
W20260709 13:08:11.306103  6688 log.cc:927] Time spent T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:08:11.293676  6516 raft_consensus.cc:493] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.289178  6509 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:11.303829  6423 raft_consensus.cc:515] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:11.206782  6685 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:11.250633  6394 raft_consensus.cc:697] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
W20260709 13:08:11.372155  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 50703.6us
I20260709 13:08:11.292196  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:11.305074  5917 raft_consensus.cc:2368] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e for earlier term 3. Current term is 4.
I20260709 13:08:11.377167  6609 raft_consensus.cc:493] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.377645  6326 leader_election.cc:290] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:11.377722  6609 raft_consensus.cc:515] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:11.380254  6685 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:11.371656  6509 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:11.371881  6516 raft_consensus.cc:515] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:11.395058  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:11.395969  5850 raft_consensus.cc:2468] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:08:11.254635  6654 raft_consensus.cc:515] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:11.418503  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:11.419337  5849 raft_consensus.cc:2468] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
W20260709 13:08:11.423197  6694 log.cc:927] Time spent T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 13:08:11.423489  5697 raft_consensus.cc:2468] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 4.
W20260709 13:08:11.496850  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 65992.6us
I20260709 13:08:11.417443  6609 leader_election.cc:290] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:11.413704  5693 raft_consensus.cc:2468] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:11.545841  6639 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.546595  6639 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:11.549248  6639 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:11.513706  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:11.431160  6595 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
W20260709 13:08:11.516136  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 53583.7us
I20260709 13:08:11.499452  6516 leader_election.cc:290] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:11.518247  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:11.373816  6538 consensus_queue.cc:260] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:11.519124  6509 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:11.501719  6653 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:11.516017  6402 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.501394  6452 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.526152  6423 leader_election.cc:290] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:11.564627  6654 leader_election.cc:290] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:11.430548  6539 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:11.531723  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1280a1f51344b42b4410fc2852da38d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:11.533756  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:11.534458  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:11.535403  5816 leader_election.cc:400] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Vote denied by peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241) with higher term. Message: Invalid argument: T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e for earlier term 3. Current term is 4.
I20260709 13:08:11.541527  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:11.560638  5888 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
W20260709 13:08:11.612884  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 52331.6us
I20260709 13:08:11.531328  6394 consensus_queue.cc:237] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:11.530376  5971 tablet_bootstrap.cc:654] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:11.605937  5816 leader_election.cc:403] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:11.605079  5771 raft_consensus.cc:2393] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:11.653759  5813 leader_election.cc:304] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
W20260709 13:08:11.655507  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 120304us
I20260709 13:08:11.582551  6595 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:11.670077  5816 leader_election.cc:343] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:11.671442  6595 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:11.672390  5698 raft_consensus.cc:2393] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:11.620271  6654 raft_consensus.cc:2804] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:11.600045  6402 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:11.603854  6452 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:11.616225  6539 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:11.620447  5923 raft_consensus.cc:2468] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:11.598994  6653 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:11.657317  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
W20260709 13:08:11.712459  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 151004us
I20260709 13:08:11.706027  6698 raft_consensus.cc:2804] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:11.707228  6509 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:11.713563  6326 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
W20260709 13:08:11.725973  5844 consensus_meta.cc:287] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.000s
I20260709 13:08:11.749354  6698 raft_consensus.cc:493] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:11.788064  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 174274us
I20260709 13:08:11.766801  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:11.773042  6326 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:11.770750  5844 raft_consensus.cc:2468] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 4.
I20260709 13:08:11.795979  5924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59976 (request call id 177) took 1043 ms. Trace:
I20260709 13:08:11.796412  6699 raft_consensus.cc:2749] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241) with higher term. Message: Invalid argument: T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e for earlier term 3. Current term is 4.
W20260709 13:08:11.796576  6697 log.cc:927] Time spent T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:08:11.797107  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:11.796583  5924 rpcz_store.cc:276] 0709 13:08:10.753026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.790687 (+ 37661us) service_pool.cc:224] Handling call
0709 13:08:11.513301 (+722614us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b5800001b98. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8eee 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:08:11.513458 (+   157us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:11.513879 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:11.513912 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:11.514429 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:08:11.772713 (+258284us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:08:11.772778 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:11.772874 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:11.795605 (+ 22731us) raft_consensus.cc:1685] finished
0709 13:08:11.795632 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:11.795885 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3925,"raft.queue_time_us":4242,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":113,"spinlock_wait_cycles":9999744,"thread_start_us":251667,"threads_started":1,"wal-append.queue_time_us":266177}
I20260709 13:08:11.800285  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:11.801029  5923 raft_consensus.cc:2468] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:11.789242  6698 raft_consensus.cc:3060] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.798676  6227 raft_consensus.cc:493] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.804634  6227 raft_consensus.cc:515] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:11.809715  6402 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:11.807539  6227 leader_election.cc:290] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:11.811916  6653 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:11.818133  6539 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:11.771235  6509 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:11.822798  6705 raft_consensus.cc:493] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.823263  6705 raft_consensus.cc:515] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:11.825091  6705 leader_election.cc:290] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:11.827199  6654 raft_consensus.cc:697] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 4 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:11.726305  5695 raft_consensus.cc:2468] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:11.765771  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:11.813375  6326 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:11.880606  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 160958us
I20260709 13:08:11.883601  5813 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:11.954150  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 71165.2us
I20260709 13:08:11.954483  6509 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:11.885133  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:11.955330  6509 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:11.955832  5692 raft_consensus.cc:2468] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:11.888586  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1280a1f51344b42b4410fc2852da38d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:11.888908  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:11.798079  5844 raft_consensus.cc:2468] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:11.890970  5697 raft_consensus.cc:1275] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:08:11.897720  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:11.912520  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:11.940527  6706 raft_consensus.cc:493] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:11.940862  6703 raft_consensus.cc:493] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.941372  5701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44372 (request call id 182) took 1127 ms. Trace:
I20260709 13:08:11.943434  6452 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:11.951472  6590 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:11.964696  5701 rpcz_store.cc:276] 0709 13:08:10.824152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:10.836992 (+ 12840us) service_pool.cc:224] Handling call
0709 13:08:11.940797 (+1103805us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b5400485b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:11.941093 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641851776}
I20260709 13:08:11.971454  5843 raft_consensus.cc:2393] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:11.975003  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:11.974928  6590 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:11.976424  5921 raft_consensus.cc:2468] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:11.977743  5698 raft_consensus.cc:2468] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:11.978417  6590 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:11.979480  5971 tablet_bootstrap.cc:492] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:11.976449  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:11.981566  5845 raft_consensus.cc:2393] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 3: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:11.984154  6703 raft_consensus.cc:515] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:11.996929  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:12.002337  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:11.947710  6654 consensus_queue.cc:237] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:12.007635  5694 raft_consensus.cc:2468] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:12.006795  6703 leader_election.cc:290] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:12.013265  6699 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:11.990176  6706 raft_consensus.cc:515] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:12.016237  6699 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:12.021617  6706 leader_election.cc:290] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:12.026222  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:12.027307  6718 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.027585  5695 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:12.028384  6718 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:12.031316  6699 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:12.014147  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:12.032389  5845 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 4.
I20260709 13:08:12.033401  5739 leader_election.cc:304] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:12.036425  6402 raft_consensus.cc:2804] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:12.037112  6402 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.037596  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:12.039108  5768 raft_consensus.cc:3060] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.039449  6639 consensus_queue.cc:1048] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:12.042145  5891 leader_election.cc:304] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:12.043771  6718 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:12.045297  5971 ts_tablet_manager.cc:1403] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.887s	user 0.014s	sys 0.010s
I20260709 13:08:12.046967  5972 tablet_bootstrap.cc:492] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:12.047361  6649 raft_consensus.cc:2804] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:12.049297  5667 leader_election.cc:304] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:12.007854  5918 raft_consensus.cc:2468] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:12.037603  6402 raft_consensus.cc:3060] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:12.056463  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 207113us
I20260709 13:08:12.057621  6320 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.058197  6538 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:12.058923  5846 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57472 (request call id 187) took 1019 ms. Trace:
I20260709 13:08:12.068670  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:12.073387  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:12.073911  5813 leader_election.cc:304] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:12.077497  6649 raft_consensus.cc:493] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:12.086234  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 96804us
I20260709 13:08:12.089511  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:12.092095  6718 raft_consensus.cc:2804] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:12.093047  6718 raft_consensus.cc:493] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.093158  5767 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:12.093649  6718 raft_consensus.cc:3060] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:12.085327  6649 raft_consensus.cc:3060] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:12.099223  5667 leader_election.cc:304] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:12.081048  5846 rpcz_store.cc:276] 0709 13:08:11.039803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.042410 (+  2607us) service_pool.cc:224] Handling call
0709 13:08:12.026954 (+984544us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b58001e6398. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8eee 0000000000375939 00007f783ba5c1e0 00007f783bb250c4 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:08:12.027270 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.027760 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:12.027798 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:12.027872 (+    74us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:12.058376 (+ 30504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.058492 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.058703 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20370304}
I20260709 13:08:12.105876  5768 raft_consensus.cc:2468] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:12.107702  6423 raft_consensus.cc:2804] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:12.108238  6423 raft_consensus.cc:493] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.108616  6423 raft_consensus.cc:3060] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:12.098932  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:12.112509  5843 raft_consensus.cc:2468] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:12.112367  5971 raft_consensus.cc:359] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:12.083693  6320 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:12.115628  6681 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.113610  5971 raft_consensus.cc:385] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:12.104219  5699 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 4.
I20260709 13:08:12.074954  6699 consensus_queue.cc:1048] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:08:12.118813  5971 raft_consensus.cc:740] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:12.116433  6681 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:12.126480  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:12.128121  6320 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:12.140403  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 104276us
I20260709 13:08:12.141719  5971 consensus_queue.cc:260] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:12.143558  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 45284.2us
I20260709 13:08:12.145285  6687 raft_consensus.cc:493] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.150038  6581 raft_consensus.cc:2804] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:12.131806  6681 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:12.151284  6687 raft_consensus.cc:515] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:12.153800  5891 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
W20260709 13:08:12.154199  6717 log.cc:927] Time spent T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:08:12.157035  6326 raft_consensus.cc:2804] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:12.157542  6326 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:12.157990  6326 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:12.152185  6581 raft_consensus.cc:493] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.161036  6581 raft_consensus.cc:3060] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.162801  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:12.163573  5739 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:12.164523  6452 raft_consensus.cc:2804] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:12.164958  6452 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.165375  6452 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:12.170040  6378 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:12.170517  6707 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:12.170755  6378 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:12.171180  6707 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:12.173177  6378 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:12.173368  6707 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:12.175293  6540 raft_consensus.cc:493] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.175864  6540 raft_consensus.cc:515] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:12.176878  5972 tablet_bootstrap.cc:654] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.143244  5843 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:12.178417  6540 leader_election.cc:290] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:12.179523  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:12.181706  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:12.184072  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e83357d1cc74a0292e36bd8947af234" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:12.184897  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:12.189060  5813 leader_election.cc:304] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:12.190310  6227 raft_consensus.cc:2804] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:12.190762  6227 raft_consensus.cc:493] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.191170  6227 raft_consensus.cc:3060] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.196943  5971 ts_tablet_manager.cc:1434] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.121s	user 0.009s	sys 0.000s
I20260709 13:08:12.198606  5971 tablet_bootstrap.cc:492] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:12.200654  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:12.206691  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:12.211002  5848 raft_consensus.cc:2468] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:12.207577  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4f21d87306453b81a417014e41e4a4" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:12.217867  6320 raft_consensus.cc:493] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:12.218677  6320 raft_consensus.cc:515] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:12.220968  6320 leader_election.cc:290] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:12.221273  6649 raft_consensus.cc:515] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:12.223245  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 78309.8us
I20260709 13:08:12.224643  6227 raft_consensus.cc:515] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:12.225631  6698 raft_consensus.cc:515] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:12.234872  5742 consensus_peers.cc:597] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Couldn't send request to peer 792907f253c744aa9392e8d572f88e06. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:12.236438  6423 raft_consensus.cc:515] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:12.239461  6564 consensus_queue.cc:1048] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:12.244307  5891 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:12.157048  6687 leader_election.cc:290] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:12.257158  6649 leader_election.cc:290] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:12.258417  5844 raft_consensus.cc:2368] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2.
I20260709 13:08:12.258642  5888 leader_election.cc:304] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:12.261277  6698 leader_election.cc:290] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:12.264562  6538 raft_consensus.cc:3060] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.254062  5739 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:12.270366  6649 raft_consensus.cc:493] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:12.270695  6320 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:12.271335  6320 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:12.271891  6320 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:12.271307  6649 raft_consensus.cc:515] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:12.251991  6707 raft_consensus.cc:2804] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:12.280723  6707 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:12.281282  6707 raft_consensus.cc:3060] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.286365  6649 leader_election.cc:290] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:12.287609  6227 leader_election.cc:290] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:12.286463  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:12.276847  5816 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:12.290474  6452 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:12.439158  5694 raft_consensus.cc:2468] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
W20260709 13:08:12.439541  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 162905us
I20260709 13:08:12.440553  6452 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:12.312886  6706 raft_consensus.cc:493] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:12.441481  6706 raft_consensus.cc:515] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:12.444087  6706 leader_election.cc:290] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:12.309926  6402 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:12.444362  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:12.446528  6402 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:12.317303  5972 tablet_bootstrap.cc:492] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:12.279135  5596 catalog_manager.cc:5697] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 1 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.306782  5742 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:12.323370  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.209915  6540 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.210587  6540 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:12.349031  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:13.212316  5696 raft_consensus.cc:3060] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:12.368155  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1280a1f51344b42b4410fc2852da38d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:13.217438  5698 raft_consensus.cc:3060] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:12.400803  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.220932  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.222505  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4f21d87306453b81a417014e41e4a4" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:12.425657  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:12.431232  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:13.225715  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:13.226317  5692 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:12.463784  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.227779  5768 raft_consensus.cc:2393] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 4: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:12.478924  5597 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39338 (request call id 63) took 3395 ms. Trace:
W20260709 13:08:12.478722  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:12.494423  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.229259  5775 raft_consensus.cc:2368] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 for earlier term 3. Current term is 4.
I20260709 13:08:12.497092  5695 raft_consensus.cc:1275] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:13.230652  6652 raft_consensus.cc:493] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.231266  6652 raft_consensus.cc:515] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:12.507607  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:12.351987  5842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57468 (request call id 166) took 1282 ms. Trace:
I20260709 13:08:12.323850  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:12.407361  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.241634  5850 raft_consensus.cc:2468] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:12.406472  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1280a1f51344b42b4410fc2852da38d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.244304  5848 raft_consensus.cc:2393] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:12.411733  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:12.512214  5739 leader_election.cc:304] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:12.517330  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:08:13.246090  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 733911us
I20260709 13:08:12.393610  5891 leader_election.cc:400] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Vote denied by peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585) with higher term. Message: Invalid argument: T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2.
I20260709 13:08:13.249619  5891 leader_election.cc:403] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:13.250018  6631 raft_consensus.cc:2804] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:13.250617  6631 raft_consensus.cc:493] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:13.250772  5667 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:12.449612  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1280a1f51344b42b4410fc2852da38d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:12.414446  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:12.413604  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1280a1f51344b42b4410fc2852da38d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:12.452674  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:13.257404  5923 raft_consensus.cc:3060] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.527688  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:12.361857  6320 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:12.336078  6423 leader_election.cc:290] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:12.342549  5971 tablet_bootstrap.cc:654] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:12.447445  5972 ts_tablet_manager.cc:1403] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.401s	user 0.014s	sys 0.008s
I20260709 13:08:12.538308  6452 raft_consensus.cc:493] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:12.316814  6509 raft_consensus.cc:2749] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:12.325265  6326 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.273377  5972 raft_consensus.cc:359] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:12.454511  6564 raft_consensus.cc:493] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.278820  6564 raft_consensus.cc:515] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:12.453727  6698 raft_consensus.cc:2749] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:12.340705  6687 raft_consensus.cc:493] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:13.283430  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 890018us
I20260709 13:08:12.346462  6707 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.284674  5972 raft_consensus.cc:385] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.285022  5972 raft_consensus.cc:740] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.285739  5972 consensus_queue.cc:260] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:12.308642  6581 raft_consensus.cc:515] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:13.286751  5921 raft_consensus.cc:2468] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:13.288812  6581 leader_election.cc:290] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:12.355783  6538 raft_consensus.cc:2749] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:13.293911  5923 raft_consensus.cc:2468] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:12.842748  6723 raft_consensus.cc:493] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.294572  6743 raft_consensus.cc:2749] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election lost for term 1. Reason: Vote denied by peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585) with higher term. Message: Invalid argument: T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2.
I20260709 13:08:13.295116  6723 raft_consensus.cc:515] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:12.939644  6734 raft_consensus.cc:493] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
W20260709 13:08:13.026636  5920 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.042s	sys 0.061s
W20260709 13:08:13.296648  5920 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.043s	sys 0.062s
I20260709 13:08:13.296372  6734 raft_consensus.cc:515] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:13.036069  6736 raft_consensus.cc:493] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.298107  6736 raft_consensus.cc:515] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.038930  6726 raft_consensus.cc:493] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.047173  6738 raft_consensus.cc:493] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:13.304417  6738 raft_consensus.cc:515] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.305789  5971 tablet_bootstrap.cc:492] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:13.306391  5971 ts_tablet_manager.cc:1403] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 1.108s	user 0.011s	sys 0.011s
I20260709 13:08:13.307173  6423 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
W20260709 13:08:13.081457  6697 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.008s	sys 0.009s
I20260709 13:08:13.307721  6423 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:13.308337  6697 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.009s	sys 0.009s
I20260709 13:08:13.097926  6539 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.309595  5971 raft_consensus.cc:359] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:13.310022  6423 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:13.310375  5971 raft_consensus.cc:385] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:13.310698  5971 raft_consensus.cc:740] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.310432  6539 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.312887  6539 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.099125  6649 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.315745  6649 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.318202  6649 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:13.169660  6402 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.033s	sys 0.072s
I20260709 13:08:13.318483  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:13.176854  6689 raft_consensus.cc:493] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:13.319295  6738 leader_election.cc:290] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.185055  6636 raft_consensus.cc:2804] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:13.320288  6636 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.320727  6636 raft_consensus.cc:3060] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:13.322268  5972 ts_tablet_manager.cc:1434] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.008s
I20260709 13:08:13.322665  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.323532  5972 tablet_bootstrap.cc:492] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:13.324420  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.325879  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:13.327303  5696 raft_consensus.cc:2468] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:13.188581  6712 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.328254  6712 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:13.328790  5846 raft_consensus.cc:1275] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:13.201510  6378 raft_consensus.cc:493] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:13.232417  5699 raft_consensus.cc:2393] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 2: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:13.329887  6378 raft_consensus.cc:515] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.228758  5597 rpcz_store.cc:276] 0709 13:08:09.082786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:09.082943 (+   157us) service_pool.cc:224] Handling call
0709 13:08:12.478560 (+3395617us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:10.978149 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:11.057615 (+ 79466us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:11.057662 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:08:11.057680 (+    18us) tablet.cc:662] Decoding operations
0709 13:08:11.058611 (+   931us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:11.058674 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:11.058706 (+    32us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:08:11.059328 (+   622us) tablet.cc:701] Row locks acquired
0709 13:08:11.059342 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:08:11.059505 (+   163us) write_op.cc:270] Start()
0709 13:08:11.059626 (+   121us) write_op.cc:276] Timestamp: P: 1783602491059457 usec, L: 0
0709 13:08:11.059643 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:08:11.219644 (+160001us) log.cc:844] Serialized 3257 byte log entry
0709 13:08:11.837291 (+617647us) op_driver.cc:464] REPLICATION: finished
0709 13:08:12.057873 (+220582us) write_op.cc:301] APPLY: starting
0709 13:08:12.058083 (+   210us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:12.058536 (+   453us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:12.058790 (+   254us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:12.122339 (+ 63549us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:12.175650 (+ 53311us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:08:12.175742 (+    92us) write_op.cc:312] APPLY: finished
0709 13:08:12.177798 (+  2056us) log.cc:844] Serialized 64 byte log entry
0709 13:08:12.179473 (+  1675us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:12.179781 (+   308us) write_op.cc:454] Released schema lock
0709 13:08:12.179853 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:12.180712 (+   859us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:12.180751 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":196369,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":177529,"apply.run_cpu_time_us":5445,"apply.run_wall_time_us":156666,"num_ops":8,"prepare.queue_time_us":230551,"prepare.run_cpu_time_us":39682,"prepare.run_wall_time_us":609214,"raft.queue_time_us":454580,"raft.run_cpu_time_us":13769,"raft.run_wall_time_us":165663,"replication_time_us":777531,"thread_start_us":636207,"threads_started":5,"wal-append.queue_time_us":387315}]]}
I20260709 13:08:13.330518  5972 tablet_bootstrap.cc:654] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.240862  5843 raft_consensus.cc:3060] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:12.420068  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e83357d1cc74a0292e36bd8947af234" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:13.251091  6631 raft_consensus.cc:3060] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.412724  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:13.373498  5918 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:13.258848  5697 raft_consensus.cc:2393] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 3: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:12.422765  6595 raft_consensus.cc:493] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:13.397131  5667 leader_election.cc:343] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.396996  6595 raft_consensus.cc:515] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.397827  5667 leader_election.cc:304] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:13.399576  6595 leader_election.cc:290] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.273674  6740 consensus_queue.cc:1048] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:08:13.387684  5849 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 4.
I20260709 13:08:13.276818  5916 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 4.
I20260709 13:08:13.406445  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.406802  5916 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59964 (request call id 164) took 1230 ms. Trace:
I20260709 13:08:13.407230  5916 rpcz_store.cc:276] 0709 13:08:12.176775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.181224 (+  4449us) service_pool.cc:224] Handling call
0709 13:08:13.406612 (+1225388us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400520b54. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:13.406725 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2516584832}
I20260709 13:08:13.386278  5598 catalog_manager.cc:5697] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 2, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:13.275420  6326 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.271523  6452 raft_consensus.cc:515] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.410351  6452 leader_election.cc:290] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.276203  5851 raft_consensus.cc:1275] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:13.411851  5851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57468 (request call id 174) took 1313 ms. Trace:
I20260709 13:08:13.412241  5851 rpcz_store.cc:276] 0709 13:08:12.097973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.100998 (+  3025us) service_pool.cc:224] Handling call
0709 13:08:12.105259 (+  4261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:13.410978 (+1305719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.411603 (+   625us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540033d954. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:08:13.411793 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2691808000}
I20260709 13:08:13.283171  6687 raft_consensus.cc:515] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:12.338371  6681 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:13.234696  5842 rpcz_store.cc:276] 0709 13:08:11.042638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:11.046764 (+  4126us) service_pool.cc:224] Handling call
0709 13:08:12.028339 (+981575us) spinlock_profiling.cc:243] Waited 9.24 ms on lock 0x7b58001e6398. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f78339e8b71 00007f78339e8a22 00007f783c901c09 00007f783ba5bee0 00007f783bb250ab 00007f783bb25131 00007f783bae6fc0 00007f783bad1929 00007f783bad13d2 00007f7835048bd4
0709 13:08:12.028771 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:12.029425 (+   654us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:08:12.029938 (+   513us) log.cc:844] Serialized 14 byte log entry
0709 13:08:12.168124 (+138186us) spinlock_profiling.cc:243] Waited 47.5 ms on lock 0x7b4400022b4c. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f78338a8db1 00007f7833946c85 00007f78339476a2 00007f78366b91db 00007f78366b4aaf 00007f7833a030ac 00007f783c85dab9 00007f7833a1f69b 00007f7833a1d93f 00007f7833a1fc02 00007f78366a720b
0709 13:08:12.309422 (+141298us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:12.311292 (+  1870us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:12.315817 (+  4525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.318212 (+  2395us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:12.325483 (+  7271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3428,"prepare.queue_time_us":1034,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":5814,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":112,"spinlock_wait_cycles":130512000,"thread_start_us":217838,"threads_started":1,"wal-append.queue_time_us":222918}
I20260709 13:08:13.284168  6564 leader_election.cc:290] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:13.286053  6652 leader_election.cc:290] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:13.286135  6707 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:12.362329  6718 raft_consensus.cc:515] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:13.417508  6718 leader_election.cc:290] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:13.291172  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:13.418457  5847 raft_consensus.cc:2393] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:13.419450  5694 raft_consensus.cc:2393] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 3: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:13.421506  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.291725  5700 raft_consensus.cc:2368] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2.
I20260709 13:08:13.223723  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:13.422617  5701 raft_consensus.cc:3060] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:13.423478  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.427075  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.428370  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:13.429908  6631 raft_consensus.cc:515] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.292820  6540 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:13.318852  6402 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.034s	sys 0.073s
I20260709 13:08:13.410780  5598 catalog_manager.cc:5697] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from 792907f253c744aa9392e8d572f88e06 (127.27.71.132) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:13.298717  6734 leader_election.cc:290] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:13.331254  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 79205us
I20260709 13:08:13.330644  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.439761  5598 catalog_manager.cc:5697] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:13.441084  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:13.441942  5699 raft_consensus.cc:2368] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3.
I20260709 13:08:13.334996  6378 leader_election.cc:290] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.331362  6726 raft_consensus.cc:515] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.338587  6736 leader_election.cc:290] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.339689  6596 raft_consensus.cc:2804] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:13.343266  5596 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39364 (request call id 62) took 8386 ms. Trace:
I20260709 13:08:13.319365  6689 raft_consensus.cc:515] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.329187  6636 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.330493  6712 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:13.361287  5971 consensus_queue.cc:260] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:13.331161  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.370409  6723 leader_election.cc:290] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:13.371018  5692 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:13.378669  5850 raft_consensus.cc:2468] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
W20260709 13:08:13.379148  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 1.07384e+06us
I20260709 13:08:13.379150  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.379703  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:13.387249  6738 raft_consensus.cc:493] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.393031  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.393399  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.395320  6538 raft_consensus.cc:493] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.393405  5697 raft_consensus.cc:1275] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:13.274698  6320 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 5 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.400938  5693 raft_consensus.cc:1275] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:13.459419  6538 raft_consensus.cc:515] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:13.403502  5813 leader_election.cc:304] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:13.403385  6600 consensus_queue.cc:1048] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:08:13.460573  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 57143.1us
I20260709 13:08:13.460811  6509 raft_consensus.cc:2804] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:13.461632  6538 leader_election.cc:290] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:13.403815  5888 leader_election.cc:304] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:13.462456  6689 leader_election.cc:290] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:13.462762  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 59011.6us
I20260709 13:08:13.460582  5739 leader_election.cc:400] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Vote denied by peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419) with higher term. Message: Invalid argument: T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3.
I20260709 13:08:13.463785  5739 leader_election.cc:403] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:08:13.464401  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.464996  5693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44350 (request call id 166) took 1126 ms. Trace:
I20260709 13:08:13.465112  5772 raft_consensus.cc:2468] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:13.414439  6681 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:13.465592  5693 rpcz_store.cc:276] 0709 13:08:12.339261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.466845 (+127584us) service_pool.cc:224] Handling call
0709 13:08:12.467215 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:13.460089 (+992874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.464736 (+  4647us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b540026c0d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:08:13.464929 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2145034496}
I20260709 13:08:13.467437  6681 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:13.415099  6687 leader_election.cc:290] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:13.176136  6706 raft_consensus.cc:493] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.433126  5701 raft_consensus.cc:2468] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 4.
I20260709 13:08:13.457784  6452 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:13.468763  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.468979  6452 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.435217  5972 tablet_bootstrap.cc:492] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:13.470410  5972 ts_tablet_manager.cc:1403] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.147s	user 0.015s	sys 0.007s
I20260709 13:08:13.470356  5739 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:13.471076  6452 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.471566  5971 ts_tablet_manager.cc:1434] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.165s	user 0.008s	sys 0.001s
I20260709 13:08:13.457496  6738 raft_consensus.cc:515] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.472666  5971 tablet_bootstrap.cc:492] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e: Bootstrap starting.
I20260709 13:08:13.473820  6738 leader_election.cc:290] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.445385  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.474642  5775 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 4.
W20260709 13:08:13.445717  5816 leader_election.cc:343] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.475188  5846 raft_consensus.cc:2468] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:13.475782  5816 leader_election.cc:304] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:13.475991  5844 raft_consensus.cc:2468] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:13.454958  6736 raft_consensus.cc:493] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.476991  5816 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:13.477345  6509 raft_consensus.cc:697] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:13.477137  6736 raft_consensus.cc:515] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.477608  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.478632  6698 raft_consensus.cc:2804] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:13.478389  6509 consensus_queue.cc:237] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.479151  6698 raft_consensus.cc:697] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:13.479460  6736 leader_election.cc:290] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.480233  6564 raft_consensus.cc:2804] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:13.468322  6706 raft_consensus.cc:515] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:13.480661  6564 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.481056  6564 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:13.449941  5700 raft_consensus.cc:2393] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 2: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:13.482525  6706 leader_election.cc:290] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:13.459443  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.485656  5700 raft_consensus.cc:1275] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:13.464902  6320 raft_consensus.cc:3060] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:13.487429  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1e618980cb143c69eb4b401ba701662" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:13.490208  5971 tablet_bootstrap.cc:654] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:13.473628  5972 raft_consensus.cc:359] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.491019  5972 raft_consensus.cc:385] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:08:13.404469  5667 leader_election.cc:343] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:13.491766  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 87335.1us
I20260709 13:08:13.450150  5598 catalog_manager.cc:5697] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 3, leader changed from 792907f253c744aa9392e8d572f88e06 (127.27.71.132) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 3 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:13.444105  6540 raft_consensus.cc:493] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.480036  6698 consensus_queue.cc:237] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:13.448756  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:13.458684  5770 raft_consensus.cc:2368] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 for earlier term 3. Current term is 4.
I20260709 13:08:13.493156  6596 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.496467  6596 raft_consensus.cc:3060] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:13.491310  5972 raft_consensus.cc:740] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:13.496814  6723 consensus_queue.cc:1048] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:13.498065  5972 consensus_queue.cc:260] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.457636  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.457418  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:13.464541  6712 raft_consensus.cc:2749] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:13.435320  5843 raft_consensus.cc:2468] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:13.503319  5918 raft_consensus.cc:2468] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
W20260709 13:08:13.503703  5843 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57474 (request call id 167) took 1208 ms (client timeout 1574 ms). Trace:
W20260709 13:08:13.504227  5843 rpcz_store.cc:269] 0709 13:08:12.295446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.297673 (+  2227us) service_pool.cc:224] Handling call
0709 13:08:13.503602 (+1205929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:13.452076  5890 leader_election.cc:400] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429) with higher term. Message: Invalid argument: T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 for earlier term 3. Current term is 4.
I20260709 13:08:13.505376  5890 leader_election.cc:403] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:08:13.505960  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 54675.5us
I20260709 13:08:13.506769  5890 leader_election.cc:304] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:13.508087  5890 leader_election.cc:304] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:13.508179  6707 raft_consensus.cc:3060] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:08:13.511065  6744 raft_consensus.cc:2749] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:13.511814  6744 raft_consensus.cc:493] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.512352  6744 raft_consensus.cc:515] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.514551  6744 leader_election.cc:290] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.515618  5816 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:13.492448  5667 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:13.468078  5694 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:13.523458  6723 raft_consensus.cc:2804] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:13.523986  6723 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.524391  6723 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:13.524646  5697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44372 (request call id 210) took 1025 ms. Trace:
I20260709 13:08:13.525259  5697 rpcz_store.cc:276] 0709 13:08:12.499207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.383395 (+884188us) service_pool.cc:224] Handling call
0709 13:08:13.383715 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:13.491206 (+107491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.524558 (+ 33352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:13.491819  5739 leader_election.cc:304] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:13.494973  5774 raft_consensus.cc:2393] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 4: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:13.511693  6326 raft_consensus.cc:2804] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:13.495353  5891 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:13.531061  5972 ts_tablet_manager.cc:1434] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.002s
I20260709 13:08:13.495199  6402 raft_consensus.cc:2804] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:13.535010  5667 leader_election.cc:304] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:13.535775  6681 raft_consensus.cc:2804] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:13.536726  5891 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:13.493292  6538 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.523986  6736 raft_consensus.cc:2804] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:13.538930  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.540315  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:13.541003  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa162ebfa5094c25a3a311044956d167" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.541363  6744 raft_consensus.cc:2804] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:13.541585  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.533618  6689 raft_consensus.cc:2804] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:13.495534  6742 consensus_queue.cc:1048] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:08:13.570649  6726 leader_election.cc:290] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:13.564685  6746 raft_consensus.cc:2804] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:13.571818  6746 raft_consensus.cc:697] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:13.542054  5770 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60934 (request call id 144) took 1228 ms. Trace:
I20260709 13:08:13.547545  6402 raft_consensus.cc:697] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:13.534266  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.538453  6652 raft_consensus.cc:493] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.573153  6652 raft_consensus.cc:515] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.572752  6746 consensus_queue.cc:237] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.562186  6538 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:13.572443  5770 rpcz_store.cc:276] 0709 13:08:12.313700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.316733 (+  3033us) service_pool.cc:224] Handling call
0709 13:08:13.541809 (+1225076us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54004fb0d4. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:13.541975 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":574173952}
I20260709 13:08:13.538951  6712 raft_consensus.cc:493] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.577273  6712 raft_consensus.cc:515] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:13.575678  6652 leader_election.cc:290] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:13.522854  5916 raft_consensus.cc:2468] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:13.578016  6538 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:13.551239  5666 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:13.579281  6712 leader_election.cc:290] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:13.551651  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.579833  6741 consensus_queue.cc:1048] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:13.581295  6595 raft_consensus.cc:2764] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 13:08:13.553992  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 89797.3us
W20260709 13:08:13.555074  5667 leader_election.cc:343] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:13.582937  5888 leader_election.cc:400] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Vote denied by peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419) with higher term. Message: Invalid argument: T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 for earlier term 1. Current term is 2.
I20260709 13:08:13.583572  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.584807  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.573216  6402 consensus_queue.cc:237] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.589293  5848 raft_consensus.cc:1275] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:13.555264  5891 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:13.493754  6540 raft_consensus.cc:515] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.590528  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.591372  6540 leader_election.cc:290] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.531898  6326 raft_consensus.cc:493] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:13.564491  6423 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.564575  6744 raft_consensus.cc:697] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 4 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:13.564987  5843 raft_consensus.cc:2468] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 0.
I20260709 13:08:13.566529  5695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44362 (request call id 125) took 1140 ms. Trace:
I20260709 13:08:13.566887  6706 raft_consensus.cc:493] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:13.449685  5596 rpcz_store.cc:276] 0709 13:08:04.960213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:04.972122 (+ 11909us) service_pool.cc:224] Handling call
0709 13:08:13.343175 (+8371053us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:12.519023 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:12.527409 (+  8386us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:12.527463 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:08:12.527501 (+    38us) tablet.cc:662] Decoding operations
0709 13:08:12.528977 (+  1476us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:12.529120 (+   143us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:12.529163 (+    43us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:12.529477 (+   314us) tablet.cc:701] Row locks acquired
0709 13:08:12.529513 (+    36us) write_op.cc:260] PREPARE: finished
0709 13:08:12.529699 (+   186us) write_op.cc:270] Start()
0709 13:08:12.529855 (+   156us) write_op.cc:276] Timestamp: P: 1783602492529639 usec, L: 0
0709 13:08:12.529879 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:08:12.530861 (+   982us) log.cc:844] Serialized 540 byte log entry
0709 13:08:12.948464 (+417603us) op_driver.cc:464] REPLICATION: finished
0709 13:08:13.192775 (+244311us) write_op.cc:301] APPLY: starting
0709 13:08:13.192859 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:13.192973 (+   114us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:13.193063 (+    90us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:13.193332 (+   269us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:13.193476 (+   144us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:08:13.193517 (+    41us) write_op.cc:312] APPLY: finished
0709 13:08:13.193714 (+   197us) log.cc:844] Serialized 22 byte log entry
0709 13:08:13.193852 (+   138us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:13.193943 (+    91us) write_op.cc:454] Released schema lock
0709 13:08:13.193977 (+    34us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:13.194344 (+   367us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:13.194372 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":59235,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":244045,"apply.run_cpu_time_us":1529,"apply.run_wall_time_us":1529,"num_ops":1,"prepare.queue_time_us":34692,"prepare.run_cpu_time_us":3618,"prepare.run_wall_time_us":12197,"replication_time_us":418356,"thread_start_us":35685,"threads_started":1}]]}
I20260709 13:08:13.538079  6681 raft_consensus.cc:493] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.600549  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.608443  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.607031  6564 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.604786  5888 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:13.547756  6736 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.610873  5891 leader_election.cc:304] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:13.613878  5666 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:13.620507  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.599134  6631 leader_election.cc:290] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:13.619223  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:13.632162  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.599162  6326 raft_consensus.cc:3060] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:13.635260  5972 tablet_bootstrap.cc:492] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:13.592341  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
W20260709 13:08:13.637171  5797 debug-util.cc:398] Leaking SignalData structure 0x7b08000b4700 after lost signal to thread 27937
I20260709 13:08:13.637806  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.638653  6704 raft_consensus.cc:2749] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:13.638867  6706 raft_consensus.cc:515] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:13.639272  6652 raft_consensus.cc:493] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.604197  6423 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.621097  6681 raft_consensus.cc:3060] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:13.640080  6227 raft_consensus.cc:493] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.633796  6736 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:13.648450  5816 leader_election.cc:304] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:13.649925  5667 leader_election.cc:304] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:13.652314  6744 consensus_queue.cc:237] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.653653  5919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:13.655638  6564 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 5 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:13.657322  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.641511  5845 raft_consensus.cc:2393] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 3: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:13.634855  5698 raft_consensus.cc:2468] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:13.647958  5890 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:13.678191  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 64393.7us
I20260709 13:08:13.678562  5890 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:13.668489  6734 raft_consensus.cc:2804] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:13.668551  6227 raft_consensus.cc:515] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.669305  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ac5c3627ca49348e39094121c135c4" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:13.669605  6699 raft_consensus.cc:2804] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:13.612524  6712 consensus_queue.cc:1048] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:08:13.671463  6320 raft_consensus.cc:2749] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419) with higher term. Message: Invalid argument: T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3.
I20260709 13:08:13.724406  6740 raft_consensus.cc:493] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.640117  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.674454  5850 raft_consensus.cc:2468] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:13.614665  6689 raft_consensus.cc:697] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:13.649307  6539 raft_consensus.cc:2804] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:13.647363  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.680187  5890 leader_election.cc:400] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429) with higher term. Message: Invalid argument: T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 for earlier term 3. Current term is 4.
I20260709 13:08:13.659246  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:13.661800  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:13.629462  6596 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.691551  5971 tablet_bootstrap.cc:492] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e: No bootstrap required, opened a new log
I20260709 13:08:13.863293  5971 ts_tablet_manager.cc:1403] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent bootstrapping tablet: real 0.391s	user 0.010s	sys 0.006s
I20260709 13:08:13.681821  6742 raft_consensus.cc:2749] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:13.696777  5666 leader_election.cc:304] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:13.682277  6734 raft_consensus.cc:493] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.725104  5842 raft_consensus.cc:2393] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 5: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:13.820636  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:13.759716  6320 raft_consensus.cc:493] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
W20260709 13:08:13.800839  6756 log.cc:927] Time spent T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:08:13.832670  6423 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:13.676357  6652 raft_consensus.cc:515] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.689353  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:13.681391  6581 raft_consensus.cc:2749] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:13.814391  5921 raft_consensus.cc:2393] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:13.691063  5698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44350 (request call id 168) took 1349 ms. Trace:
I20260709 13:08:13.689476  6743 raft_consensus.cc:2749] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:13.764735  6636 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:13.831259  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96bcc183bf584d4e8fbd4d0a87c2e2ea" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:13.759729  6748 raft_consensus.cc:2804] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:13.798383  6706 leader_election.cc:290] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:13.821771  6227 leader_election.cc:290] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:13.761917  5774 raft_consensus.cc:2393] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:13.763842  6689 consensus_queue.cc:237] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:13.762286  6739 raft_consensus.cc:493] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:13.729610  6740 raft_consensus.cc:515] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.821153  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:13.762892  6539 raft_consensus.cc:697] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:13.764067  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3f19012a0a24bd88cecafe95b9b461c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:13.688934  6741 raft_consensus.cc:493] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:13.727900  6699 raft_consensus.cc:493] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:13.692147  5797 debug-util.cc:398] Leaking SignalData structure 0x7b0800150de0 after lost signal to thread 5633
I20260709 13:08:13.765085  6760 raft_consensus.cc:493] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:13.969106  5972 tablet_bootstrap.cc:654] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:13.969555  5797 debug-util.cc:398] Leaking SignalData structure 0x7b08001b1e00 after lost signal to thread 5725
I20260709 13:08:13.969281  6760 raft_consensus.cc:515] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:13.970867  5797 debug-util.cc:398] Leaking SignalData structure 0x7b08001505a0 after lost signal to thread 5800
W20260709 13:08:13.971200  5797 debug-util.cc:398] Leaking SignalData structure 0x7b08002b79c0 after lost signal to thread 5801
I20260709 13:08:13.971618  6760 leader_election.cc:290] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:13.859597  5890 leader_election.cc:403] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:13.823335  6698 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.846693  6712 raft_consensus.cc:493] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.847134  6378 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
W20260709 13:08:13.881942  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 185273us
I20260709 13:08:13.825037  6723 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.868029  5844 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:13.880880  6734 raft_consensus.cc:3060] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:13.862984  6452 raft_consensus.cc:493] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:13.904829  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.892115  6704 raft_consensus.cc:2749] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:13.900959  6707 raft_consensus.cc:2749] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429) with higher term. Message: Invalid argument: T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 for earlier term 3. Current term is 4.
I20260709 13:08:13.898904  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:13.895596  6320 raft_consensus.cc:515] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:13.908504  6748 raft_consensus.cc:493] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:13.959259  6739 raft_consensus.cc:515] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.964977  6539 consensus_queue.cc:237] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:13.965808  6766 raft_consensus.cc:493] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:13.966449  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:13.967161  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:13.967303  6740 leader_election.cc:290] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:13.900528  5698 rpcz_store.cc:276] 0709 13:08:12.340009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.367481 (+ 27472us) service_pool.cc:224] Handling call
0709 13:08:13.688982 (+1321501us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:13.959508  5843 raft_consensus.cc:2393] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:13.800343  6595 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:13.979046  5666 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:13.961702  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:13.982467  5891 leader_election.cc:304] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:13.972193  6699 raft_consensus.cc:3060] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.060030  6748 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:14.082726  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 103790us
W20260709 13:08:13.993384  5890 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 313424us
I20260709 13:08:13.982409  6746 raft_consensus.cc:3060] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:08:13.976806  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:08:13.998868  5797 debug-util.cc:398] Leaking SignalData structure 0x7b08003cac80 after lost signal to thread 5875
I20260709 13:08:13.972461  6741 raft_consensus.cc:515] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:13.992624  6712 raft_consensus.cc:515] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:13.609921  5695 rpcz_store.cc:276] 0709 13:08:12.425843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.233025 (+807182us) service_pool.cc:224] Handling call
0709 13:08:13.233431 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:13.334178 (+100747us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:08:13.362602 (+ 28424us) log.cc:844] Serialized 14 byte log entry
0709 13:08:13.453655 (+ 91053us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:13.455422 (+  1767us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:13.456093 (+   671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:13.541212 (+ 85119us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b1800107bc0. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f783c8e8de1 00007f783c8f55b1 00007f78339ee2a6 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419
0709 13:08:13.566120 (+ 24908us) spinlock_profiling.cc:243] Waited 65.7 ms on lock 0x7b5400486054. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b0de01 00007f7836b0c83e 00007f783bad1419 00007f7835048bd4 00007f7835048b62 00007f7835048a91 00007f7835048a0c 00007f7835047cc2 00007f7834e4848c 00007f7834e478c6
0709 13:08:13.566199 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:13.566409 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":239,"prepare.queue_time_us":28310,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":3794,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":1803,"spinlock_wait_cycles":231297152,"thread_start_us":113842,"threads_started":2,"wal-append.queue_time_us":141755}
I20260709 13:08:14.107051  6320 leader_election.cc:290] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:14.107822  5918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.100505  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.110296  6687 consensus_queue.cc:1048] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:14.110540  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:13.987236  6378 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:14.018128  5971 raft_consensus.cc:359] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:14.112994  5971 raft_consensus.cc:385] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.113399  5971 raft_consensus.cc:740] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.114109  5971 consensus_queue.cc:260] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:14.085435  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 103041us
W20260709 13:08:14.089133  6761 log.cc:927] Time spent T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:08:14.106702  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:13.979365  6698 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:14.120720  5815 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.71.130:39429 timed out after 1.563s (SENT) [suppressed 4 similar messages]
I20260709 13:08:14.112908  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.062417  6452 raft_consensus.cc:515] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.071419  5847 raft_consensus.cc:1240] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:14.120465  5971 ts_tablet_manager.cc:1434] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e: Time spent starting tablet: real 0.257s	user 0.007s	sys 0.002s
I20260709 13:08:14.089084  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.097851  6538 raft_consensus.cc:2804] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:14.123032  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.100924  6704 raft_consensus.cc:2749] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:14.082290  6595 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:14.120854  6652 leader_election.cc:290] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:14.072206  6766 raft_consensus.cc:515] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:14.106727  6739 leader_election.cc:290] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:14.171303  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 87480.8us
W20260709 13:08:14.145255  6764 log.cc:927] Time spent T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:08:14.145970  5918 raft_consensus.cc:2468] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:14.146533  5919 raft_consensus.cc:2468] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 4.
I20260709 13:08:14.147092  6378 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:14.149207  6741 leader_election.cc:290] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:14.151435  6326 raft_consensus.cc:515] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.134737  6540 raft_consensus.cc:2804] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:14.138337  6538 raft_consensus.cc:493] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:14.184309  6538 raft_consensus.cc:3060] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:14.187721  6452 leader_election.cc:290] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:14.152179  6767 log.cc:927] Time spent T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 13:08:14.145262  6769 log.cc:927] Time spent T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 13:08:14.151473  6770 log.cc:927] Time spent T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 13:08:14.180441  6762 log.cc:927] Time spent T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.219s	user 0.001s	sys 0.001s
I20260709 13:08:14.220376  6766 leader_election.cc:290] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:14.243011  6596 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:14.243816  6771 raft_consensus.cc:493] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:14.244782  6771 raft_consensus.cc:515] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:14.254395  5797 debug-util.cc:398] Leaking SignalData structure 0x7b08002ae9a0 after lost signal to thread 5949
I20260709 13:08:14.256549  6723 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
W20260709 13:08:14.257249  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 460151us
I20260709 13:08:14.259171  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
W20260709 13:08:14.253594  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 81288.5us
I20260709 13:08:14.264654  5922 raft_consensus.cc:1240] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:14.248170  6326 leader_election.cc:290] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:14.267011  5956 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.IsCreateTableDone -> {remote=127.27.71.190:45203, user_credentials={real_user=slave}} blocked reactor thread for 70206.4us
W20260709 13:08:14.267524  6681 consensus_meta.cc:287] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.001s
I20260709 13:08:14.267854  5972 tablet_bootstrap.cc:492] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:14.268406  5972 ts_tablet_manager.cc:1403] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.656s	user 0.037s	sys 0.028s
I20260709 13:08:14.263116  6712 leader_election.cc:290] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:14.268006  6681 raft_consensus.cc:515] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:14.278550  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 48401.2us
I20260709 13:08:14.236519  5844 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:14.296782  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:08:14.295281  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 274473us
W20260709 13:08:14.276767  6736 consensus_meta.cc:287] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: Time spent flushing consensus metadata: real 0.599s	user 0.004s	sys 0.008s
I20260709 13:08:14.300034  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.300700  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.301350  5742 leader_election.cc:304] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:14.263274  6771 leader_election.cc:290] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:14.301982  5797 debug-util.cc:398] Leaking SignalData structure 0x7b0800306c00 after lost signal to thread 6608
I20260709 13:08:14.303789  6698 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:14.304457  6452 consensus_queue.cc:1048] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:14.307161  6681 leader_election.cc:290] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:14.307195  5697 raft_consensus.cc:1240] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:14.308337  6595 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:14.309373  6699 raft_consensus.cc:515] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.301373  5972 raft_consensus.cc:359] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:14.350744  5972 raft_consensus.cc:385] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.299338  6748 raft_consensus.cc:515] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:14.303824  6768 log.cc:927] Time spent T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.330s	user 0.003s	sys 0.000s
I20260709 13:08:14.318090  5770 raft_consensus.cc:3060] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:14.347205  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.310454  6227 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.295042  6652 consensus_queue.cc:1059] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:08:14.346804  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.351176  5972 raft_consensus.cc:740] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
W20260709 13:08:14.318295  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 483730us
W20260709 13:08:14.349570  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 48308.5us
W20260709 13:08:14.352910  5797 debug-util.cc:398] Leaking SignalData structure 0x7b080028e040 after lost signal to thread 6703
I20260709 13:08:14.359193  5972 consensus_queue.cc:260] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:14.313035  6766 consensus_queue.cc:1048] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:08:14.358721  5742 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:14.361608  5667 leader_election.cc:304] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:14.361938  5924 raft_consensus.cc:2368] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 4. Current term is 5.
I20260709 13:08:14.363571  6699 leader_election.cc:290] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:14.365885  6748 leader_election.cc:290] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:14.367344  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1e618980cb143c69eb4b401ba701662" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:14.353780  6739 raft_consensus.cc:2749] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:14.358346  6227 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:14.326077  6734 raft_consensus.cc:515] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:14.376081  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.376674  5972 ts_tablet_manager.cc:1434] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.108s	user 0.008s	sys 0.000s
I20260709 13:08:14.386889  5775 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:14.355670  6538 raft_consensus.cc:515] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.378768  6540 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:14.397051  6540 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:14.383248  5923 raft_consensus.cc:3060] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.400153  6785 raft_consensus.cc:2749] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:14.402323  6538 leader_election.cc:290] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:14.402460  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.383466  5816 leader_election.cc:304] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:14.404599  5775 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 5.
I20260709 13:08:14.405294  6648 raft_consensus.cc:2804] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:14.405714  6648 raft_consensus.cc:493] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:14.406082  6648 raft_consensus.cc:3060] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:14.407265  5692 raft_consensus.cc:1275] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:14.408972  5920 raft_consensus.cc:2393] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:14.409952  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.382157  6746 raft_consensus.cc:2749] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429) with higher term. Message: Invalid argument: T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 for earlier term 3. Current term is 4.
I20260709 13:08:14.418500  5922 raft_consensus.cc:2468] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 4.
I20260709 13:08:14.420851  6648 raft_consensus.cc:515] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.421319  6734 leader_election.cc:290] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:14.383580  5742 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:14.419230  6540 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.440682  6540 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:14.373797  5770 raft_consensus.cc:2468] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:14.392627  6227 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:14.384191  6402 raft_consensus.cc:2749] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:14.438685  6648 leader_election.cc:290] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:14.396046  5891 leader_election.cc:304] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
W20260709 13:08:14.442653  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 46839us
I20260709 13:08:14.427052  6452 raft_consensus.cc:2749] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:14.427316  6723 raft_consensus.cc:3055] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:14.431172  5890 leader_election.cc:304] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:14.431867  5816 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:14.448824  5847 raft_consensus.cc:1275] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:14.451292  6699 raft_consensus.cc:2804] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:14.449115  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.449260  6539 raft_consensus.cc:3060] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.455258  5697 raft_consensus.cc:1275] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:14.449653  6723 raft_consensus.cc:740] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:14.457260  6723 consensus_queue.cc:260] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.458608  6723 raft_consensus.cc:3060] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:14.461586  5890 leader_election.cc:304] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
W20260709 13:08:14.427636  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 44185.1us
I20260709 13:08:14.314821  6736 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:14.436317  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.465531  5742 leader_election.cc:304] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:14.456487  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.474390  5923 raft_consensus.cc:2468] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:14.475767  5816 leader_election.cc:304] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:14.480075  5920 raft_consensus.cc:1275] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:14.480892  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f9da7df7034a9f99e973517496cd22" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:14.480892  6736 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:14.481716  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.482859  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:14.483062  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:14.483577  5922 raft_consensus.cc:3060] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:14.483681  5848 raft_consensus.cc:2393] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 2: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:14.484200  5693 raft_consensus.cc:1240] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:14.485280  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:14.485918  5848 raft_consensus.cc:3060] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:14.386114  5667 leader_election.cc:304] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
W20260709 13:08:14.489948  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 103905us
I20260709 13:08:14.490897  5972 tablet_bootstrap.cc:492] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
W20260709 13:08:14.493165  5667 leader_election.cc:343] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:14.498025  5842 raft_consensus.cc:1240] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:14.498729  6739 consensus_queue.cc:1059] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:14.451792  6699 raft_consensus.cc:493] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.469489  6540 raft_consensus.cc:493] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.472160  6748 raft_consensus.cc:2804] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:14.471909  6771 raft_consensus.cc:2804] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:14.507999  5667 leader_election.cc:400] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Vote denied by peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241) with higher term. Message: Invalid argument: T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 4. Current term is 5.
I20260709 13:08:14.499598  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:14.494154  6736 raft_consensus.cc:2749] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:14.512941  5813 leader_election.cc:304] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:14.515057  5667 leader_election.cc:403] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:14.496089  6227 raft_consensus.cc:2804] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.520334  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.520653  6227 raft_consensus.cc:697] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:14.521056  5774 raft_consensus.cc:2468] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:14.521543  6227 consensus_queue.cc:237] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.522490  5768 raft_consensus.cc:1240] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:14.472618  6402 raft_consensus.cc:2749] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:14.524062  6782 raft_consensus.cc:3055] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:14.524487  6782 raft_consensus.cc:740] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:14.499830  6741 consensus_queue.cc:1059] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:08:14.519035  6681 raft_consensus.cc:3060] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:08:14.508208  6699 raft_consensus.cc:3060] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:14.496533  5694 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:14.535096  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.505667  6698 consensus_queue.cc:1048] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 13:08:14.523680  6786 consensus_queue.cc:1059] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:08:14.536710  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892918414c974de38c7050ccd7151f94" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.542549  5843 raft_consensus.cc:2393] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 3: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:14.544310  5770 raft_consensus.cc:1240] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:14.529392  6771 raft_consensus.cc:697] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:14.544899  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.533658  6423 raft_consensus.cc:2804] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:14.545293  6402 raft_consensus.cc:3055] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:14.517643  6748 raft_consensus.cc:697] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 5 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:14.547370  5775 raft_consensus.cc:1275] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:14.537302  5597 catalog_manager.cc:5697] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 2 to 3, leader changed from 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 3 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } }
I20260709 13:08:14.514281  6540 raft_consensus.cc:515] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:14.563820  5891 leader_election.cc:304] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:14.563750  5813 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:14.545387  5696 raft_consensus.cc:2468] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:14.569502  6781 consensus_queue.cc:1048] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:14.540771  6648 consensus_queue.cc:1048] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:08:14.571489  6726 raft_consensus.cc:2804] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:14.567008  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.545392  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "422b96951eda4c1c8b8d86612ac6c163" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.580646  6771 consensus_queue.cc:237] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.581470  6540 leader_election.cc:290] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:14.566973  5697 raft_consensus.cc:1275] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:14.585448  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1280a1f51344b42b4410fc2852da38d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:14.555881  6402 raft_consensus.cc:740] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:14.587858  6539 raft_consensus.cc:2749] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.578168  6726 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:14.588331  6402 consensus_queue.cc:260] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:14.596140  5775 raft_consensus.cc:2468] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:14.589165  6726 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:14.599804  5891 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:14.564911  6423 raft_consensus.cc:697] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:14.565743  6564 consensus_queue.cc:1048] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 13:08:14.610734  5770 raft_consensus.cc:1275] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:14.612207  5699 raft_consensus.cc:1240] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:14.612118  6748 consensus_queue.cc:237] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:14.613318  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.616047  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1e618980cb143c69eb4b401ba701662" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.616454  5972 tablet_bootstrap.cc:654] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:14.616706  5775 raft_consensus.cc:2468] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 0.
I20260709 13:08:14.566938  6780 consensus_queue.cc:1048] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:08:14.621778  5775 raft_consensus.cc:1275] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:14.626093  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.627600  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.627905  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.602280  6539 raft_consensus.cc:2749] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:14.628974  5848 raft_consensus.cc:2468] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:14.629344  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440b21f4e5f94cf2a96ffb8b2a631d60" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.629547  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.630442  6743 consensus_queue.cc:1059] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:14.600503  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.630780  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.631268  5694 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:14.631522  6539 raft_consensus.cc:3055] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:14.631907  6539 raft_consensus.cc:740] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:14.632691  6539 consensus_queue.cc:260] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.628614  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddbd585b0264150a1fbd1b5cb8b5ebb" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.627319  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.636438  5699 raft_consensus.cc:1240] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Rejecting Update request from peer 792907f253c744aa9392e8d572f88e06 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:14.636668  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1e618980cb143c69eb4b401ba701662" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.637353  5769 raft_consensus.cc:3060] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.642445  5597 catalog_manager.cc:5697] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 1 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:14.648416  5922 raft_consensus.cc:2468] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:14.649156  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.649893  5816 leader_election.cc:304] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
W20260709 13:08:14.695344  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 45581.8us
I20260709 13:08:14.694826  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.651859  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.653453  5694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1280a1f51344b42b4410fc2852da38d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.653555  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:14.624260  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcc03c962c314adf95650bc3bf8e9d34" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.650504  6378 raft_consensus.cc:2749] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:14.647535  6775 raft_consensus.cc:493] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.641219  6402 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:14.661347  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.622786  6741 consensus_queue.cc:1059] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:08:14.627264  6423 consensus_queue.cc:237] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:14.667260  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:14.672353  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:14.679025  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:14.737444  5694 raft_consensus.cc:2393] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:14.725736  6734 raft_consensus.cc:493] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:14.740473  6734 raft_consensus.cc:515] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:14.741529  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:14.742225  5847 raft_consensus.cc:3060] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.727339  6775 raft_consensus.cc:515] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:14.745221  6775 leader_election.cc:290] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:14.730762  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e78f9ad8be2a4dd6bf2aa1d922e471a2" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.682643  5888 leader_election.cc:304] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:14.677939  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.703512  6652 raft_consensus.cc:2804] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:14.659754  6726 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:14.702028  5773 raft_consensus.cc:2393] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 4: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:14.698206  5776 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 4.
I20260709 13:08:14.724483  5848 raft_consensus.cc:1275] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:14.761660  6726 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:14.761989  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.699822  5775 raft_consensus.cc:2376] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3. Re-sending same reply.
I20260709 13:08:14.697265  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:14.690689  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:14.700973  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.784096  6740 raft_consensus.cc:3055] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:14.790093  6740 raft_consensus.cc:740] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:14.791121  6740 consensus_queue.cc:260] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:14.836393  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:14.835932  5597 catalog_manager.cc:5697] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } }
W20260709 13:08:14.752609  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 70091.5us
I20260709 13:08:14.837743  5888 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:14.839215  5888 leader_election.cc:304] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:14.690373  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.704385  5667 leader_election.cc:304] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:14.759326  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.841379  5776 raft_consensus.cc:2468] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
W20260709 13:08:14.841420  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 137171us
I20260709 13:08:14.842269  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.737515  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.759857  5770 raft_consensus.cc:2376] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1. Re-sending same reply.
I20260709 13:08:14.765312  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.863891  5775 raft_consensus.cc:2368] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 4. Current term is 5.
I20260709 13:08:14.774003  6734 leader_election.cc:290] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:14.862892  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:14.873677  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:14.775043  6539 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:14.794023  5814 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.786065  5847 raft_consensus.cc:2468] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 1.
I20260709 13:08:14.876963  5847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:14.856880  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.787774  6699 raft_consensus.cc:515] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.879678  6699 leader_election.cc:290] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:14.804407  6652 raft_consensus.cc:697] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:14.757892  6707 raft_consensus.cc:2749] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:14.862490  6782 consensus_queue.cc:260] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:14.881006  5972 tablet_bootstrap.cc:492] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:14.881851  6782 raft_consensus.cc:3060] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:14.881942  5972 ts_tablet_manager.cc:1403] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.494s	user 0.023s	sys 0.024s
I20260709 13:08:14.882704  5843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:14.813406  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa162ebfa5094c25a3a311044956d167" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:14.814590  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:14.821297  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.822671  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa162ebfa5094c25a3a311044956d167" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.823678  6648 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.825543  5918 raft_consensus.cc:1240] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:14.826602  5924 raft_consensus.cc:1240] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:14.829062  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:14.829998  5848 raft_consensus.cc:3060] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:14.697497  5921 raft_consensus.cc:1240] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 1. Current term is 2. Ops: []
W20260709 13:08:14.833590  5739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.833819  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1e618980cb143c69eb4b401ba701662" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:14.739495  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77f29b60a32f4f79a3f1bd59278b00bd" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:14.843803  5597 catalog_manager.cc:5697] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 2 to 3, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 3 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:14.909400  5597 catalog_manager.cc:5697] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 2, leader changed from 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } }
I20260709 13:08:14.910378  5597 catalog_manager.cc:5697] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 2 to 3, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 3 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } }
I20260709 13:08:14.845547  5888 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:14.846375  6687 raft_consensus.cc:2749] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:14.857710  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.859354  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.860339  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1e618980cb143c69eb4b401ba701662" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.860399  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.861369  5701 raft_consensus.cc:1275] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:14.784509  5769 raft_consensus.cc:2468] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:14.851714  5598 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39344 (request call id 72) took 7101 ms. Trace:
W20260709 13:08:14.876770  5815 leader_election.cc:336] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Timed out: RequestConsensusVote RPC to 127.27.71.130:39429 timed out after 1.847s (SENT)
I20260709 13:08:14.812140  6681 raft_consensus.cc:2749] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241) with higher term. Message: Invalid argument: T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 4. Current term is 5.
I20260709 13:08:14.884994  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:14.809044  5891 leader_election.cc:304] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:14.891311  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:14.863821  6540 raft_consensus.cc:2804] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:08:14.924558  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 115609us
I20260709 13:08:14.865085  6378 raft_consensus.cc:2749] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:14.884724  6740 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:14.912843  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 67399.9us
I20260709 13:08:14.911298  5597 catalog_manager.cc:5697] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 0 to 2, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } }
I20260709 13:08:14.914728  5972 raft_consensus.cc:359] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:14.901583  5695 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:14.918613  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ba1e6d2a3344631b647a567e5280513" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.920326  5815 leader_election.cc:304] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:14.925850  5925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:14.915697  5849 raft_consensus.cc:2393] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 4: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:14.931417  5888 leader_election.cc:304] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:14.932512  5597 catalog_manager.cc:5697] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } }
I20260709 13:08:14.934357  5972 raft_consensus.cc:385] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:14.934762  5972 raft_consensus.cc:740] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
I20260709 13:08:14.934307  5597 catalog_manager.cc:5697] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 3, leader changed from 792907f253c744aa9392e8d572f88e06 (127.27.71.132) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 3 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:14.935370  5813 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.918257  6707 raft_consensus.cc:2804] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:14.903604  6648 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:14.881094  6652 consensus_queue.cc:237] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:14.943245  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "106cdfafda414cf6981dbcaf1e28cc9d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.935521  5972 consensus_queue.cc:260] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:14.940462  5739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.918712  5768 raft_consensus.cc:2376] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3. Re-sending same reply.
I20260709 13:08:14.962149  5922 raft_consensus.cc:2468] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 4.
I20260709 13:08:14.924688  6540 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:14.939997  5597 catalog_manager.cc:5697] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 1 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:14.931759  6748 raft_consensus.cc:2804] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:14.972025  6564 raft_consensus.cc:2749] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:14.918618  6766 consensus_queue.cc:1048] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:08:14.975680  5597 catalog_manager.cc:5697] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 2, leader changed from 792907f253c744aa9392e8d572f88e06 (127.27.71.132) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } }
I20260709 13:08:14.920066  5598 rpcz_store.cc:276] 0709 13:08:07.750213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:07.752103 (+  1890us) service_pool.cc:224] Handling call
0709 13:08:14.851626 (+7099523us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:13.617298 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:13.618862 (+  1564us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:13.621288 (+  2426us) write_op.cc:435] Acquired schema lock
0709 13:08:13.630533 (+  9245us) tablet.cc:662] Decoding operations
0709 13:08:13.809984 (+179451us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:13.822999 (+ 13015us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:13.825323 (+  2324us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:08:13.893770 (+ 68447us) tablet.cc:701] Row locks acquired
0709 13:08:13.895840 (+  2070us) write_op.cc:260] PREPARE: finished
0709 13:08:13.903981 (+  8141us) write_op.cc:270] Start()
0709 13:08:13.936922 (+ 32941us) write_op.cc:276] Timestamp: P: 1783602493903897 usec, L: 0
0709 13:08:13.939335 (+  2413us) op_driver.cc:348] REPLICATION: starting
0709 13:08:13.968678 (+ 29343us) log.cc:844] Serialized 1705 byte log entry
0709 13:08:14.213644 (+244966us) op_driver.cc:464] REPLICATION: finished
0709 13:08:14.291375 (+ 77731us) write_op.cc:301] APPLY: starting
0709 13:08:14.291498 (+   123us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:14.291699 (+   201us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:14.291875 (+   176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:14.292585 (+   710us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:14.292846 (+   261us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:08:14.294307 (+  1461us) write_op.cc:312] APPLY: finished
0709 13:08:14.348593 (+ 54286us) log.cc:844] Serialized 40 byte log entry
0709 13:08:14.357710 (+  9117us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:14.359631 (+  1921us) write_op.cc:454] Released schema lock
0709 13:08:14.382342 (+ 22711us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:14.422069 (+ 39727us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:14.422104 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16266,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":77309,"apply.run_cpu_time_us":3342,"apply.run_wall_time_us":105358,"num_ops":4,"prepare.queue_time_us":102467,"prepare.run_cpu_time_us":5780,"prepare.run_wall_time_us":351726,"replication_time_us":273977,"thread_start_us":183672,"threads_started":2}]]}
I20260709 13:08:14.961546  5916 raft_consensus.cc:2393] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 1: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:14.940171  6326 raft_consensus.cc:2749] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:14.940745  6707 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:14.958525  5843 raft_consensus.cc:1240] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:15.003142  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:14.940270  6744 consensus_queue.cc:1048] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:08:14.978307  6540 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.019567  5916 raft_consensus.cc:1275] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:14.998178  6707 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:14.976744  6743 consensus_queue.cc:1059] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:08:14.971747  6786 consensus_queue.cc:1048] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:08:15.013043  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:15.013633  6748 raft_consensus.cc:697] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:15.039844  5845 raft_consensus.cc:1240] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:15.051056  5845 raft_consensus.cc:1240] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 2. Current term is 3. Ops: []
W20260709 13:08:15.051697  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 174997us
I20260709 13:08:15.055631  6564 consensus_queue.cc:1059] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:15.056298  5844 raft_consensus.cc:2468] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:15.056437  6564 consensus_queue.cc:1243] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Peer 373ee2f4cc9d4b86b49ada724f55e16a log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:08:15.058637  5815 leader_election.cc:304] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:15.047616  6807 log.cc:927] Time spent T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:08:15.062734  5742 leader_election.cc:304] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.082566  6423 raft_consensus.cc:2749] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.082338  6748 consensus_queue.cc:237] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:15.062734  5891 leader_election.cc:304] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.090981  5742 leader_election.cc:304] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.095736  6689 raft_consensus.cc:2749] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:15.097909  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 136967us
I20260709 13:08:15.099287  5815 leader_election.cc:304] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.111582  5742 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.112454  5815 leader_election.cc:304] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.113183  5742 leader_election.cc:304] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.114413  5815 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.121671  6739 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:15.122215  6227 raft_consensus.cc:2749] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.122987  6698 raft_consensus.cc:2749] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.123715  6812 raft_consensus.cc:2749] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.125483  5815 leader_election.cc:304] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:15.126183  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 63502.3us
I20260709 13:08:15.130929  5815 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.131197  5742 leader_election.cc:304] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.132795  5742 leader_election.cc:304] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.061542  6699 consensus_queue.cc:1048] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:08:15.136986  5815 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
W20260709 13:08:15.141140  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 62131us
I20260709 13:08:15.142881  5972 ts_tablet_manager.cc:1434] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.260s	user 0.027s	sys 0.027s
I20260709 13:08:15.130366  6746 raft_consensus.cc:493] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.130038  6723 raft_consensus.cc:2749] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.135247  6813 raft_consensus.cc:2749] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.134048  6817 raft_consensus.cc:3060] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.152694  5850 raft_consensus.cc:1240] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:15.139827  6423 raft_consensus.cc:2749] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.141916  6776 raft_consensus.cc:2749] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:15.142488  6689 raft_consensus.cc:2749] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.148012  6452 raft_consensus.cc:2749] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.161427  5972 tablet_bootstrap.cc:492] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06: Bootstrap starting.
I20260709 13:08:15.141546  6698 consensus_queue.cc:1048] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:08:15.152191  6746 raft_consensus.cc:515] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:15.176070  5740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.177783  6746 leader_election.cc:290] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:15.152737  6814 consensus_queue.cc:1059] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:08:15.199662  5771 raft_consensus.cc:1275] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:15.199759  6816 raft_consensus.cc:3055] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:15.210029  6746 raft_consensus.cc:493] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:15.226285  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e83357d1cc74a0292e36bd8947af234" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:08:15.240334  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 105500us
I20260709 13:08:15.230443  6816 raft_consensus.cc:740] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:15.247844  6812 consensus_queue.cc:1059] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:08:15.255441  6402 raft_consensus.cc:3055] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:15.255846  6402 raft_consensus.cc:740] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:15.256187  5768 raft_consensus.cc:1275] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.256768  6402 consensus_queue.cc:260] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:15.258033  6402 raft_consensus.cc:3060] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:15.259595  5767 raft_consensus.cc:2468] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:15.261008  6772 raft_consensus.cc:3055] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:15.261439  6772 raft_consensus.cc:740] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:15.262254  6772 consensus_queue.cc:260] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:15.265097  6816 consensus_queue.cc:260] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:15.266386  6816 raft_consensus.cc:3060] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:15.278250  5776 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:15.240517  6746 raft_consensus.cc:515] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:15.280359  6423 raft_consensus.cc:493] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:15.296527  6423 raft_consensus.cc:515] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:15.297665  6746 leader_election.cc:290] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:15.298710  6423 leader_election.cc:290] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:15.281692  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 343648us
I20260709 13:08:15.286232  5849 raft_consensus.cc:1275] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.300863  5667 leader_election.cc:304] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:15.301873  6648 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:15.304122  6681 raft_consensus.cc:2804] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:15.304697  6681 raft_consensus.cc:697] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:15.305619  6681 consensus_queue.cc:237] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:15.308983  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:15.309312  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e83357d1cc74a0292e36bd8947af234" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:15.309988  5692 raft_consensus.cc:1275] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:15.279801  5972 tablet_bootstrap.cc:654] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:15.305835  5667 leader_election.cc:343] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:15.318293  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:15.319903  5667 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.287333  6786 raft_consensus.cc:493] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.290642  6707 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:15.244918  5742 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:15.332885  6771 consensus_queue.cc:1048] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:15.337224  6540 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:15.337407  6781 consensus_queue.cc:1048] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:15.348987  5667 leader_election.cc:304] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.273855  5695 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 4.
W20260709 13:08:15.358114  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 113301us
I20260709 13:08:15.365540  5742 leader_election.cc:304] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.351118  6817 raft_consensus.cc:2749] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:15.357509  6784 raft_consensus.cc:2749] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:08:15.451107  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 77352.8us
I20260709 13:08:15.361377  6782 raft_consensus.cc:2804] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:08:15.458356  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 100261us
I20260709 13:08:15.349928  6652 consensus_queue.cc:1048] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.100s
I20260709 13:08:15.447409  6600 raft_consensus.cc:2749] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:15.351289  6786 raft_consensus.cc:515] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:15.454341  6402 raft_consensus.cc:2749] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.461057  6648 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:15.390136  6378 consensus_queue.cc:1048] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260709 13:08:15.470054  5694 raft_consensus.cc:1240] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:08:15.470098  6782 raft_consensus.cc:697] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:15.497738  6707 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:15.498548  5972 tablet_bootstrap.cc:492] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06: No bootstrap required, opened a new log
I20260709 13:08:15.499212  5972 ts_tablet_manager.cc:1403] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06: Time spent bootstrapping tablet: real 0.338s	user 0.014s	sys 0.002s
I20260709 13:08:15.501966  5888 leader_election.cc:304] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.505137  6538 raft_consensus.cc:2749] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:15.505937  6538 consensus_queue.cc:1048] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:15.476104  6227 raft_consensus.cc:2749] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:08:15.524574  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 159119us
I20260709 13:08:15.526561  5742 leader_election.cc:304] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:15.529302  6832 log.cc:927] Time spent T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:08:15.529925  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:15.527745  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:15.489496  6648 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:15.531374  6782 consensus_queue.cc:237] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:15.533510  6648 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:15.530141  6740 consensus_meta.cc:287] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.000s
I20260709 13:08:15.535907  5888 leader_election.cc:304] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.513793  6739 raft_consensus.cc:2749] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:15.573159  5972 raft_consensus.cc:359] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:15.550865  6326 raft_consensus.cc:2749] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.574476  6540 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:15.582150  5699 consensus_meta.cc:287] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: Time spent flushing consensus metadata: real 0.610s	user 0.012s	sys 0.007s
I20260709 13:08:15.577617  6712 raft_consensus.cc:493] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:15.591255  5972 raft_consensus.cc:385] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:08:15.597538  5815 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.604745  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 78245.7us
I20260709 13:08:15.623845  5972 raft_consensus.cc:740] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Initialized, Role: FOLLOWER
W20260709 13:08:15.646628  5739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.649611  5972 consensus_queue.cc:260] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:15.662211  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa162ebfa5094c25a3a311044956d167" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:15.604530  6739 raft_consensus.cc:2749] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.665308  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:15.666640  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:15.669332  5666 leader_election.cc:304] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:15.675078  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:15.675850  5742 leader_election.cc:304] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.676193  5972 ts_tablet_manager.cc:1434] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06: Time spent starting tablet: real 0.177s	user 0.011s	sys 0.000s
I20260709 13:08:15.647367  5699 raft_consensus.cc:2468] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 5.
I20260709 13:08:15.679823  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e47f79b0ad84696b05ddc28ce9bef20" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:15.680141  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:15.680719  5775 raft_consensus.cc:2368] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 4. Current term is 5.
I20260709 13:08:15.682873  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa162ebfa5094c25a3a311044956d167" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:15.685125  5815 leader_election.cc:304] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.686185  5815 leader_election.cc:304] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.687692  6829 consensus_queue.cc:1048] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:08:15.694619  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
W20260709 13:08:15.695281  5815 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.695907  6652 raft_consensus.cc:2749] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:15.699471  6837 raft_consensus.cc:2749] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:15.705448  6786 leader_election.cc:290] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:15.705802  6772 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:15.708555  5922 raft_consensus.cc:1240] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:15.709770  5925 raft_consensus.cc:1275] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.660676  6712 raft_consensus.cc:515] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:15.712105  6652 consensus_queue.cc:1048] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:15.712634  6712 leader_election.cc:290] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:15.699559  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:15.699549  5696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:15.701706  6836 raft_consensus.cc:2804] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:08:15.741575  5666 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 72306.9us
I20260709 13:08:15.742594  5666 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.745550  6707 raft_consensus.cc:493] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.752486  6707 raft_consensus.cc:515] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:15.754846  6707 leader_election.cc:290] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:15.746371  6600 raft_consensus.cc:2749] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.734757  6836 raft_consensus.cc:493] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:15.758112  5890 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.766582  6839 raft_consensus.cc:2749] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.765626  6707 raft_consensus.cc:2749] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:15.762565  6836 raft_consensus.cc:3060] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:15.779659  5666 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.784155  6839 raft_consensus.cc:2804] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:08:15.789352  5741 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:15.791692  5740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.798271  6839 raft_consensus.cc:697] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:15.803781  6839 consensus_queue.cc:237] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:15.814978  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:15.818167  5698 raft_consensus.cc:1275] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:15.829389  5849 raft_consensus.cc:1275] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:15.830574  5850 raft_consensus.cc:2393] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 1: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
W20260709 13:08:15.842387  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 166610us
I20260709 13:08:15.845074  5742 leader_election.cc:304] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.810817  6772 consensus_queue.cc:1048] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
W20260709 13:08:15.916280  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 73171.1us
W20260709 13:08:15.918596  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 73649.8us
I20260709 13:08:15.919811  5742 leader_election.cc:304] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:15.927225  6744 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:15.927860  6744 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:15.929963  6766 raft_consensus.cc:2749] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:15.930055  6744 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:15.811545  6813 consensus_queue.cc:1059] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.105s
I20260709 13:08:15.939219  6813 raft_consensus.cc:3055] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:15.939639  6813 raft_consensus.cc:740] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:15.940518  6813 consensus_queue.cc:260] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:15.946305  6452 raft_consensus.cc:2749] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.946962  5742 leader_election.cc:304] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.947281  6452 consensus_queue.cc:1059] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:15.947826  6739 raft_consensus.cc:2749] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.953776  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:15.935992  6712 consensus_queue.cc:1048] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:08:15.844463  5699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44362 (request call id 145) took 1194 ms. Trace:
I20260709 13:08:15.955849  5699 rpcz_store.cc:276] 0709 13:08:14.648724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.650364 (+  1640us) service_pool.cc:224] Handling call
0709 13:08:15.816107 (+1165743us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400734454. stack: 00007f78339ee343 00007f78339ee240 00007f783337d1d4 00007f783d0bb304 00007f783d0bb299 00007f7836a76571 00007f7836b112fa 00007f783bad2b17 00007f783504d6d4 00007f783504d662 00007f783504d591 00007f783504d50c 00007f783504c7c2 00007f7834e4848c 00007f7834e478c6 00007f7834e4726f
0709 13:08:15.842726 (+ 26619us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507678208}
I20260709 13:08:15.964061  5815 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:15.964128  6852 consensus_queue.cc:1059] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:08:15.966987  6699 raft_consensus.cc:2749] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:15.967192  6852 consensus_queue.cc:1243] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer 52b726da57c24dd2aa72488ebe2ff21e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:08:15.970081  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:15.970782  5851 raft_consensus.cc:2468] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:15.972146  5666 leader_election.cc:304] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:15.982933  5666 leader_election.cc:400] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Vote denied by peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429) with higher term. Message: Invalid argument: T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 4. Current term is 5.
I20260709 13:08:15.983235  5920 raft_consensus.cc:3060] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:15.987565  5925 raft_consensus.cc:1240] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:15.989476  5923 raft_consensus.cc:1240] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Rejecting Update request from peer 373ee2f4cc9d4b86b49ada724f55e16a for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:15.987828  6687 raft_consensus.cc:2749] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.004750  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75595b8e97ef436992a8acba2c9724a7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:15.958252  6836 raft_consensus.cc:515] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:16.000813  5924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:16.017431  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:16.018185  5776 raft_consensus.cc:2393] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792907f253c744aa9392e8d572f88e06 in current term 3: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:16.018806  5666 leader_election.cc:304] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:16.020058  5773 raft_consensus.cc:3055] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:16.020444  5773 raft_consensus.cc:740] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:16.023206  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:16.024511  5924 raft_consensus.cc:2468] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:16.026582  5771 raft_consensus.cc:1275] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:16.001703  6452 consensus_queue.cc:1059] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:08:16.029934  6452 raft_consensus.cc:3055] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:16.030103  6837 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.030367  6452 raft_consensus.cc:740] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:16.030714  6837 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:16.033129  6837 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:16.026696  5774 raft_consensus.cc:1240] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Rejecting Update request from peer 52b726da57c24dd2aa72488ebe2ff21e for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:08:16.035228  5775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:16.021462  6681 raft_consensus.cc:2749] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:16.021462  5742 leader_election.cc:304] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:16.061945  6712 consensus_queue.cc:1048] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:08:16.062171  6814 raft_consensus.cc:2804] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:16.085110  6814 raft_consensus.cc:493] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.085485  6814 raft_consensus.cc:3060] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.089087  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:16.090240  6836 leader_election.cc:290] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:16.014647  6766 consensus_queue.cc:1059] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 2, Time since last communication: 0.078s
W20260709 13:08:16.089125  5815 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.012888  6846 consensus_queue.cc:1059] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:08:16.096393  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd06784a6f3244309ebdd41cf8127741" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:16.031421  6452 consensus_queue.cc:260] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:16.104094  6813 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:16.104553  5699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:16.106902  5666 leader_election.cc:304] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:16.108896  6846 raft_consensus.cc:3055] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:16.109301  6846 raft_consensus.cc:740] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:16.109766  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:16.110368  6687 raft_consensus.cc:2749] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.107285  5773 consensus_queue.cc:260] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:16.110718  6855 log.cc:927] Time spent T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: Append to log took a long time: real 0.188s	user 0.004s	sys 0.017s
I20260709 13:08:16.112516  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:16.116259  5849 raft_consensus.cc:2468] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:16.116561  6227 consensus_queue.cc:1048] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:16.119099  5917 raft_consensus.cc:3060] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:16.113304  5773 raft_consensus.cc:3060] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.124205  5888 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:16.124554  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f81180a2c94fe5874f1a402e44c52f" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:16.113878  6452 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:16.127740  6784 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:16.128345  6784 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:16.130761  6784 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:16.136029  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:16.135831  6846 consensus_queue.cc:260] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:16.137270  6846 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.137781  6837 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.138296  6837 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:16.141458  5742 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:16.143319  6812 raft_consensus.cc:2804] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:16.143754  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:16.143734  6812 raft_consensus.cc:493] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.145167  6812 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:16.147027  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:16.147954  5695 raft_consensus.cc:2410] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:08:16.149806  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:16.150492  5923 raft_consensus.cc:2468] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:16.152817  5666 leader_election.cc:304] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:16.153435  5891 leader_election.cc:304] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:16.153937  5813 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:16.154222  6848 raft_consensus.cc:2749] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.156359  6698 raft_consensus.cc:2804] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:16.157100  6698 raft_consensus.cc:697] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:16.157193  6842 consensus_queue.cc:1048] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:16.158335  6698 consensus_queue.cc:237] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:16.160624  5815 leader_election.cc:304] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:16.162797  6772 raft_consensus.cc:2749] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:16.163520  6837 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:16.164832  6785 consensus_queue.cc:1048] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:08:16.167824  6854 log.cc:927] Time spent T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.242s	user 0.001s	sys 0.004s
I20260709 13:08:16.176947  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:16.177811  5849 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 5.
I20260709 13:08:16.179580  5700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deecf916d19d4f5ca9ac8200dc931c5c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:16.180546  5700 raft_consensus.cc:2393] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:16.155750  5813 leader_election.cc:304] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:16.187731  6227 raft_consensus.cc:2749] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:16.196504  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 72368.3us
I20260709 13:08:16.197588  5813 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:16.199831  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 46483.2us
I20260709 13:08:16.198599  5888 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:16.213681  5773 raft_consensus.cc:1275] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:16.198989  6772 raft_consensus.cc:2749] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.218197  5849 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:16.218788  5923 raft_consensus.cc:1275] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:08:16.228360  5888 leader_election.cc:304] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:16.229120  5849 raft_consensus.cc:2410] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:08:16.230571  5667 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:16.231907  5850 raft_consensus.cc:1275] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:08:16.197506  6816 raft_consensus.cc:2749] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:08:16.238493  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:16.223790  6712 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:16.224050  6689 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:08:16.215777  6771 raft_consensus.cc:2749] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.236143  6848 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:16.215240  6781 consensus_queue.cc:1048] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:08:16.285207  5774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:16.305101  5774 raft_consensus.cc:2393] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 5: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
W20260709 13:08:16.305311  6861 log.cc:927] Time spent T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:08:16.239343  6814 raft_consensus.cc:515] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:16.287868  6848 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:16.275475  6772 consensus_queue.cc:1048] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
W20260709 13:08:16.346369  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 64672.6us
I20260709 13:08:16.347802  5742 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:16.342087  6848 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:16.400779  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 44016.8us
W20260709 13:08:16.402776  5891 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 142004us
I20260709 13:08:16.337029  6841 consensus_queue.cc:1048] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:08:16.413492  6739 raft_consensus.cc:2749] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:16.438937  6781 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.452452  5769 raft_consensus.cc:2468] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:16.462602  6786 consensus_queue.cc:1048] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:08:16.478017  6814 leader_election.cc:290] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:16.473583  6781 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:16.483142  6781 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:16.527329  6868 log.cc:927] Time spent T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:08:16.540061  6744 raft_consensus.cc:2749] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:16.590159  6812 raft_consensus.cc:515] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:16.596226  6812 leader_election.cc:290] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:16.601649  6687 consensus_queue.cc:1048] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:16.655368  6726 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:16.656141  6726 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:16.659686  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:16.660514  5917 raft_consensus.cc:2468] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:16.662024  5816 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:16.662860  6698 raft_consensus.cc:2804] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:16.663231  6698 raft_consensus.cc:493] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:16.663561  6698 raft_consensus.cc:3060] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:16.664793  6848 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:16.669593  6712 raft_consensus.cc:2749] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.672282  5922 raft_consensus.cc:1275] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:08:16.680260  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 270110us
I20260709 13:08:16.676980  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:16.672570  6726 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:16.672277  6848 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:16.688014  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:16.731592  5701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:16.680488  6595 consensus_queue.cc:1048] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:08:16.688968  5773 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 5.
I20260709 13:08:16.732450  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:16.733143  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "057217b46a0d4d188d28fecbdf744762" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:16.734711  5815 leader_election.cc:304] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:16.737218  5701 raft_consensus.cc:2393] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:16.737843  5922 raft_consensus.cc:3060] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.737669  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:16.738673  5773 raft_consensus.cc:2393] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 5: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:16.737260  5850 raft_consensus.cc:3060] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.741717  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:16.742450  5923 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:16.682008  6813 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:16.769025  5848 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:16.740811  5775 raft_consensus.cc:1275] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:16.769729  5848 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 5.
I20260709 13:08:16.779585  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffd891d1488c409d87be1300b3e91325" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:16.780308  5845 raft_consensus.cc:3060] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:16.788256  5923 raft_consensus.cc:2468] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 4.
I20260709 13:08:16.772332  6842 raft_consensus.cc:2804] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:16.809167  5890 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:16.812618  5742 leader_election.cc:304] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:16.807822  5922 raft_consensus.cc:2468] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:16.818265  5775 raft_consensus.cc:1275] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:08:16.818739  5775 pending_rounds.cc:85] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a: Aborting all ops after (but not including) 1
I20260709 13:08:16.819000  5775 pending_rounds.cc:107] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:08:16.819291  5775 raft_consensus.cc:2889] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:16.784754  6698 raft_consensus.cc:515] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:16.821805  6739 raft_consensus.cc:2804] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:16.821930  6698 leader_election.cc:290] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:16.822410  6739 raft_consensus.cc:697] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:16.823406  6739 consensus_queue.cc:237] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:16.825716  6848 consensus_queue.cc:1048] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:16.830219  5850 raft_consensus.cc:2468] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:16.845440  6877 raft_consensus.cc:2804] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:16.846024  6877 raft_consensus.cc:493] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:16.846196  6880 log.cc:927] Time spent T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:08:16.846498  6877 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:16.842391  5742 leader_election.cc:304] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:16.848035  6882 raft_consensus.cc:2804] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:16.848640  6882 raft_consensus.cc:697] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:16.776082  6813 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:16.849587  6882 consensus_queue.cc:237] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:16.851404  6813 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:16.834566  6842 raft_consensus.cc:697] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:16.853071  6842 consensus_queue.cc:237] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:16.831368  6595 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:16.855660  6595 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:16.856186  5698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:16.857806  6595 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:16.859730  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:16.870862  5597 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39354 (request call id 57) took 3334 ms. Trace:
I20260709 13:08:16.874521  6600 raft_consensus.cc:493] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:16.878245  5597 rpcz_store.cc:276] 0709 13:08:13.536146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:13.540108 (+  3962us) service_pool.cc:224] Handling call
0709 13:08:16.870786 (+3330678us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:15.179338 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:15.182056 (+  2718us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:15.182100 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:08:15.182120 (+    20us) tablet.cc:662] Decoding operations
0709 13:08:15.183313 (+  1193us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:15.183378 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:15.183406 (+    28us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:08:15.184466 (+  1060us) tablet.cc:701] Row locks acquired
0709 13:08:15.184491 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:08:15.184646 (+   155us) write_op.cc:270] Start()
0709 13:08:15.197267 (+ 12621us) write_op.cc:276] Timestamp: P: 1783602495184600 usec, L: 0
0709 13:08:15.198280 (+  1013us) op_driver.cc:348] REPLICATION: starting
0709 13:08:15.260193 (+ 61913us) log.cc:844] Serialized 4421 byte log entry
0709 13:08:15.554255 (+294062us) op_driver.cc:464] REPLICATION: finished
0709 13:08:15.749702 (+195447us) write_op.cc:301] APPLY: starting
0709 13:08:15.759255 (+  9553us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:15.770035 (+ 10780us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:15.770181 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:15.771432 (+  1251us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:15.771751 (+   319us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:08:15.771816 (+    65us) write_op.cc:312] APPLY: finished
0709 13:08:15.785038 (+ 13222us) log.cc:844] Serialized 82 byte log entry
0709 13:08:15.788070 (+  3032us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:15.788222 (+   152us) write_op.cc:454] Released schema lock
0709 13:08:15.788280 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:15.788669 (+   389us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:15.788709 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":288757,"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":180250,"apply.run_cpu_time_us":3686,"apply.run_wall_time_us":42582,"num_ops":11,"prepare.queue_time_us":96610,"prepare.run_cpu_time_us":5376,"prepare.run_wall_time_us":81184,"replication_time_us":328255,"thread_start_us":149297,"threads_started":2}]]}
I20260709 13:08:16.878435  5923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:16.885769  5923 raft_consensus.cc:2393] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 4: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:16.908417  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:16.887197  6699 consensus_queue.cc:1048] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:08:16.884310  6600 raft_consensus.cc:515] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:16.918190  6600 leader_election.cc:290] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:16.921545  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:16.922638  5773 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 4.
I20260709 13:08:16.865072  6877 raft_consensus.cc:515] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:16.930663  5922 raft_consensus.cc:2468] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:16.941715  5816 leader_election.cc:304] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:16.949275  6227 raft_consensus.cc:2804] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:16.949911  6227 raft_consensus.cc:697] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:16.950914  5742 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:16.953850  5666 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:16.956727  5845 raft_consensus.cc:2468] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 4.
I20260709 13:08:16.958866  5773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:16.957036  6227 consensus_queue.cc:237] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
W20260709 13:08:16.970168  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 67633.8us
W20260709 13:08:16.976387  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 78757.4us
I20260709 13:08:16.964267  6595 raft_consensus.cc:2804] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:16.963251  6766 raft_consensus.cc:2749] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.982014  6595 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:17.008175  6785 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.016672  6877 leader_election.cc:290] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 6 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:17.019806  5770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:17.020558  5770 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:17.005172  6595 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:17.026571  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee9614d828d4ea8ab18a900dbc063c1" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:17.029805  5692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9dfeaefea774ae19834e0a619330eb5" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
W20260709 13:08:17.076501  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 81971.6us
I20260709 13:08:17.078372  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674fd655eaf14b10bc2e013c27c40557" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:17.079034  5851 raft_consensus.cc:3060] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:17.011698  6785 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:17.089862  5667 leader_election.cc:304] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:17.085493  5692 raft_consensus.cc:2393] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate d9952df48cf245608db9181dedad0aeb in this term.
I20260709 13:08:17.133653  6681 raft_consensus.cc:2749] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:17.162055  5697 raft_consensus.cc:1275] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:08:17.171340  5846 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 5.
I20260709 13:08:17.172104  6890 consensus_queue.cc:1048] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:17.172861  6785 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:17.202735  6898 log.cc:927] Time spent T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 13:08:17.239521  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 148436us
I20260709 13:08:17.268678  5842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:17.281029  5770 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 6.
I20260709 13:08:17.283215  5890 leader_election.cc:304] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:17.287156  5851 raft_consensus.cc:2468] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 6.
I20260709 13:08:17.289264  5667 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:17.291602  6876 raft_consensus.cc:2804] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:17.299311  6785 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:17.313953  6785 raft_consensus.cc:697] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 5 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:17.314901  6785 consensus_queue.cc:237] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:17.316910  6876 raft_consensus.cc:697] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 6 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:17.317840  6876 consensus_queue.cc:237] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:17.292119  6595 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:17.321959  6595 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:17.361168  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:17.401865  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:17.409627  5666 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:17.409967  5844 raft_consensus.cc:1275] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:17.413403  6784 raft_consensus.cc:2749] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:17.381155  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:17.439003  5775 raft_consensus.cc:1275] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:08:17.465904  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 82716.1us
W20260709 13:08:17.458189  6893 log.cc:927] Time spent T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:08:17.461383  6707 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.449131  6812 consensus_queue.cc:1048] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:08:17.464833  6883 consensus_queue.cc:1048] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:08:17.483839  5694 raft_consensus.cc:1275] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:17.481801  6707 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:17.496608  6707 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:17.497933  5920 raft_consensus.cc:1275] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:17.500142  6652 consensus_queue.cc:1048] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:17.491475  6890 consensus_queue.cc:1048] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:08:17.505483  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
W20260709 13:08:17.506065  6911 log.cc:927] Time spent T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:08:17.506462  5921 raft_consensus.cc:2393] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 4: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:17.535077  5695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:17.535843  5695 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 4.
I20260709 13:08:17.539047  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:17.546598  5888 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:17.548300  6712 raft_consensus.cc:2804] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:17.548728  6712 raft_consensus.cc:493] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.549118  6712 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:17.577596  5693 raft_consensus.cc:1275] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:17.533907  5772 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 4.
I20260709 13:08:17.596028  5922 raft_consensus.cc:1275] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:17.595206  5596 catalog_manager.cc:5697] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 3, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 3 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } }
I20260709 13:08:17.599450  5596 catalog_manager.cc:5697] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:17.612727  5596 catalog_manager.cc:5697] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 1 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } }
I20260709 13:08:17.630373  5596 catalog_manager.cc:5697] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 3, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 3 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:17.618990  5666 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 792907f253c744aa9392e8d572f88e06
I20260709 13:08:17.620992  6227 consensus_queue.cc:1048] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:17.623008  5916 raft_consensus.cc:1275] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:17.625263  5846 raft_consensus.cc:1275] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:17.629063  6813 consensus_queue.cc:1048] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:17.605764  6712 raft_consensus.cc:515] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:17.643759  6600 raft_consensus.cc:2804] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:17.648064  5596 catalog_manager.cc:5697] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } }
I20260709 13:08:17.650652  5596 catalog_manager.cc:5697] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 4, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 4 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } }
I20260709 13:08:17.652067  6712 leader_election.cc:290] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:17.654436  5596 catalog_manager.cc:5697] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 2, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 2 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } }
I20260709 13:08:17.656265  5697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:17.657244  6600 raft_consensus.cc:697] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 4 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:17.656639  5596 catalog_manager.cc:5697] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 1 to 3, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 3 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } }
I20260709 13:08:17.658253  6600 consensus_queue.cc:237] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:17.658288  5596 catalog_manager.cc:5697] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } }
I20260709 13:08:17.660188  5596 catalog_manager.cc:5697] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 1 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } }
I20260709 13:08:17.661852  5596 catalog_manager.cc:5697] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } }
I20260709 13:08:17.663571  6914 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:17.665634  6914 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:17.665931  5596 catalog_manager.cc:5697] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 0 to 1, leader changed from <none> to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 1 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:17.668139  6813 consensus_queue.cc:1048] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:17.668992  5596 catalog_manager.cc:5697] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } }
I20260709 13:08:17.671388  6814 consensus_queue.cc:1048] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:17.673868  6914 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:17.680886  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:17.681710  5772 raft_consensus.cc:2468] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:17.682766  5666 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:17.693480  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c979e51cdc44dac9ae6b2117e8ac0c5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:17.697419  5769 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:17.690601  6914 raft_consensus.cc:2804] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:17.701068  6914 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 373ee2f4cc9d4b86b49ada724f55e16a)
I20260709 13:08:17.701480  6914 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.713599  5846 raft_consensus.cc:1275] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 6 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:08:17.715517  6712 consensus_queue.cc:1048] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:17.725736  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:17.752657  5922 raft_consensus.cc:2393] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb in current term 2: Already voted for candidate 792907f253c744aa9392e8d572f88e06 in this term.
I20260709 13:08:17.765039  6876 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.765568  6876 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:17.766011  5767 raft_consensus.cc:1275] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:08:17.768007  6876 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:17.768950  6876 consensus_queue.cc:1048] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:17.769856  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:17.770599  5767 raft_consensus.cc:2468] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:17.774603  5890 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:17.775501  6815 raft_consensus.cc:2804] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:17.775933  6815 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.776285  6815 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.806329  6914 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:17.824244  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb" is_pre_election: true
I20260709 13:08:17.829211  5693 raft_consensus.cc:2468] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:17.833349  6914 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:17.837668  5776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:17.858376  5776 raft_consensus.cc:2468] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:17.859961  5666 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:17.863526  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:17.861583  6785 raft_consensus.cc:2804] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:17.865624  6785 raft_consensus.cc:697] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 2 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:17.866863  6785 consensus_queue.cc:237] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:17.882032  6815 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:17.884644  5922 raft_consensus.cc:2368] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 2. Current term is 3.
I20260709 13:08:17.891024  6815 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:17.891243  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:17.891963  5768 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.901197  5771 raft_consensus.cc:1275] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:17.904173  6681 consensus_queue.cc:1048] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:08:17.909121  6681 consensus_queue.cc:1243] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer 373ee2f4cc9d4b86b49ada724f55e16a log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:08:17.904145  5769 raft_consensus.cc:2468] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 5.
I20260709 13:08:17.923530  5850 raft_consensus.cc:1275] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:17.920210  5693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9952df48cf245608db9181dedad0aeb"
I20260709 13:08:17.926451  5890 leader_election.cc:304] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:17.928011  6876 raft_consensus.cc:2804] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:17.929576  6876 raft_consensus.cc:697] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 5 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:17.927887  5768 raft_consensus.cc:2468] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:17.931736  6876 consensus_queue.cc:237] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:17.937570  5667 leader_election.cc:400] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Vote denied by peer 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241) with higher term. Message: Invalid argument: T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9952df48cf245608db9181dedad0aeb for earlier term 2. Current term is 3.
I20260709 13:08:17.938431  5890 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:17.944823  6815 raft_consensus.cc:2804] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:17.945380  6815 raft_consensus.cc:697] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:17.946245  6681 consensus_queue.cc:1048] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:17.946296  6815 consensus_queue.cc:237] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:17.950608  5768 pending_rounds.cc:85] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a: Aborting all ops after (but not including) 1
I20260709 13:08:17.955097  5768 pending_rounds.cc:107] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:08:17.955453  5768 raft_consensus.cc:2889] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:18.059618  5597 catalog_manager.cc:5697] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } }
I20260709 13:08:18.061261  5597 catalog_manager.cc:5697] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:18.063256  5597 catalog_manager.cc:5697] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } }
I20260709 13:08:18.071970  5597 catalog_manager.cc:5697] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 2 to 3, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 3 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } }
I20260709 13:08:18.074555  5596 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39338 (request call id 64) took 1663 ms. Trace:
I20260709 13:08:18.083429  5597 catalog_manager.cc:5697] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 1 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } }
I20260709 13:08:18.079012  5596 rpcz_store.cc:276] 0709 13:08:16.434026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.448327 (+ 14301us) service_pool.cc:224] Handling call
0709 13:08:18.074465 (+1626138us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:17.702203 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:17.728866 (+ 26663us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:17.728921 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:08:17.728948 (+    27us) tablet.cc:662] Decoding operations
0709 13:08:17.742917 (+ 13969us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:17.743012 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:17.743044 (+    32us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:08:17.748776 (+  5732us) tablet.cc:701] Row locks acquired
0709 13:08:17.748811 (+    35us) write_op.cc:260] PREPARE: finished
0709 13:08:17.749045 (+   234us) write_op.cc:270] Start()
0709 13:08:17.749223 (+   178us) write_op.cc:276] Timestamp: P: 1783602497748978 usec, L: 0
0709 13:08:17.749258 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:08:17.779434 (+ 30176us) log.cc:844] Serialized 5193 byte log entry
0709 13:08:17.985740 (+206306us) op_driver.cc:464] REPLICATION: finished
0709 13:08:18.009615 (+ 23875us) write_op.cc:301] APPLY: starting
0709 13:08:18.009710 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:18.009984 (+   274us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:18.010131 (+   147us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:18.013159 (+  3028us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:18.013495 (+   336us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:08:18.013566 (+    71us) write_op.cc:312] APPLY: finished
0709 13:08:18.013968 (+   402us) log.cc:844] Serialized 94 byte log entry
0709 13:08:18.014916 (+   948us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:18.015108 (+   192us) write_op.cc:454] Released schema lock
0709 13:08:18.015180 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:18.015659 (+   479us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:18.015697 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":56519,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":23571,"apply.run_cpu_time_us":4682,"apply.run_wall_time_us":6000,"num_ops":13,"prepare.queue_time_us":9410,"prepare.run_cpu_time_us":30303,"prepare.run_wall_time_us":186099,"raft.queue_time_us":85452,"raft.run_cpu_time_us":7714,"raft.run_wall_time_us":29243,"replication_time_us":236255,"thread_start_us":155305,"threads_started":4,"wal-append.queue_time_us":110211}]]}
I20260709 13:08:18.136773  5917 raft_consensus.cc:1240] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 5 LEADER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:18.138485  6848 consensus_queue.cc:1059] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:18.139202  6848 consensus_queue.cc:1243] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer 792907f253c744aa9392e8d572f88e06 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:08:18.139639  6848 raft_consensus.cc:3055] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:18.139961  6848 raft_consensus.cc:740] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:18.140786  6848 consensus_queue.cc:260] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:18.142066  6848 raft_consensus.cc:3060] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:18.172011  5597 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39344 (request call id 73) took 1256 ms. Trace:
I20260709 13:08:18.172540  5597 rpcz_store.cc:276] 0709 13:08:16.915703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.927572 (+ 11869us) service_pool.cc:224] Handling call
0709 13:08:18.110195 (+1182623us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:18.110224 (+    29us) write_op.cc:454] Released schema lock
0709 13:08:18.171906 (+ 61682us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:18.100976 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:18.101833 (+   857us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:18.101878 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:08:18.101894 (+    16us) tablet.cc:662] Decoding operations
0709 13:08:18.102648 (+   754us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:18.102711 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:18.102752 (+    41us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:08:18.103598 (+   846us) tablet.cc:701] Row locks acquired
0709 13:08:18.103622 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:08:18.103800 (+   178us) write_op.cc:270] Start()
0709 13:08:18.103902 (+   102us) write_op.cc:276] Timestamp: P: 1783602498103746 usec, L: 0
0709 13:08:18.103963 (+    61us) op_driver.cc:348] REPLICATION: starting
0709 13:08:18.104663 (+   700us) log.cc:844] Serialized 2093 byte log entry
0709 13:08:18.106911 (+  2248us) op_driver.cc:464] REPLICATION: finished
0709 13:08:18.107378 (+   467us) write_op.cc:301] APPLY: starting
0709 13:08:18.107457 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:18.107627 (+   170us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:18.107709 (+    82us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:18.108266 (+   557us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:18.108439 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:08:18.108499 (+    60us) write_op.cc:312] APPLY: finished
0709 13:08:18.108711 (+   212us) log.cc:844] Serialized 46 byte log entry
0709 13:08:18.108888 (+   177us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:18.109000 (+   112us) write_op.cc:454] Released schema lock
0709 13:08:18.109047 (+    47us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":17996,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":261,"apply.run_cpu_time_us":1962,"apply.run_wall_time_us":1962,"num_ops":5,"prepare.queue_time_us":123,"prepare.run_cpu_time_us":3609,"prepare.run_wall_time_us":9133,"replication_time_us":2803}]]}
I20260709 13:08:18.166947  5598 catalog_manager.cc:5697] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 1 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } }
I20260709 13:08:18.184298  5598 catalog_manager.cc:5697] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 1 to 2, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 2 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } }
I20260709 13:08:18.274571  5768 raft_consensus.cc:1240] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Rejecting Update request from peer d9952df48cf245608db9181dedad0aeb for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:18.278142  6785 consensus_queue.cc:1059] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:18.279053  6785 raft_consensus.cc:3055] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:18.279441  6785 raft_consensus.cc:740] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:18.280892  6785 consensus_queue.cc:260] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:18.289149  6785 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.350593  5768 raft_consensus.cc:1275] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:18.353021  6876 consensus_queue.cc:1048] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:18.361246  5694 raft_consensus.cc:1275] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:08:18.361727  5694 pending_rounds.cc:85] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Aborting all ops after (but not including) 1
I20260709 13:08:18.361967  5694 pending_rounds.cc:107] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:08:18.362202  5694 raft_consensus.cc:2889] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:18.365334  6876 consensus_queue.cc:1048] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:18.369820  5599 catalog_manager.cc:5697] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:18.378469  5599 catalog_manager.cc:5697] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 4, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 4 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.382563  5768 raft_consensus.cc:1275] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:18.386441  6815 consensus_queue.cc:1048] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:18.398252  5599 catalog_manager.cc:5697] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 2 to 3, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 3 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.412986  5598 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:39364 (request call id 63) took 1148 ms. Trace:
I20260709 13:08:18.413487  5598 rpcz_store.cc:276] 0709 13:08:17.264618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.264855 (+   237us) service_pool.cc:224] Handling call
0709 13:08:18.412921 (+1148066us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:18.251881 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:18.304042 (+ 52161us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:18.304095 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:08:18.304114 (+    19us) tablet.cc:662] Decoding operations
0709 13:08:18.313592 (+  9478us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:18.313655 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:18.313689 (+    34us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:08:18.314038 (+   349us) tablet.cc:701] Row locks acquired
0709 13:08:18.314060 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:08:18.314233 (+   173us) write_op.cc:270] Start()
0709 13:08:18.314358 (+   125us) write_op.cc:276] Timestamp: P: 1783602498314183 usec, L: 0
0709 13:08:18.314381 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:08:18.315034 (+   653us) log.cc:844] Serialized 928 byte log entry
0709 13:08:18.320269 (+  5235us) op_driver.cc:464] REPLICATION: finished
0709 13:08:18.322824 (+  2555us) write_op.cc:301] APPLY: starting
0709 13:08:18.322912 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:18.323038 (+   126us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:18.323156 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:18.323437 (+   281us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:18.323605 (+   168us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:08:18.323656 (+    51us) write_op.cc:312] APPLY: finished
0709 13:08:18.323865 (+   209us) log.cc:844] Serialized 28 byte log entry
0709 13:08:18.324066 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:18.324158 (+    92us) write_op.cc:454] Released schema lock
0709 13:08:18.324202 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:18.362819 (+ 38617us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:18.362852 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12939,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":2311,"apply.run_cpu_time_us":1696,"apply.run_wall_time_us":1696,"num_ops":2,"prepare.queue_time_us":116,"prepare.run_cpu_time_us":3169,"prepare.run_wall_time_us":110900,"replication_time_us":5740}]]}
I20260709 13:08:18.429316  5596 catalog_manager.cc:5697] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 3 to 4, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 4 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:18.430857  5694 raft_consensus.cc:1275] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:08:18.431300  5694 pending_rounds.cc:85] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: Aborting all ops after (but not including) 2
I20260709 13:08:18.431540  5694 pending_rounds.cc:107] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:08:18.431799  5694 raft_consensus.cc:2889] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:18.432215  5596 catalog_manager.cc:5697] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 4 to 5, leader changed from 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 5 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.433243  6816 consensus_queue.cc:1048] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:18.433924  5596 catalog_manager.cc:5697] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 1 to 2, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 2 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:18.496829  5597 catalog_manager.cc:5697] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 3 to 6, leader changed from 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 6 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } }
I20260709 13:08:18.500682  5597 catalog_manager.cc:5697] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 0 to 2, leader changed from <none> to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.502737  5597 catalog_manager.cc:5697] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:18.521090  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:18.561697  5598 catalog_manager.cc:5697] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 4 to 5, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 5 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:18.564357  5598 catalog_manager.cc:5697] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 2 to 3, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 3 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:18.606195 27934 tablet_server.cc:179] TabletServer@127.27.71.129:0 shutting down...
W20260709 13:08:18.627422  5813 consensus_peers.cc:597] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.631855  5739 consensus_peers.cc:597] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.646729  5813 consensus_peers.cc:597] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.674193  5813 consensus_peers.cc:597] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.697918  5739 consensus_peers.cc:597] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.718717  5739 consensus_peers.cc:597] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.758445  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 49010us
W20260709 13:08:18.774544  5813 consensus_peers.cc:597] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.786597  5739 consensus_peers.cc:597] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.792001  5888 consensus_peers.cc:597] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.792706  5888 consensus_peers.cc:597] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.834018  5888 consensus_peers.cc:597] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.840850  5888 consensus_peers.cc:597] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.858672  5813 consensus_peers.cc:597] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.859529  5813 consensus_peers.cc:597] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.881965  5813 consensus_peers.cc:597] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.888790  5888 consensus_peers.cc:597] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.889915  5888 consensus_peers.cc:597] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.914842  5813 consensus_peers.cc:597] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.945237  5739 consensus_peers.cc:597] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.952188  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:37585, user_credentials={real_user=slave}} blocked reactor thread for 101857us
W20260709 13:08:18.956314  5667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 61185.5us
W20260709 13:08:18.959820  5739 consensus_peers.cc:597] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.962587  5739 consensus_peers.cc:597] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.968595  5813 consensus_peers.cc:597] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.979532  5739 consensus_peers.cc:597] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.976181  5813 consensus_peers.cc:597] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:18.985124  5813 consensus_peers.cc:597] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:19.003194  5888 consensus_peers.cc:597] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:19.004199 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:19.006309 27934 tablet_replica.cc:333] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.007447 27934 raft_consensus.cc:2243] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.008030 27934 raft_consensus.cc:2272] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.011204 27934 tablet_replica.cc:333] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
W20260709 13:08:19.014904  5888 consensus_peers.cc:597] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:19.021382  5739 consensus_peers.cc:597] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:19.028636 27934 raft_consensus.cc:2243] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.041213 27934 raft_consensus.cc:2272] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:19.044062  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 45591.5us
W20260709 13:08:19.061667  5739 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 35 similar messages]
W20260709 13:08:19.065227  5739 consensus_peers.cc:597] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:19.073992 27934 tablet_replica.cc:333] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.075070 27934 raft_consensus.cc:2243] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.103220 27934 raft_consensus.cc:2272] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.106209 27934 tablet_replica.cc:333] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.107106 27934 raft_consensus.cc:2243] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:19.110697  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 64097.8us
W20260709 13:08:19.117139  5813 consensus_peers.cc:597] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:19.107702 27934 raft_consensus.cc:2272] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.125334 27934 tablet_replica.cc:333] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.126284 27934 raft_consensus.cc:2243] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.126850 27934 raft_consensus.cc:2272] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.130839 27934 tablet_replica.cc:333] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.131796 27934 raft_consensus.cc:2243] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.142652 27934 raft_consensus.cc:2272] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.147419 27934 tablet_replica.cc:333] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.162438 27934 raft_consensus.cc:2243] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.163105 27934 raft_consensus.cc:2272] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.177270 27934 tablet_replica.cc:333] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.180723 27934 raft_consensus.cc:2243] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.183873 27934 raft_consensus.cc:2272] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:19.191064  5813 consensus_peers.cc:597] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:19.191867 27934 tablet_replica.cc:333] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.192919 27934 raft_consensus.cc:2243] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.193614 27934 raft_consensus.cc:2272] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.197079 27934 tablet_replica.cc:333] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.198025 27934 raft_consensus.cc:2243] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.198695 27934 raft_consensus.cc:2272] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.203429 27934 tablet_replica.cc:333] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.212113 27934 raft_consensus.cc:2243] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.212960 27934 raft_consensus.cc:2272] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.239306 27934 tablet_replica.cc:333] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.241042 27934 raft_consensus.cc:2243] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.241737 27934 raft_consensus.cc:2272] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.245180 27934 tablet_replica.cc:333] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.246129 27934 raft_consensus.cc:2243] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.246712 27934 raft_consensus.cc:2272] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.279259 27934 tablet_replica.cc:333] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.281324 27934 raft_consensus.cc:2243] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.282464 27934 raft_consensus.cc:2272] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.285483 27934 tablet_replica.cc:333] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.301299 27934 raft_consensus.cc:2243] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.302057 27934 raft_consensus.cc:2272] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.315659 27934 tablet_replica.cc:333] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.316730 27934 raft_consensus.cc:2243] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.317381 27934 raft_consensus.cc:2272] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.321434 27934 tablet_replica.cc:333] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.322386 27934 raft_consensus.cc:2243] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.323670 27934 raft_consensus.cc:2272] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.328651 27934 tablet_replica.cc:333] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.329836 27934 raft_consensus.cc:2243] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.330451 27934 raft_consensus.cc:2272] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.340842 27934 tablet_replica.cc:333] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.341831 27934 raft_consensus.cc:2243] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.342474 27934 raft_consensus.cc:2272] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.345724 27934 tablet_replica.cc:333] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.346586 27934 raft_consensus.cc:2243] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.348084 27934 raft_consensus.cc:2272] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.355000 27934 tablet_replica.cc:333] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.356086 27934 raft_consensus.cc:2243] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.357124 27934 raft_consensus.cc:2272] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.360095 27934 tablet_replica.cc:333] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.361411 27934 raft_consensus.cc:2243] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.362622 27934 raft_consensus.cc:2272] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.369292 27934 tablet_replica.cc:333] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.370276 27934 raft_consensus.cc:2243] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.371421 27934 raft_consensus.cc:2272] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.375075 27934 tablet_replica.cc:333] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.384613 27934 raft_consensus.cc:2243] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.385739 27934 raft_consensus.cc:2272] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.393085 27934 tablet_replica.cc:333] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.394138 27934 raft_consensus.cc:2243] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.395054 27934 raft_consensus.cc:2272] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.437506 27934 tablet_replica.cc:333] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.440752 27934 raft_consensus.cc:2243] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.444392 27934 raft_consensus.cc:2272] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.510344 27934 tablet_replica.cc:333] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.529556 27934 raft_consensus.cc:2243] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.535109 27934 raft_consensus.cc:2272] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.550179 27934 tablet_replica.cc:333] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.557608 27934 raft_consensus.cc:2243] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.558483 27934 raft_consensus.cc:2272] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.561766 27934 tablet_replica.cc:333] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.562698 27934 raft_consensus.cc:2243] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.564595 27934 raft_consensus.cc:2272] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.569751 27934 tablet_replica.cc:333] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.570990 27934 raft_consensus.cc:2243] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.572772 27934 raft_consensus.cc:2272] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.586220 27934 tablet_replica.cc:333] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.624321 27934 raft_consensus.cc:2243] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.625190 27934 raft_consensus.cc:2272] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.633858 27934 tablet_replica.cc:333] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.634892 27934 raft_consensus.cc:2243] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.635617 27934 raft_consensus.cc:2272] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.644970  6945 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 792907f253c744aa9392e8d572f88e06)
I20260709 13:08:19.645534  6945 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:19.646270 27934 tablet_replica.cc:333] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.647502 27934 raft_consensus.cc:2243] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.647718  6945 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:19.648166 27934 raft_consensus.cc:2272] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.650876  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:19.651499  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:19.655103  5667 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:19.655576 27934 tablet_replica.cc:333] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.656466 27934 raft_consensus.cc:2243] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.657024 27934 raft_consensus.cc:2272] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.657214  6784 raft_consensus.cc:2749] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:19.659958 27934 tablet_replica.cc:333] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.660995 27934 raft_consensus.cc:2243] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.661571 27934 raft_consensus.cc:2272] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.664680 27934 tablet_replica.cc:333] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.665350 27934 raft_consensus.cc:2243] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.666347 27934 raft_consensus.cc:2272] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.669234 27934 tablet_replica.cc:333] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.670090 27934 raft_consensus.cc:2243] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.670641 27934 raft_consensus.cc:2272] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.674079 27934 tablet_replica.cc:333] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.674930 27934 raft_consensus.cc:2243] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.675824 27934 raft_consensus.cc:2272] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.679013 27934 tablet_replica.cc:333] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.679842 27934 raft_consensus.cc:2243] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.680514 27934 raft_consensus.cc:2272] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.683630 27934 tablet_replica.cc:333] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.684311 27934 raft_consensus.cc:2243] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.684872 27934 raft_consensus.cc:2272] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.688117 27934 tablet_replica.cc:333] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.688807 27934 raft_consensus.cc:2243] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.689340 27934 raft_consensus.cc:2272] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.692060 27934 tablet_replica.cc:333] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.692713 27934 raft_consensus.cc:2243] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.693774 27934 raft_consensus.cc:2272] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.696769 27934 tablet_replica.cc:333] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.697124  6945 raft_consensus.cc:493] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52b726da57c24dd2aa72488ebe2ff21e)
I20260709 13:08:19.697660  6945 raft_consensus.cc:515] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:19.698392 27934 raft_consensus.cc:2243] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.700969 27934 raft_consensus.cc:2272] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.701452  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:19.704389  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f95c9f03a2848e29a1ed53170f6bc85" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:19.706528  5666 leader_election.cc:304] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
W20260709 13:08:19.707048  5666 raft_consensus.cc:2686] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:08:19.699734  6945 leader_election.cc:290] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:19.711139 27934 tablet_replica.cc:333] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.711840 27934 raft_consensus.cc:2243] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:19.713102 27934 raft_consensus.cc:2272] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:19.715458 27934 tablet_replica.cc:333] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: stopping tablet replica
I20260709 13:08:19.716172 27934 raft_consensus.cc:2243] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:19.716727 27934 raft_consensus.cc:2272] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:20.278898  6814 raft_consensus.cc:493] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.279560  6814 raft_consensus.cc:515] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.282110  6814 leader_election.cc:290] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.284801  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:20.384624  6712 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.385241  6712 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.387416  6712 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:20.396162  6781 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.396767  6781 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.400199  6781 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:20.417465  5813 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 49931.4us
W20260709 13:08:20.421146  5739 leader_election.cc:336] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.422254  5739 leader_election.cc:304] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
W20260709 13:08:20.421726  5888 leader_election.cc:336] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.428011  6814 raft_consensus.cc:2749] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:20.438753  5888 leader_election.cc:336] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.454856  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:20.460559  6712 raft_consensus.cc:493] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.461226  6712 raft_consensus.cc:515] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.464773  6712 leader_election.cc:290] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:20.468348  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:20.482142  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:20.483494  6814 raft_consensus.cc:493] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.483999  6944 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.484083  6814 raft_consensus.cc:515] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:20.484611  6944 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.486359  6814 leader_election.cc:290] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:20.486600  6944 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.482909  5772 raft_consensus.cc:2468] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:20.488572  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:20.490085  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:20.490756  5916 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:20.491772  6950 raft_consensus.cc:493] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.492332  6950 raft_consensus.cc:515] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:20.489574  5888 leader_election.cc:336] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.494417  6950 leader_election.cc:290] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:20.491820  5742 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:20.497442  6814 raft_consensus.cc:2804] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:20.497892  6814 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.498335  6814 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.504197  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:20.505044  5772 raft_consensus.cc:2468] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:20.504165  5890 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.507050  6816 raft_consensus.cc:2749] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:20.507155  5890 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.508225  5815 leader_election.cc:304] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:20.508411  6781 raft_consensus.cc:2749] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:20.513134  6950 raft_consensus.cc:2804] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:20.513720  6950 raft_consensus.cc:493] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.514256  6950 raft_consensus.cc:3060] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.539599  6814 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.557245  6814 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.541451  5890 leader_election.cc:304] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.558073  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
W20260709 13:08:20.547626  5739 leader_election.cc:336] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.558674  5916 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.559047  5739 leader_election.cc:304] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.531666  6950 raft_consensus.cc:515] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:20.561339  6950 leader_election.cc:290] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:20.561760  6712 raft_consensus.cc:2804] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:20.561863  6958 raft_consensus.cc:2749] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:20.562665  5739 leader_election.cc:336] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.563254  6712 raft_consensus.cc:493] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.563812  6712 raft_consensus.cc:3060] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.570328  5916 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 4.
I20260709 13:08:20.571652  5742 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:20.571491  6712 raft_consensus.cc:515] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.572484  6958 raft_consensus.cc:2804] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:20.573297  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:20.573861  5772 raft_consensus.cc:3060] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.576974  6958 raft_consensus.cc:697] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:20.577934  6958 consensus_queue.cc:237] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:20.579244  5888 leader_election.cc:336] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.580590  6712 leader_election.cc:290] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:20.584717  5813 leader_election.cc:336] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.588130  5772 raft_consensus.cc:2468] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:20.596573  5815 leader_election.cc:304] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
W20260709 13:08:20.596436  5739 leader_election.cc:336] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.602288  6952 raft_consensus.cc:2804] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.601293  6781 raft_consensus.cc:493] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.603647  6781 raft_consensus.cc:515] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.606772  6781 leader_election.cc:290] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:20.608237  5813 leader_election.cc:336] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.614603  6814 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.624154  6965 raft_consensus.cc:493] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.626122  6816 raft_consensus.cc:493] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.626765  6816 raft_consensus.cc:515] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:20.624897  6965 raft_consensus.cc:515] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:20.629298  6816 leader_election.cc:290] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.631299  6965 leader_election.cc:290] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:20.633738  5888 leader_election.cc:336] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.634045  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f9347b11ec44d1b07c475379a51881" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:20.615960  6814 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.620126  6952 raft_consensus.cc:697] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:20.640790  6952 consensus_queue.cc:237] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:20.644737  6814 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.645517  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:20.647936  5916 raft_consensus.cc:2468] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:20.649430  5742 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:20.650379  6958 raft_consensus.cc:2804] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:20.650811  6958 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.651170  6958 raft_consensus.cc:3060] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.655408  6814 raft_consensus.cc:493] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.656106  6814 raft_consensus.cc:515] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.658867  6958 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.659531  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:20.660252  5916 raft_consensus.cc:2468] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:20.660331  5597 catalog_manager.cc:5697] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 3 to 4, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 4 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:20.661851  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
W20260709 13:08:20.662268  5888 leader_election.cc:336] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.662391  5916 raft_consensus.cc:3060] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.662937  5742 leader_election.cc:304] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:20.664587  6962 raft_consensus.cc:2804] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:20.665958  6962 raft_consensus.cc:493] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.666451  6962 raft_consensus.cc:3060] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.723946  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:20.728705  5916 raft_consensus.cc:2468] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:20.730839  6957 raft_consensus.cc:493] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.731418  6957 raft_consensus.cc:515] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:20.733464  6957 leader_election.cc:290] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:20.664664  6814 leader_election.cc:290] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.727337  5891 leader_election.cc:304] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.739553  5742 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:20.739553  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f9347b11ec44d1b07c475379a51881" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:20.740013  6781 raft_consensus.cc:2749] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:20.740723  5850 raft_consensus.cc:2468] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:20.740896  6957 raft_consensus.cc:2804] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.741369  6950 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.741873  6950 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:20.742004  6957 raft_consensus.cc:697] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:20.742936  6962 raft_consensus.cc:515] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.744892  6950 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.743531  6957 consensus_queue.cc:237] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:20.747232  5813 leader_election.cc:336] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.747691  5813 leader_election.cc:304] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.748430  6965 raft_consensus.cc:2749] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:20.751852  5813 leader_election.cc:336] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.750592  5742 leader_election.cc:304] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:20.753551  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:20.755245  5816 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.721526  6944 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.764580  6958 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.761343  6944 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:20.760445  6814 raft_consensus.cc:2804] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:20.768350  6814 raft_consensus.cc:493] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.768805  6814 raft_consensus.cc:3060] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.769048  6781 raft_consensus.cc:493] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.769577  6781 raft_consensus.cc:515] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.774279  6962 leader_election.cc:290] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.770738  6950 raft_consensus.cc:2749] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:20.774926  6781 leader_election.cc:290] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:20.780689  6950 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.781256  6950 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:20.785166  6814 raft_consensus.cc:515] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:20.787837  6944 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:20.788456  6950 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:20.789621  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:20.791625  5916 raft_consensus.cc:3060] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.794708  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:20.795733  5768 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 5.
W20260709 13:08:20.797557  5813 leader_election.cc:336] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.799361  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:20.799712  5815 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.800150  5846 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 5.
I20260709 13:08:20.801214  6950 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:20.801664  6950 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.802058  6950 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:20.803205  6814 leader_election.cc:290] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:20.805327  5742 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:20.806391  6944 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:20.807778  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f9347b11ec44d1b07c475379a51881" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:20.809644  5846 raft_consensus.cc:3060] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.814565  5916 raft_consensus.cc:2468] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:20.816629  5742 leader_election.cc:304] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
W20260709 13:08:20.807940  5888 leader_election.cc:336] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.823669  5595 catalog_manager.cc:5697] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 2 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:20.809346  6944 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.834311  6944 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:20.819834  6962 raft_consensus.cc:2804] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.838236  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bf52f47b87456c94c45846de53788d" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:20.839004  5768 raft_consensus.cc:3060] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.844380  6965 raft_consensus.cc:493] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.845057  6965 raft_consensus.cc:515] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:20.847302  6965 leader_election.cc:290] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.848052  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:20.848759  5922 raft_consensus.cc:2468] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 1.
I20260709 13:08:20.850033  5816 leader_election.cc:304] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:20.851282  6963 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.851845  6963 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:20.852465 27934 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 5722)
I20260709 13:08:20.855801  6964 raft_consensus.cc:493] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.856273  6952 raft_consensus.cc:2804] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:20.856817  6952 raft_consensus.cc:493] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.856433  6964 raft_consensus.cc:515] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.857234  6952 raft_consensus.cc:3060] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.857944  6963 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.859256  6964 leader_election.cc:290] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:20.863518  6950 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:20.853813  5768 raft_consensus.cc:2468] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:20.854736  6962 raft_consensus.cc:697] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:20.855010  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8aa6b3cf215a40a5b6bc25bd3dddc408" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:08:20.855283  5813 leader_election.cc:336] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.875420  6962 consensus_queue.cc:237] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.877864  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:20.878201  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:20.878671  5850 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 3.
I20260709 13:08:20.878820  5772 raft_consensus.cc:2468] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 1.
I20260709 13:08:20.881726  5891 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:20.885442  6964 raft_consensus.cc:2804] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:20.885761  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
W20260709 13:08:20.886904  5813 leader_election.cc:336] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.888046  6964 raft_consensus.cc:493] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.888701  6964 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.891129  6963 raft_consensus.cc:493] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.891537  5846 raft_consensus.cc:2468] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:20.891822  6963 raft_consensus.cc:515] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.895068  5742 leader_election.cc:304] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
W20260709 13:08:20.895992  5888 leader_election.cc:336] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:20.896823  5888 leader_election.cc:336] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.900504  6950 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 6 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:20.901289  6814 raft_consensus.cc:2804] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.901479  6963 leader_election.cc:290] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:20.901964  6814 raft_consensus.cc:697] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:20.903632  5772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:20.887190  6952 raft_consensus.cc:515] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:20.903091  6814 consensus_queue.cc:237] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:20.906206  6952 leader_election.cc:290] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.910305  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:20.912703  5890 leader_election.cc:304] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.914027  5890 leader_election.cc:304] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.915115  6816 raft_consensus.cc:2804] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.915242  6781 raft_consensus.cc:2749] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:20.915803  6816 raft_consensus.cc:697] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:20.916893  5890 leader_election.cc:304] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.916906  6816 consensus_queue.cc:237] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.919275  5916 raft_consensus.cc:3060] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.919764  6961 raft_consensus.cc:2804] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:20.920181  6961 raft_consensus.cc:493] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.920531  6961 raft_consensus.cc:3060] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:20.921329  5813 leader_election.cc:336] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.927692  5890 leader_election.cc:304] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.933305  6944 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:20.929379  6781 raft_consensus.cc:2749] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:20.938781  6964 raft_consensus.cc:515] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:20.960763  5888 leader_election.cc:336] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.962338  6964 leader_election.cc:290] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:20.942551  6961 raft_consensus.cc:515] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:20.943308  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:20.964845  6961 leader_election.cc:290] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:20.945063  5739 leader_election.cc:336] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.951998  5916 raft_consensus.cc:2468] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:20.967720  5816 leader_election.cc:304] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:20.968806  6965 raft_consensus.cc:2804] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:20.968721  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f08c26e76934d3c94ef7060e15caed0" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:20.969466  5846 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.969820  6965 raft_consensus.cc:697] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:20.970752  6965 consensus_queue.cc:237] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:20.973719  6781 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:20.974345  6781 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:20.980177  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:20.983270  5846 raft_consensus.cc:2468] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 4.
I20260709 13:08:20.984467  5891 leader_election.cc:304] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:20.985266  6961 raft_consensus.cc:2804] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:20.985994  6961 raft_consensus.cc:697] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 4 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:20.986855  6961 consensus_queue.cc:237] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:20.987985  5739 leader_election.cc:336] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:20.988791  5739 leader_election.cc:336] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:20.989516  5888 leader_election.cc:336] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:20.989711  5739 leader_election.cc:336] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:20.990496  5739 leader_election.cc:336] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:20.991267  5739 leader_election.cc:336] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:20.992493  5739 leader_election.cc:336] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:20.994695  5888 leader_election.cc:336] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:20.995754  5772 raft_consensus.cc:2393] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 6: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
I20260709 13:08:20.997107  6781 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:21.002743  5890 leader_election.cc:304] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:21.003932  6781 raft_consensus.cc:2749] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:21.006127  6966 raft_consensus.cc:493] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:21.006678  6966 raft_consensus.cc:515] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:21.009629  5888 leader_election.cc:336] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:21.010147  5888 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:21.013011  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4890d4f381d74b37a99616c1b1dc65d4" candidate_uuid: "792907f253c744aa9392e8d572f88e06" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:21.013680  5768 raft_consensus.cc:3060] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:21.013219  6781 raft_consensus.cc:2749] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:08:21.020452  5739 consensus_peers.cc:597] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:21.021545  6966 leader_election.cc:290] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:21.028766  5815 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:21.030412  5768 raft_consensus.cc:2468] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792907f253c744aa9392e8d572f88e06 in term 2.
I20260709 13:08:21.031379  5890 leader_election.cc:304] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:21.037343  6963 raft_consensus.cc:2804] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:21.038158  6963 raft_consensus.cc:697] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Leader. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:21.039161  6963 consensus_queue.cc:237] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:21.041283  5739 leader_election.cc:336] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:21.042335  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:21.042982  5916 raft_consensus.cc:2468] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:21.041445  6952 raft_consensus.cc:2749] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:21.046267  5742 leader_election.cc:304] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:21.047019  6966 raft_consensus.cc:2804] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:21.047395  6966 raft_consensus.cc:493] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:21.047752  6966 raft_consensus.cc:3060] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:21.057777  5916 raft_consensus.cc:1275] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:21.060657  6814 consensus_queue.cc:1048] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:21.062145  6944 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 6 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:21.073762  5597 catalog_manager.cc:5697] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:21.075503  5597 catalog_manager.cc:5697] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:21.077033  5597 catalog_manager.cc:5697] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:21.083846  5595 catalog_manager.cc:5697] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 2 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:21.087857  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:21.088997  5846 raft_consensus.cc:2393] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 6: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:21.090615  5598 catalog_manager.cc:5697] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:21.085425  6966 raft_consensus.cc:515] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:21.131999  6950 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:21.133886  6950 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:21.136178  6950 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:21.138121  5813 consensus_peers.cc:597] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.140681  5739 leader_election.cc:336] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:21.141207  5739 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:21.142059  6814 raft_consensus.cc:2749] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:21.145011  6966 leader_election.cc:290] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:21.146708  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b715b22a0343fbb87245b6bf0fb522" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:21.147539  5917 raft_consensus.cc:3060] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:21.150031  5813 leader_election.cc:336] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:21.150720  5813 consensus_peers.cc:597] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.153247  5739 consensus_peers.cc:597] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.154124  5813 consensus_peers.cc:597] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:21.156018  5739 leader_election.cc:336] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:21.156976  6944 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:21.157317  5917 raft_consensus.cc:2468] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:21.157721  6944 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:21.160504  6966 raft_consensus.cc:493] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:21.161247  6966 raft_consensus.cc:515] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:21.159509  5742 leader_election.cc:304] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:21.164021  6962 raft_consensus.cc:2804] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:21.164877  6962 raft_consensus.cc:697] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:21.166111  6962 consensus_queue.cc:237] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:21.166875  6966 leader_election.cc:290] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:21.169808  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:21.170475  5920 raft_consensus.cc:2468] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 1.
I20260709 13:08:21.177073  5742 leader_election.cc:304] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:21.164095  6944 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:21.180481  6957 raft_consensus.cc:2804] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:21.181135  6957 raft_consensus.cc:493] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:21.189657  5846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:21.190410  5846 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:21.190357  5768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:21.191540  5768 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 2.
I20260709 13:08:21.193540  5767 raft_consensus.cc:1275] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:21.195830  5742 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:21.196875  6944 raft_consensus.cc:2804] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:21.191771  6957 raft_consensus.cc:3060] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:21.200313  5739 leader_election.cc:336] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:21.201182  5815 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: d9952df48cf245608db9181dedad0aeb
W20260709 13:08:21.203445  5739 leader_election.cc:336] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:21.201216  6950 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:21.205497  6950 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:21.207695  6950 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:21.197353  6944 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:21.211128  6944 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:21.215868  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:21.216732  5920 raft_consensus.cc:2468] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 3.
I20260709 13:08:21.217924  5816 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
W20260709 13:08:21.218827  5813 leader_election.cc:336] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:21.219672  6982 raft_consensus.cc:2804] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:21.220178  6982 raft_consensus.cc:493] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:21.220532  6982 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:21.221772  5888 consensus_peers.cc:597] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.224426  6985 raft_consensus.cc:2804] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:21.224907  6985 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Starting leader election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:21.225292  6985 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:21.231834  6950 consensus_queue.cc:1048] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:21.269345  6982 raft_consensus.cc:515] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:21.266701  5598 catalog_manager.cc:5697] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 2 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:21.271819  5598 catalog_manager.cc:5697] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 reported cstate change: term changed from 3 to 4, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 792907f253c744aa9392e8d572f88e06 (127.27.71.132). New cstate: current_term: 4 leader_uuid: "792907f253c744aa9392e8d572f88e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:21.282336  5597 catalog_manager.cc:5697] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:21.297091  5813 consensus_peers.cc:597] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.317063  6982 leader_election.cc:290] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:21.316895  6957 raft_consensus.cc:515] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:21.321290  6944 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:21.319631  6957 leader_election.cc:290] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
W20260709 13:08:21.357728  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 77336.5us
I20260709 13:08:21.361627  5920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cb2353dae2d41588ef00b83e6cc10d1" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:21.362278  5920 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:21.362941  5888 consensus_peers.cc:597] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.366536  5739 leader_election.cc:336] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:21.366714  6944 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
W20260709 13:08:21.367166  5739 consensus_peers.cc:597] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.367635  5739 consensus_peers.cc:597] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:21.369776  5813 consensus_peers.cc:597] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.383036  6985 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:21.384125  5739 leader_election.cc:336] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:21.384749  5739 consensus_peers.cc:597] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.391484  5888 consensus_peers.cc:597] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.392043  5888 consensus_peers.cc:597] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.381152  5920 raft_consensus.cc:2468] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b726da57c24dd2aa72488ebe2ff21e in term 4.
I20260709 13:08:21.395535  5816 leader_election.cc:304] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:21.399560  6993 raft_consensus.cc:2804] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:21.400296  6993 raft_consensus.cc:697] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 LEADER]: Becoming Leader. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:21.401407  6993 consensus_queue.cc:237] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:21.413686  5813 consensus_peers.cc:597] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.416860  5813 leader_election.cc:336] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:21.420635  5813 consensus_peers.cc:597] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.423595  5739 consensus_peers.cc:597] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:21.424149  5739 consensus_peers.cc:597] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.424633  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:21.425436  5851 raft_consensus.cc:2393] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 373ee2f4cc9d4b86b49ada724f55e16a in current term 3: Already voted for candidate 52b726da57c24dd2aa72488ebe2ff21e in this term.
I20260709 13:08:21.425587  5916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e48769916844ce8d464a5bd1bda6de" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:21.426739  5916 raft_consensus.cc:3060] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:21.426829  5888 consensus_peers.cc:597] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.429948  5851 raft_consensus.cc:1275] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:21.432003  6985 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:21.434036  5917 raft_consensus.cc:1275] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:21.440883  5916 raft_consensus.cc:2468] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
W20260709 13:08:21.446419  5888 consensus_peers.cc:597] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:21.441455  5742 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:21.456552  6991 raft_consensus.cc:2749] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:08:21.460815  5813 leader_election.cc:336] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:21.459838  6991 consensus_queue.cc:1048] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:08:21.468387  5922 raft_consensus.cc:1275] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:08:21.461457  5813 consensus_peers.cc:597] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.471698  5813 consensus_peers.cc:597] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:21.471777  5922 raft_consensus.cc:1275] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:21.474923  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:21.475733  5769 raft_consensus.cc:2393] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52b726da57c24dd2aa72488ebe2ff21e in current term 3: Already voted for candidate 373ee2f4cc9d4b86b49ada724f55e16a in this term.
W20260709 13:08:21.481231  5739 consensus_peers.cc:597] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.482942  5742 leader_election.cc:304] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:21.483587  6993 consensus_queue.cc:1048] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:21.498553  6957 raft_consensus.cc:2804] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:21.499085  6957 raft_consensus.cc:697] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
W20260709 13:08:21.499711  5888 consensus_peers.cc:597] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.499617  6995 consensus_queue.cc:1048] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:21.499997  6957 consensus_queue.cc:237] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:21.501562  6962 consensus_queue.cc:1048] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:21.506680  5739 consensus_peers.cc:597] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.507445  5815 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:21.509244  6993 raft_consensus.cc:2749] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:21.492645  5597 catalog_manager.cc:5697] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e reported cstate change: term changed from 3 to 4, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131). New cstate: current_term: 4 leader_uuid: "52b726da57c24dd2aa72488ebe2ff21e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:21.497067  5813 consensus_peers.cc:597] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.528905  5768 raft_consensus.cc:1275] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:08:21.539232  5813 consensus_peers.cc:597] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.540266  5888 consensus_peers.cc:597] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.541962  6983 consensus_queue.cc:1048] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:21.544605  5739 consensus_peers.cc:597] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.544740  5888 consensus_peers.cc:597] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:21.548339  5739 consensus_peers.cc:597] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:21.582141  5888 consensus_peers.cc:597] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:21.637212  5769 raft_consensus.cc:1275] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:21.640822  6964 consensus_queue.cc:1048] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:21.647011  5851 raft_consensus.cc:1275] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Refusing update from remote peer 792907f253c744aa9392e8d572f88e06: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:21.649358  5598 catalog_manager.cc:5697] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:21.652211  5739 consensus_peers.cc:597] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:21.658211  5813 consensus_peers.cc:597] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.663050  6984 consensus_queue.cc:1048] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:08:21.665627  5917 raft_consensus.cc:1275] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:21.668637  6957 consensus_queue.cc:1048] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:21.665938  5888 consensus_peers.cc:597] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.790747  5739 consensus_peers.cc:597] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.804800  5813 consensus_peers.cc:597] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.816123  5739 consensus_peers.cc:597] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.845314  5813 consensus_peers.cc:597] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:21.852331  5917 raft_consensus.cc:1275] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Refusing update from remote peer 52b726da57c24dd2aa72488ebe2ff21e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:08:21.856098 27934 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 5722)
I20260709 13:08:21.858172  6985 consensus_queue.cc:1048] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:21.899633  5920 raft_consensus.cc:1275] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:08:21.920346  5813 consensus_peers.cc:597] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:21.924532  6944 consensus_queue.cc:1048] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:08:21.995817  5739 consensus_peers.cc:597] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:21.996320  5739 consensus_peers.cc:597] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:22.005378  5813 consensus_peers.cc:597] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:08:22.843782  6944 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.845331  6944 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:22.848232  6944 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:22.849490  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:22.850675  5850 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 6.
I20260709 13:08:22.856971  5742 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
W20260709 13:08:22.856977 27934 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 5722)
I20260709 13:08:22.858667  6957 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:22.859169  6957 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.859565  6957 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:22.889390  6957 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:22.892002  5739 leader_election.cc:336] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:22.894037  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:22.894690  5850 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:22.900104  6957 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 7 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:22.911070  5850 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 7.
W20260709 13:08:22.914394  5739 leader_election.cc:336] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:22.916500  5742 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:22.917850  6944 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:22.918429  6944 raft_consensus.cc:697] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 7 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:22.919715  6944 consensus_queue.cc:237] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:22.963121  5598 catalog_manager.cc:5697] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 5 to 7, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 7 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:23.188726  6957 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.189463  6957 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:23.195340  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:23.196584  5850 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 3.
I20260709 13:08:23.199447  5742 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: 
I20260709 13:08:23.200424  6966 raft_consensus.cc:2804] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:23.200826  6966 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:23.201135  5739 leader_election.cc:336] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:23.201156  6966 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:23.208621  6957 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:23.211697  6966 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:23.215060  5850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:23.216022  5850 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:23.223881  5739 leader_election.cc:336] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:23.233239  5850 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 4.
I20260709 13:08:23.234403  6966 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Requested vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:23.239100  5742 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e; no voters: d9952df48cf245608db9181dedad0aeb
I20260709 13:08:23.243736  6957 raft_consensus.cc:2804] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:23.245859  6957 raft_consensus.cc:697] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:23.247133  6957 consensus_queue.cc:237] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:23.276540  5598 catalog_manager.cc:5697] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 2 to 4, leader changed from d9952df48cf245608db9181dedad0aeb (127.27.71.129) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 4 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:23.396492  5850 raft_consensus.cc:1275] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 7 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:08:23.401746  6957 consensus_queue.cc:1048] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:08:23.456125  5888 consensus_peers.cc:597] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.585877  5888 consensus_peers.cc:597] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.612587  5739 consensus_peers.cc:597] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:23.667569  5739 consensus_peers.cc:597] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.692114  5739 consensus_peers.cc:597] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.694305  5813 consensus_peers.cc:597] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.695832  5813 consensus_peers.cc:597] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.696429  5813 consensus_peers.cc:597] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.698141  5739 consensus_peers.cc:597] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.703464  5813 consensus_peers.cc:597] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.733886  5813 consensus_peers.cc:597] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.749593  5739 consensus_peers.cc:597] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.750151  5739 consensus_peers.cc:597] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.750587  5888 consensus_peers.cc:597] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:23.794221  5888 consensus_peers.cc:597] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.800108  5813 consensus_peers.cc:597] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.858781 27934 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 5722)
I20260709 13:08:23.879696  5851 raft_consensus.cc:1275] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 13:08:23.888157  5739 consensus_peers.cc:597] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:23.891255  7030 consensus_queue.cc:1048] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:08:23.917968  5813 consensus_peers.cc:597] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.952541  5813 consensus_peers.cc:597] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.953835  5888 consensus_peers.cc:597] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:23.962064  5739 consensus_peers.cc:597] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:24.012230  5739 consensus_peers.cc:597] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.017133  5888 consensus_peers.cc:597] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:24.034834  5888 consensus_peers.cc:597] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.063040  5888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111) [suppressed 412 similar messages]
W20260709 13:08:24.085405  5813 consensus_peers.cc:597] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.086927  5739 consensus_peers.cc:597] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.098538  5888 consensus_peers.cc:597] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:24.100586  5813 consensus_peers.cc:597] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:24.125452  5739 consensus_peers.cc:597] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:24.138854  5739 consensus_peers.cc:597] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.175799  5813 consensus_peers.cc:597] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.185483  5888 consensus_peers.cc:597] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.208842  5739 consensus_peers.cc:597] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.232162  5888 consensus_peers.cc:597] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.244161  5739 consensus_peers.cc:597] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.249182  5813 consensus_peers.cc:597] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.281917  5739 consensus_peers.cc:597] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:24.297077  5739 consensus_peers.cc:597] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.319708  5813 consensus_peers.cc:597] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.327658  5888 consensus_peers.cc:597] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.378988  5739 consensus_peers.cc:597] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.404058  5813 consensus_peers.cc:597] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:24.475152  5813 consensus_peers.cc:597] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.559363  5813 consensus_peers.cc:597] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:24.859407 27934 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 5722)
W20260709 13:08:24.887435  5739 consensus_peers.cc:597] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:25.860004 27934 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 5722)
W20260709 13:08:25.959676  5813 consensus_peers.cc:597] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:25.965574  5813 consensus_peers.cc:597] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:25.977164  5888 consensus_peers.cc:597] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:25.978890  5739 consensus_peers.cc:597] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:25.997143  5888 consensus_peers.cc:597] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.047875  5739 consensus_peers.cc:597] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:26.082084  5888 consensus_peers.cc:597] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.152173  5813 consensus_peers.cc:597] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.162184  5739 consensus_peers.cc:597] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.165287  5739 consensus_peers.cc:597] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.173275  5739 consensus_peers.cc:597] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.192245  5813 consensus_peers.cc:597] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.247736  5888 consensus_peers.cc:597] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.505664  5813 consensus_peers.cc:597] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.507788  5813 consensus_peers.cc:597] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.534744  5813 consensus_peers.cc:597] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.572233  5813 consensus_peers.cc:597] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.603062  5739 consensus_peers.cc:597] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.603979  5888 consensus_peers.cc:597] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.606238  5888 consensus_peers.cc:597] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.606792  5888 consensus_peers.cc:597] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.607362  5888 consensus_peers.cc:597] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.610875  5739 consensus_peers.cc:597] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.632143  5813 consensus_peers.cc:597] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.644951  5739 consensus_peers.cc:597] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.683298  5739 consensus_peers.cc:597] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:26.686146  5739 consensus_peers.cc:597] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.695504  5739 consensus_peers.cc:597] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.713301  5739 consensus_peers.cc:597] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.740757  5888 consensus_peers.cc:597] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.756901  5813 consensus_peers.cc:597] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.775393  5739 consensus_peers.cc:597] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.811957  5888 consensus_peers.cc:597] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.819243  5813 consensus_peers.cc:597] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.840576  5813 consensus_peers.cc:597] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.860615 27934 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 5722)
W20260709 13:08:26.917568  5813 consensus_peers.cc:597] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.921160  5813 consensus_peers.cc:597] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:26.970932  5888 consensus_peers.cc:597] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:26.993856  5739 consensus_peers.cc:597] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.014007  5739 consensus_peers.cc:597] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.035282  5739 consensus_peers.cc:597] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:27.113405  5813 consensus_peers.cc:597] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.137356  5739 consensus_peers.cc:597] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.188283  5813 consensus_peers.cc:597] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:27.265226  5739 consensus_peers.cc:597] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:27.979737 27934 tablet_server.cc:196] TabletServer@127.27.71.129:0 shutdown complete.
I20260709 13:08:28.224778 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:28.241852  7055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:28.242861  7056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:28.253367 27934 server_base.cc:1061] running on GCE node
W20260709 13:08:28.253957  7058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:28.260202 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:28.262396 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:28.262732 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602508262707 us; error 0 us; skew 500 ppm
I20260709 13:08:28.271170 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260709 13:08:28.324486  5813 consensus_peers.cc:597] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.325127  5739 consensus_peers.cc:597] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:28.374032  5888 consensus_peers.cc:597] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:28.404769 27934 webserver.cc:533] Webserver started at http://127.27.71.129:46079/ using document root <none> and password file <none>
I20260709 13:08:28.405680 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:28.405951 27934 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:08:28.412101 27934 fs_manager.cc:714] Time spent opening directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:08:28.425534  7063 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:28.426982 27934 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:08:28.427404 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-0-root
uuid: "d9952df48cf245608db9181dedad0aeb"
format_stamp: "Formatted at 2026-07-09 13:07:52 on dist-test-slave-mlt7"
W20260709 13:08:28.440814  5888 consensus_peers.cc:597] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:28.435457 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602179482400-27934-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:08:28.493155  5813 consensus_peers.cc:597] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:28.495788  5813 consensus_peers.cc:597] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.600085  5739 consensus_peers.cc:597] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.621834  5813 consensus_peers.cc:597] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:28.625324 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:28.633742 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260709 13:08:28.635710  5739 consensus_peers.cc:597] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:28.646225 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
W20260709 13:08:28.718875  5739 consensus_peers.cc:597] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:28.720363  7071 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:08:28.777709  5739 consensus_peers.cc:597] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.787621  5888 consensus_peers.cc:597] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.803993  5813 consensus_peers.cc:597] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.831228  5888 consensus_peers.cc:597] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.832896  5739 consensus_peers.cc:597] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:28.863818  5888 consensus_peers.cc:597] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:28.877821  5739 consensus_peers.cc:597] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.932475  5739 consensus_peers.cc:597] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:28.992751  5813 consensus_peers.cc:597] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:29.007694  5888 consensus_peers.cc:597] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:29.074080  5739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111) [suppressed 447 similar messages]
W20260709 13:08:29.099742  5813 consensus_peers.cc:597] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:29.107488  5739 consensus_peers.cc:597] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:29.123306  5888 consensus_peers.cc:597] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:29.166661  5888 consensus_peers.cc:597] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:29.172605  5813 consensus_peers.cc:597] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:29.208848 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:08:29.209338 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.531s	user 0.017s	sys 0.025s
I20260709 13:08:29.209729 27934 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:08:29.222980  7071 tablet_bootstrap.cc:492] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:29.225821  5813 consensus_peers.cc:597] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:29.232941  5888 consensus_peers.cc:597] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:29.269387  5813 consensus_peers.cc:597] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:29.297732  7071 tablet_bootstrap.cc:492] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:29.298933  7071 tablet_bootstrap.cc:492] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:29.299726  7071 ts_tablet_manager.cc:1403] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.077s	user 0.019s	sys 0.005s
I20260709 13:08:29.304317  7071 raft_consensus.cc:359] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:29.305133  7071 raft_consensus.cc:740] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.306002  7071 consensus_queue.cc:260] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:29.321513  5739 consensus_peers.cc:597] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:29.324261  7071 ts_tablet_manager.cc:1434] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:08:29.325364  7071 tablet_bootstrap.cc:492] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:29.340454  5813 consensus_peers.cc:597] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:29.345494  7071 tablet_bootstrap.cc:492] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:29.346735  7071 tablet_bootstrap.cc:492] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:29.347539  7071 ts_tablet_manager.cc:1403] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.004s
I20260709 13:08:29.350607  7071 raft_consensus.cc:359] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:29.351402  7071 raft_consensus.cc:740] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.352586  7071 consensus_queue.cc:260] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:29.354902  7071 ts_tablet_manager.cc:1434] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:08:29.355825  7071 tablet_bootstrap.cc:492] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:29.363463  5888 consensus_peers.cc:597] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:29.397948  5739 consensus_peers.cc:597] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:29.409138  7071 tablet_bootstrap.cc:492] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:29.410283  7071 tablet_bootstrap.cc:492] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:29.411023  7071 ts_tablet_manager.cc:1403] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.008s
I20260709 13:08:29.424796  7071 raft_consensus.cc:359] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:29.425693  7071 raft_consensus.cc:740] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.426677  7071 consensus_queue.cc:260] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:29.438896  7071 ts_tablet_manager.cc:1434] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.000s
I20260709 13:08:29.439884  7071 tablet_bootstrap.cc:492] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:29.455552  5739 consensus_peers.cc:597] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:29.471730  5739 consensus_peers.cc:597] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:29.481052  5813 consensus_peers.cc:597] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:29.512742  7071 tablet_bootstrap.cc:492] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:29.513831  7071 tablet_bootstrap.cc:492] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:29.520488  7071 ts_tablet_manager.cc:1403] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.081s	user 0.005s	sys 0.020s
I20260709 13:08:29.524821  7071 raft_consensus.cc:359] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:29.525595  7071 raft_consensus.cc:740] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.526443  7071 consensus_queue.cc:260] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
W20260709 13:08:29.533496  5813 consensus_peers.cc:597] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:29.534945  7071 ts_tablet_manager.cc:1434] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
W20260709 13:08:29.543720  5739 consensus_peers.cc:597] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:29.546948  5739 consensus_peers.cc:597] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:29.548743  5813 consensus_peers.cc:597] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:29.550536  7071 tablet_bootstrap.cc:492] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:29.569360  5739 consensus_peers.cc:597] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:29.609323  7071 tablet_bootstrap.cc:492] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:29.610553  7071 tablet_bootstrap.cc:492] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:29.611265  7071 ts_tablet_manager.cc:1403] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.076s	user 0.021s	sys 0.004s
I20260709 13:08:29.614073  7071 raft_consensus.cc:359] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:29.617426  5739 consensus_peers.cc:597] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:29.620023  7071 raft_consensus.cc:740] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.621922  7071 consensus_queue.cc:260] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:29.624270  7071 ts_tablet_manager.cc:1434] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.004s
I20260709 13:08:29.626385  7071 tablet_bootstrap.cc:492] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:29.659650  5813 consensus_peers.cc:597] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:29.665382  5888 consensus_peers.cc:597] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:29.682574  7071 tablet_bootstrap.cc:492] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:29.683821  7071 tablet_bootstrap.cc:492] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:29.684698  7071 ts_tablet_manager.cc:1403] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.059s	user 0.020s	sys 0.000s
I20260709 13:08:29.687784  7071 raft_consensus.cc:359] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:29.688628  7071 raft_consensus.cc:740] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.689538  7071 consensus_queue.cc:260] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:29.693032  7071 ts_tablet_manager.cc:1434] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:08:29.693971  7071 tablet_bootstrap.cc:492] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:29.713822  5813 consensus_peers.cc:597] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:29.715411 27934 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:08:29.715865 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.506s	user 0.311s	sys 0.003s
I20260709 13:08:29.781152  7071 tablet_bootstrap.cc:492] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:29.783705  7071 tablet_bootstrap.cc:492] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:29.786896  7071 ts_tablet_manager.cc:1403] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.093s	user 0.023s	sys 0.007s
I20260709 13:08:29.804036  7071 raft_consensus.cc:359] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:29.804848  7071 raft_consensus.cc:740] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.805717  7071 consensus_queue.cc:260] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:29.817217  7071 ts_tablet_manager.cc:1434] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:08:29.822587  7071 tablet_bootstrap.cc:492] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:29.883613  7071 tablet_bootstrap.cc:492] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:29.889236  7071 tablet_bootstrap.cc:492] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:29.890319  7071 ts_tablet_manager.cc:1403] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.072s	user 0.017s	sys 0.002s
W20260709 13:08:29.902318  5739 consensus_peers.cc:597] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:08:29.939391  7071 raft_consensus.cc:359] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:29.956800  7071 raft_consensus.cc:740] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.958850  7071 consensus_queue.cc:260] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:29.961181  7071 ts_tablet_manager.cc:1434] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.000s
I20260709 13:08:29.969063  7071 tablet_bootstrap.cc:492] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:30.069101  7071 tablet_bootstrap.cc:492] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:30.076596  7071 tablet_bootstrap.cc:492] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:30.078214  7071 ts_tablet_manager.cc:1403] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.113s	user 0.022s	sys 0.000s
I20260709 13:08:30.088518  7071 raft_consensus.cc:359] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:30.092558  7071 raft_consensus.cc:740] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.093467  7071 consensus_queue.cc:260] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:30.097044  7071 ts_tablet_manager.cc:1434] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:08:30.098093  7071 tablet_bootstrap.cc:492] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:30.170760  7071 tablet_bootstrap.cc:492] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:30.171882  7071 tablet_bootstrap.cc:492] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:30.172667  7071 ts_tablet_manager.cc:1403] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.075s	user 0.019s	sys 0.007s
I20260709 13:08:30.183542  7071 raft_consensus.cc:359] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:30.184453  7071 raft_consensus.cc:740] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.185441  7071 consensus_queue.cc:260] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:30.188361  7071 ts_tablet_manager.cc:1434] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:08:30.191020  7071 tablet_bootstrap.cc:492] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:30.256640  7071 tablet_bootstrap.cc:492] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:30.257867  7071 tablet_bootstrap.cc:492] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:30.258842  7071 ts_tablet_manager.cc:1403] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.007s
I20260709 13:08:30.262017  7071 raft_consensus.cc:359] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:30.262852  7071 raft_consensus.cc:740] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.263849  7071 consensus_queue.cc:260] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:30.266475  7071 ts_tablet_manager.cc:1434] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:08:30.267436  7071 tablet_bootstrap.cc:492] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:30.601426  7071 tablet_bootstrap.cc:492] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:30.604007  7071 tablet_bootstrap.cc:492] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:30.616753  7071 ts_tablet_manager.cc:1403] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.350s	user 0.089s	sys 0.094s
I20260709 13:08:30.619867  7071 raft_consensus.cc:359] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:30.620792  7071 raft_consensus.cc:740] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.621757  7071 consensus_queue.cc:260] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:30.624766  7071 ts_tablet_manager.cc:1434] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:08:30.625869  7071 tablet_bootstrap.cc:492] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:30.669062  7071 tablet_bootstrap.cc:492] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:30.670174  7071 tablet_bootstrap.cc:492] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:30.671061  7071 ts_tablet_manager.cc:1403] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.045s	user 0.020s	sys 0.001s
I20260709 13:08:30.673946  7071 raft_consensus.cc:359] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:30.684636  7071 raft_consensus.cc:740] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.686474  7071 consensus_queue.cc:260] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:30.691094  7071 ts_tablet_manager.cc:1434] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:08:30.692127  7071 tablet_bootstrap.cc:492] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:30.731819  5888 consensus_peers.cc:597] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:30.733641  7071 tablet_bootstrap.cc:492] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:08:30.783077  5888 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 51312.9us
I20260709 13:08:30.784081  7071 tablet_bootstrap.cc:492] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:30.784998  7071 ts_tablet_manager.cc:1403] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.093s	user 0.018s	sys 0.004s
I20260709 13:08:30.788316  7071 raft_consensus.cc:359] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:30.789145  7071 raft_consensus.cc:740] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.790236  7071 consensus_queue.cc:260] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:30.793016  7071 ts_tablet_manager.cc:1434] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:08:30.805253  7071 tablet_bootstrap.cc:492] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:30.824641  5739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:40419, user_credentials={real_user=slave}} blocked reactor thread for 50188.2us
I20260709 13:08:30.854899  7151 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.129:40419 every 8 connection(s)
I20260709 13:08:30.855697 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.129:40419
I20260709 13:08:30.878590  7071 tablet_bootstrap.cc:492] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=1 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:30.880004  7071 tablet_bootstrap.cc:492] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:30.880795  7071 ts_tablet_manager.cc:1403] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.076s	user 0.027s	sys 0.000s
I20260709 13:08:30.894941  7071 raft_consensus.cc:359] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:30.889009  5815 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.130:39429, user_credentials={real_user=slave}} blocked reactor thread for 47850us
I20260709 13:08:30.900610  7071 raft_consensus.cc:740] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.901535  7071 consensus_queue.cc:260] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:30.904609  7071 ts_tablet_manager.cc:1434] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:08:30.905916  7071 tablet_bootstrap.cc:492] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:30.956473  7071 tablet_bootstrap.cc:492] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:30.957607  7071 tablet_bootstrap.cc:492] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:30.958359  7071 ts_tablet_manager.cc:1403] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.004s
W20260709 13:08:30.960822  5888 consensus_peers.cc:597] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:30.961576  7071 raft_consensus.cc:359] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:30.970431  7071 raft_consensus.cc:740] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.971266  7071 consensus_queue.cc:260] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:30.978185  7071 ts_tablet_manager.cc:1434] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:08:30.980649  7071 tablet_bootstrap.cc:492] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:31.003888  7114 raft_consensus.cc:3060] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.018883  7112 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.056555  7153 heartbeater.cc:344] Connected to a master server at 127.27.71.190:45203
I20260709 13:08:31.057130  7153 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:31.058162  7153 heartbeater.cc:507] Master 127.27.71.190:45203 requested a full tablet report, sending...
W20260709 13:08:31.071444  5813 consensus_peers.cc:597] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:31.103428  7161 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.104188  7161 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:31.115432  7071 tablet_bootstrap.cc:492] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:31.121568  7071 tablet_bootstrap.cc:492] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:31.124070  7071 ts_tablet_manager.cc:1403] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.144s	user 0.024s	sys 0.001s
I20260709 13:08:31.116019  7161 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:31.144671  7071 raft_consensus.cc:359] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:31.154579  7071 raft_consensus.cc:740] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:31.158775  7071 consensus_queue.cc:260] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:31.162883  7161 raft_consensus.cc:493] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.163463  7161 raft_consensus.cc:515] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:31.178404  7161 leader_election.cc:290] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:31.187583  7112 raft_consensus.cc:3060] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.195192  7071 ts_tablet_manager.cc:1434] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.001s
I20260709 13:08:31.203960  7111 raft_consensus.cc:3060] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.204768  7071 tablet_bootstrap.cc:492] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:31.209582  7161 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.210260  7161 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:31.212786  7161 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:31.271203  7114 raft_consensus.cc:3060] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.272048  5845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:31.274811  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e" is_pre_election: true
I20260709 13:08:31.301569  5598 ts_manager.cc:194] Re-registered known tserver with Master: d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
W20260709 13:08:31.311070  5888 consensus_peers.cc:597] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:31.314291  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
I20260709 13:08:31.330336  7115 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:08:31.340116  7108 raft_consensus.cc:3060] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:31.378240  5739 consensus_peers.cc:597] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:31.379161  5739 consensus_peers.cc:597] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:31.384467  5769 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:31.387156  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8842506bb6124bf9adf9b13c6300c796" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:31.395992  7067 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:31.397015  7161 raft_consensus.cc:2749] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:08:31.398244  5739 consensus_peers.cc:597] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:31.401515  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" is_pre_election: true
I20260709 13:08:31.404177  7067 leader_election.cc:304] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 52b726da57c24dd2aa72488ebe2ff21e
I20260709 13:08:31.405714  7161 raft_consensus.cc:2749] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:31.412099  5598 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36690
I20260709 13:08:31.424961  7067 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9952df48cf245608db9181dedad0aeb; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06
I20260709 13:08:31.426235  7161 raft_consensus.cc:2749] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:31.427565  7153 heartbeater.cc:499] Master 127.27.71.190:45203 was elected leader, sending a full tablet report...
I20260709 13:08:31.457382  7071 tablet_bootstrap.cc:492] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:08:31.475162  5739 consensus_peers.cc:597] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:08:31.482849  5813 consensus_peers.cc:597] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:31.492388  7071 tablet_bootstrap.cc:492] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:31.493372  7071 ts_tablet_manager.cc:1403] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.289s	user 0.017s	sys 0.014s
I20260709 13:08:31.502701  7071 raft_consensus.cc:359] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:31.503712  7071 raft_consensus.cc:740] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:31.504719  7071 consensus_queue.cc:260] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:31.506932  7071 ts_tablet_manager.cc:1434] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:08:31.508137  7071 tablet_bootstrap.cc:492] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:31.524298  5739 consensus_peers.cc:597] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:31.541528  7071 tablet_bootstrap.cc:492] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:31.542716  7071 tablet_bootstrap.cc:492] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:31.543484  7071 ts_tablet_manager.cc:1403] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.004s
W20260709 13:08:31.544819  5888 consensus_peers.cc:597] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:31.550000  5813 consensus_peers.cc:597] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:31.569793  7071 raft_consensus.cc:359] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:31.570690  7071 raft_consensus.cc:740] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:31.572494  7071 consensus_queue.cc:260] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:31.575114  7071 ts_tablet_manager.cc:1434] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:08:31.584236  7071 tablet_bootstrap.cc:492] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:31.627300  7071 tablet_bootstrap.cc:492] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:31.628180  7071 tablet_bootstrap.cc:492] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:31.628756  7071 ts_tablet_manager.cc:1403] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.025s
I20260709 13:08:31.632383  7071 raft_consensus.cc:359] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:31.633085  7071 raft_consensus.cc:740] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:31.633816  7071 consensus_queue.cc:260] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:31.635774  7071 ts_tablet_manager.cc:1434] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:08:31.637223  7071 tablet_bootstrap.cc:492] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:31.645483  5813 consensus_peers.cc:597] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:31.683355  5813 consensus_peers.cc:597] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:31.824862  5813 consensus_peers.cc:597] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:31.876835  7071 tablet_bootstrap.cc:492] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:31.878029  7071 tablet_bootstrap.cc:492] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:31.878829  7071 ts_tablet_manager.cc:1403] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.242s	user 0.072s	sys 0.126s
I20260709 13:08:31.882084  7071 raft_consensus.cc:359] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:31.882886  7071 raft_consensus.cc:740] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:31.883710  7071 consensus_queue.cc:260] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
W20260709 13:08:31.884960  5888 consensus_peers.cc:597] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:31.891008  5888 consensus_peers.cc:597] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:31.892360  5888 consensus_peers.cc:597] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:31.886366  7071 ts_tablet_manager.cc:1434] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:08:31.893915  7071 tablet_bootstrap.cc:492] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:31.953598  5888 consensus_peers.cc:597] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:31.968036  5888 consensus_peers.cc:597] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:31.979993  5888 consensus_peers.cc:597] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:32.033592  7111 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:08:32.132742  5739 consensus_peers.cc:597] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:32.158854  5813 consensus_peers.cc:597] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:32.183053  7071 tablet_bootstrap.cc:492] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=1 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:32.189654  7071 tablet_bootstrap.cc:492] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:32.190415  7071 ts_tablet_manager.cc:1403] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.297s	user 0.101s	sys 0.130s
W20260709 13:08:32.192437  5739 consensus_peers.cc:597] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:32.193486  7071 raft_consensus.cc:359] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:32.194308  7071 raft_consensus.cc:740] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:32.195072  7071 consensus_queue.cc:260] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:32.199047  7071 ts_tablet_manager.cc:1434] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:08:32.200080  7071 tablet_bootstrap.cc:492] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:32.220777  7071 tablet_bootstrap.cc:492] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:32.221849  7071 tablet_bootstrap.cc:492] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:32.222661  7071 ts_tablet_manager.cc:1403] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.000s
I20260709 13:08:32.306098  7071 raft_consensus.cc:359] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:32.308620  7071 raft_consensus.cc:740] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:32.309886  7071 consensus_queue.cc:260] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
W20260709 13:08:32.317121  5739 consensus_peers.cc:597] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:32.317117  7071 ts_tablet_manager.cc:1434] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.094s	user 0.009s	sys 0.000s
I20260709 13:08:32.320261  7071 tablet_bootstrap.cc:492] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:32.320606  5813 consensus_peers.cc:597] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:32.330181  5813 consensus_peers.cc:597] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:32.340327  5739 consensus_peers.cc:597] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:08:32.409307  5816 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 114559us
I20260709 13:08:32.518262  7071 tablet_bootstrap.cc:492] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:32.519430  7071 tablet_bootstrap.cc:492] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:32.520358  7071 ts_tablet_manager.cc:1403] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.201s	user 0.039s	sys 0.082s
I20260709 13:08:32.524261  7071 raft_consensus.cc:359] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:32.525343  7071 raft_consensus.cc:740] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
W20260709 13:08:32.526862  5813 consensus_peers.cc:597] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:08:32.526664  7071 consensus_queue.cc:260] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:32.562441  7071 ts_tablet_manager.cc:1434] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.001s
I20260709 13:08:32.563951  7071 tablet_bootstrap.cc:492] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:32.616104  7071 tablet_bootstrap.cc:492] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:32.617480  7071 tablet_bootstrap.cc:492] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:32.618321  7071 ts_tablet_manager.cc:1403] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.020s
I20260709 13:08:32.621709  7071 raft_consensus.cc:359] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:32.622584  7071 raft_consensus.cc:740] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:32.623513  7071 consensus_queue.cc:260] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:32.626358  7071 ts_tablet_manager.cc:1434] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:08:32.627521  7071 tablet_bootstrap.cc:492] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:32.655229  7071 tablet_bootstrap.cc:492] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:32.656430  7071 tablet_bootstrap.cc:492] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:32.657186  7071 ts_tablet_manager.cc:1403] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.011s
I20260709 13:08:32.661893  7071 raft_consensus.cc:359] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:32.664525  7071 raft_consensus.cc:740] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:32.665431  7071 consensus_queue.cc:260] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:32.667443  7071 ts_tablet_manager.cc:1434] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:08:32.670288  7071 tablet_bootstrap.cc:492] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:32.705133  7071 tablet_bootstrap.cc:492] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:32.706318  7071 tablet_bootstrap.cc:492] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:32.707091  7071 ts_tablet_manager.cc:1403] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.008s
I20260709 13:08:32.709909  7071 raft_consensus.cc:359] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:32.710690  7071 raft_consensus.cc:740] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:32.711486  7071 consensus_queue.cc:260] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:32.713984  7071 ts_tablet_manager.cc:1434] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:08:32.715058  7071 tablet_bootstrap.cc:492] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:32.812505  7071 tablet_bootstrap.cc:492] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:32.813608  7071 tablet_bootstrap.cc:492] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:32.814366  7071 ts_tablet_manager.cc:1403] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.100s	user 0.043s	sys 0.025s
I20260709 13:08:32.817092  7071 raft_consensus.cc:359] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:32.817857  7071 raft_consensus.cc:740] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:32.819399  7071 consensus_queue.cc:260] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:32.824635  7071 ts_tablet_manager.cc:1434] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:08:32.825788  7071 tablet_bootstrap.cc:492] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:32.873440  7071 tablet_bootstrap.cc:492] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:32.874773  7071 tablet_bootstrap.cc:492] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:32.875500  7071 ts_tablet_manager.cc:1403] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.050s	user 0.019s	sys 0.005s
I20260709 13:08:32.878361  7071 raft_consensus.cc:359] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:32.879146  7071 raft_consensus.cc:740] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:32.880055  7071 consensus_queue.cc:260] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:32.883822  7071 ts_tablet_manager.cc:1434] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:08:32.884953  7071 tablet_bootstrap.cc:492] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:32.912873  7071 tablet_bootstrap.cc:492] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:32.913996  7071 tablet_bootstrap.cc:492] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:32.914808  7071 ts_tablet_manager.cc:1403] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.030s	user 0.025s	sys 0.002s
I20260709 13:08:32.918956  7071 raft_consensus.cc:359] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:32.919881  7071 raft_consensus.cc:740] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:32.920881  7071 consensus_queue.cc:260] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:32.925463  7071 ts_tablet_manager.cc:1434] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:08:32.926633  7071 tablet_bootstrap.cc:492] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:32.950062  7071 tablet_bootstrap.cc:492] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:32.951251  7071 tablet_bootstrap.cc:492] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:32.952040  7071 ts_tablet_manager.cc:1403] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.007s
I20260709 13:08:32.954859  7071 raft_consensus.cc:359] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:32.955638  7071 raft_consensus.cc:740] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:32.956485  7071 consensus_queue.cc:260] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:32.958302  7071 ts_tablet_manager.cc:1434] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:08:32.959290  7071 tablet_bootstrap.cc:492] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:32.996475  7111 raft_consensus.cc:3060] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:33.132125  7071 tablet_bootstrap.cc:492] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:33.133242  7071 tablet_bootstrap.cc:492] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:33.133991  7071 ts_tablet_manager.cc:1403] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.175s	user 0.028s	sys 0.075s
I20260709 13:08:33.136909  7071 raft_consensus.cc:359] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:33.137657  7071 raft_consensus.cc:740] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:33.138487  7071 consensus_queue.cc:260] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:33.150672  7071 ts_tablet_manager.cc:1434] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:08:33.151700  7071 tablet_bootstrap.cc:492] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:33.218679  7071 tablet_bootstrap.cc:492] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=1 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:33.220031  7071 tablet_bootstrap.cc:492] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:33.220829  7071 ts_tablet_manager.cc:1403] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.069s	user 0.021s	sys 0.015s
I20260709 13:08:33.223564  7071 raft_consensus.cc:359] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:33.224186  7071 raft_consensus.cc:740] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:33.224856  7071 consensus_queue.cc:260] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:33.230201  7071 ts_tablet_manager.cc:1434] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:08:33.231247  7071 tablet_bootstrap.cc:492] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:33.286598  7071 tablet_bootstrap.cc:492] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:33.287770  7071 tablet_bootstrap.cc:492] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:33.288604  7071 ts_tablet_manager.cc:1403] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.058s	user 0.028s	sys 0.006s
I20260709 13:08:33.291538  7071 raft_consensus.cc:359] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:33.292347  7071 raft_consensus.cc:740] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:33.293118  7071 consensus_queue.cc:260] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:33.311784  7071 ts_tablet_manager.cc:1434] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:08:33.314312  7071 tablet_bootstrap.cc:492] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:33.366122  7071 tablet_bootstrap.cc:492] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:33.371984  7109 raft_consensus.cc:3060] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:33.396039  7108 raft_consensus.cc:3060] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:33.396777  7071 tablet_bootstrap.cc:492] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:33.397675  7071 ts_tablet_manager.cc:1403] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.084s	user 0.020s	sys 0.000s
I20260709 13:08:33.401018  7071 raft_consensus.cc:359] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:33.403149  7071 raft_consensus.cc:740] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:33.404102  7071 consensus_queue.cc:260] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:33.406450  7071 ts_tablet_manager.cc:1434] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:08:33.407459  7071 tablet_bootstrap.cc:492] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:33.499858  7071 tablet_bootstrap.cc:492] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:33.508185  7071 tablet_bootstrap.cc:492] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:33.509050  7071 ts_tablet_manager.cc:1403] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.102s	user 0.014s	sys 0.009s
I20260709 13:08:33.513180  7071 raft_consensus.cc:359] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:33.513967  7071 raft_consensus.cc:740] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:33.514747  7071 consensus_queue.cc:260] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:33.522648  7071 ts_tablet_manager.cc:1434] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
W20260709 13:08:33.523639  5888 consensus_peers.cc:597] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 31: this message will repeat every 5th retry.
I20260709 13:08:33.524024  7071 tablet_bootstrap.cc:492] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:33.583243  7071 tablet_bootstrap.cc:492] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:33.584396  7071 tablet_bootstrap.cc:492] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:33.585151  7071 ts_tablet_manager.cc:1403] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.061s	user 0.019s	sys 0.009s
I20260709 13:08:33.588495  7071 raft_consensus.cc:359] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:33.589310  7071 raft_consensus.cc:740] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:33.590189  7071 consensus_queue.cc:260] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:33.592552  7071 ts_tablet_manager.cc:1434] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:08:33.593250  7113 raft_consensus.cc:3060] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:33.633224  7071 tablet_bootstrap.cc:492] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:33.635960  5888 consensus_peers.cc:597] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 31: this message will repeat every 5th retry.
I20260709 13:08:33.678390  7071 tablet_bootstrap.cc:492] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:33.679790  7071 tablet_bootstrap.cc:492] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:33.680711  7071 ts_tablet_manager.cc:1403] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.048s	user 0.030s	sys 0.005s
I20260709 13:08:33.684651  7071 raft_consensus.cc:359] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:33.687191  7071 raft_consensus.cc:740] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:33.708130  7071 consensus_queue.cc:260] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:33.711983  7071 ts_tablet_manager.cc:1434] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:08:33.713490  7071 tablet_bootstrap.cc:492] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:33.738173  7071 tablet_bootstrap.cc:492] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:33.739418  7071 tablet_bootstrap.cc:492] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:33.740180  7071 ts_tablet_manager.cc:1403] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.010s
I20260709 13:08:33.742733  7071 raft_consensus.cc:359] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:33.743459  7071 raft_consensus.cc:740] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:33.744323  7071 consensus_queue.cc:260] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:33.756613  7071 ts_tablet_manager.cc:1434] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:08:33.758627  7071 tablet_bootstrap.cc:492] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:33.763675  5739 consensus_peers.cc:597] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:33.814299  5739 consensus_peers.cc:597] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 26: this message will repeat every 5th retry.
W20260709 13:08:33.815372  5813 consensus_peers.cc:597] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 31: this message will repeat every 5th retry.
W20260709 13:08:33.932479  5739 consensus_peers.cc:597] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 31: this message will repeat every 5th retry.
I20260709 13:08:34.007864  7071 tablet_bootstrap.cc:492] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:34.016892  7071 tablet_bootstrap.cc:492] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:34.017680  7071 ts_tablet_manager.cc:1403] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.259s	user 0.099s	sys 0.120s
I20260709 13:08:34.020570  7071 raft_consensus.cc:359] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:34.021385  7071 raft_consensus.cc:740] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:34.022338  7071 consensus_queue.cc:260] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:34.043856  7071 ts_tablet_manager.cc:1434] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 13:08:34.045347  7071 tablet_bootstrap.cc:492] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:34.086246  7071 tablet_bootstrap.cc:492] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:34.087481  7071 tablet_bootstrap.cc:492] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:34.089080  7071 ts_tablet_manager.cc:1403] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.021s
I20260709 13:08:34.094328  7071 raft_consensus.cc:359] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:34.095227  7071 raft_consensus.cc:740] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:34.096299  7071 consensus_queue.cc:260] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:34.099346  7071 ts_tablet_manager.cc:1434] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.006s
I20260709 13:08:34.100745  7071 tablet_bootstrap.cc:492] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:34.150127  7071 tablet_bootstrap.cc:492] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:34.151582  7071 tablet_bootstrap.cc:492] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:34.152439  7071 ts_tablet_manager.cc:1403] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.052s	user 0.024s	sys 0.006s
I20260709 13:08:34.157809  7071 raft_consensus.cc:359] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:34.158792  7071 raft_consensus.cc:740] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:34.159881  7071 consensus_queue.cc:260] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:34.162331  7071 ts_tablet_manager.cc:1434] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 13:08:34.163277  7071 tablet_bootstrap.cc:492] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:34.232923  7071 tablet_bootstrap.cc:492] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:34.233991  7071 tablet_bootstrap.cc:492] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:34.234717  7071 ts_tablet_manager.cc:1403] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.072s	user 0.020s	sys 0.012s
I20260709 13:08:34.237947  7071 raft_consensus.cc:359] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:34.238706  7071 raft_consensus.cc:740] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:34.239418  7071 consensus_queue.cc:260] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:34.246723  7071 ts_tablet_manager.cc:1434] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:08:34.255765  7071 tablet_bootstrap.cc:492] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
I20260709 13:08:34.304733  7071 tablet_bootstrap.cc:492] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:34.305934  7071 tablet_bootstrap.cc:492] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:34.306715  7071 ts_tablet_manager.cc:1403] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.051s	user 0.019s	sys 0.003s
I20260709 13:08:34.309597  7071 raft_consensus.cc:359] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:34.310282  7071 raft_consensus.cc:740] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:34.311084  7071 consensus_queue.cc:260] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:34.323318  7071 ts_tablet_manager.cc:1434] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:08:34.327499  7071 tablet_bootstrap.cc:492] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: Bootstrap starting.
W20260709 13:08:34.360916  5888 consensus_peers.cc:597] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 26: this message will repeat every 5th retry.
I20260709 13:08:34.403604  7071 tablet_bootstrap.cc:492] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:34.404702  7071 tablet_bootstrap.cc:492] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: Bootstrap complete.
I20260709 13:08:34.405494  7071 ts_tablet_manager.cc:1403] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: Time spent bootstrapping tablet: real 0.078s	user 0.033s	sys 0.027s
I20260709 13:08:34.408417  7071 raft_consensus.cc:359] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:34.409165  7071 raft_consensus.cc:740] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Initialized, Role: FOLLOWER
I20260709 13:08:34.410020  7071 consensus_queue.cc:260] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:34.412653  7071 ts_tablet_manager.cc:1434] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:08:34.608287  7111 raft_consensus.cc:3060] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:34.840119  7109 raft_consensus.cc:3060] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:36.671962  5797 debug-util.cc:398] Leaking SignalData structure 0x7b0800302c20 after lost signal to thread 27937
W20260709 13:08:36.673177  5797 debug-util.cc:398] Leaking SignalData structure 0x7b0800149d00 after lost signal to thread 5633
W20260709 13:08:36.674191  5797 debug-util.cc:398] Leaking SignalData structure 0x7b08000cfe40 after lost signal to thread 5800
W20260709 13:08:36.675048  5797 debug-util.cc:398] Leaking SignalData structure 0x7b08002a6960 after lost signal to thread 5875
W20260709 13:08:36.675894  5797 debug-util.cc:398] Leaking SignalData structure 0x7b08000c38c0 after lost signal to thread 5949
W20260709 13:08:36.677307  5797 debug-util.cc:398] Leaking SignalData structure 0x7b080032c720 after lost signal to thread 7151
W20260709 13:08:36.938558  5630 debug-util.cc:398] Leaking SignalData structure 0x7b0800381da0 after lost signal to thread 27937
W20260709 13:08:36.940025  5630 debug-util.cc:398] Leaking SignalData structure 0x7b08002f7400 after lost signal to thread 5633
W20260709 13:08:36.941666  5630 debug-util.cc:398] Leaking SignalData structure 0x7b08001bc260 after lost signal to thread 5800
W20260709 13:08:36.943076  5630 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc080 after lost signal to thread 5875
W20260709 13:08:36.944463  5630 debug-util.cc:398] Leaking SignalData structure 0x7b0800327ba0 after lost signal to thread 5949
W20260709 13:08:36.946102  5630 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1980 after lost signal to thread 7151
I20260709 13:08:41.091691 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:41.162868  5767 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512"
dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:53026
I20260709 13:08:41.163609  5767 raft_consensus.cc:606] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:41.176910  5917 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5"
dest_uuid: "792907f253c744aa9392e8d572f88e06"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:39686
I20260709 13:08:41.177459  5917 raft_consensus.cc:606] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:41.179880  5767 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30f9347b11ec44d1b07c475379a51881"
dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:53026
I20260709 13:08:41.180521  5767 raft_consensus.cc:606] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:41.193703  5844 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c"
dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:57310
I20260709 13:08:41.194379  5844 raft_consensus.cc:606] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:41.197500  5767 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2"
dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:53026
I20260709 13:08:41.198052  5767 raft_consensus.cc:606] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:41.200716  5767 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8"
dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:53026
I20260709 13:08:41.201336  5767 raft_consensus.cc:606] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:41.204010  5767 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d"
dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:53026
I20260709 13:08:41.204730  5767 raft_consensus.cc:606] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 7 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:41.206070 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:08:41.207607 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:41.281463  7251 raft_consensus.cc:993] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.282343  7257 raft_consensus.cc:1081] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.284327  7256 raft_consensus.cc:993] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.284957  7249 raft_consensus.cc:1081] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.284776  7108 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54206
I20260709 13:08:41.285588  7108 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:41.286016  7108 raft_consensus.cc:3060] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.286726  7113 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54190
I20260709 13:08:41.287345  7113 raft_consensus.cc:493] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:41.287729  7113 raft_consensus.cc:3060] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.291185  7108 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:41.292953  7108 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:41.294211  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:41.294104  7113 raft_consensus.cc:515] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:41.294497  5922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "400faa6e9aed42f19cda604767b955c2" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:41.294982  5771 raft_consensus.cc:3055] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:41.295197  5922 raft_consensus.cc:3060] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.295415  5771 raft_consensus.cc:740] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:41.296279  7113 leader_election.cc:290] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:41.296350  5771 consensus_queue.cc:260] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:41.297389  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:41.297722  5771 raft_consensus.cc:3060] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.298008  5844 raft_consensus.cc:3055] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:41.298368  5844 raft_consensus.cc:740] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:41.299521  5844 consensus_queue.cc:260] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:41.300777  5844 raft_consensus.cc:3060] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.301075  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ada8ac4d923408997f9511c6910789c" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:41.301676  5917 raft_consensus.cc:3060] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.305713  5771 raft_consensus.cc:2468] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:41.308430  7067 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:41.309132  5844 raft_consensus.cc:2468] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:41.310813  7068 leader_election.cc:304] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:41.313122  7265 raft_consensus.cc:2804] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:41.316159  5917 raft_consensus.cc:2468] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:41.325616  5922 raft_consensus.cc:2468] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:41.333194  7264 raft_consensus.cc:2804] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:41.333523  7265 raft_consensus.cc:697] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:41.334744  7265 consensus_queue.cc:237] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:41.336007  7264 raft_consensus.cc:697] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:41.337340  7264 consensus_queue.cc:237] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:41.378558  5595 catalog_manager.cc:5697] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 3, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 3 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.383133  7259 raft_consensus.cc:993] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.383751  7259 raft_consensus.cc:1081] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.386435  7113 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "30f9347b11ec44d1b07c475379a51881"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54206
I20260709 13:08:41.386996  7113 raft_consensus.cc:493] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:41.387377  7113 raft_consensus.cc:3060] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.395516  7113 raft_consensus.cc:515] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:41.400369  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f9347b11ec44d1b07c475379a51881" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:41.401002  5844 raft_consensus.cc:3060] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.400635  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30f9347b11ec44d1b07c475379a51881" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:41.401682  5767 raft_consensus.cc:3055] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:41.402119  5767 raft_consensus.cc:740] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:41.403436  7113 leader_election.cc:290] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:41.403343  5767 consensus_queue.cc:260] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:41.414261  5767 raft_consensus.cc:3060] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.414795  5844 raft_consensus.cc:2468] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:41.415970  7068 leader_election.cc:304] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:41.416726  7264 raft_consensus.cc:2804] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:41.417268  7264 raft_consensus.cc:697] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:41.417668  7257 raft_consensus.cc:993] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.418169  7259 raft_consensus.cc:1081] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.418511  7264 consensus_queue.cc:237] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:41.420091  7106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54206
I20260709 13:08:41.420759  7106 raft_consensus.cc:493] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:41.421181  7106 raft_consensus.cc:3060] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.425908  5767 raft_consensus.cc:2468] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:41.433305  7106 raft_consensus.cc:515] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:41.437139  7106 leader_election.cc:290] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:41.437362  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:41.437759  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e982efc662b84721b51ef9101f0cf5a8" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:41.438153  5767 raft_consensus.cc:3055] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:41.438510  5844 raft_consensus.cc:3060] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.438589  5767 raft_consensus.cc:740] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:41.439646  5767 consensus_queue.cc:260] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:41.441934  5767 raft_consensus.cc:3060] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.446552  5844 raft_consensus.cc:2468] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:41.448239  7068 leader_election.cc:304] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:41.449084  7264 raft_consensus.cc:2804] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:41.449568  7264 raft_consensus.cc:697] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:41.450187  5767 raft_consensus.cc:2468] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:41.450408  7264 consensus_queue.cc:237] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:41.474068  5595 catalog_manager.cc:5697] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 3, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 3 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.479382  5595 catalog_manager.cc:5697] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 3, leader changed from 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 3 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:41.481545  5595 catalog_manager.cc:5697] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 3, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 3 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.506857  7276 raft_consensus.cc:993] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.507640  7239 raft_consensus.cc:1081] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.510814  7106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54196
I20260709 13:08:41.511627  7106 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:41.512181  7106 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:41.519726  7106 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:41.521627  7106 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:41.522308  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:41.522644  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12df4d7a5aad4ece999b2b3722b294e5" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:41.522992  5917 raft_consensus.cc:3055] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:41.523097  5771 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:41.523662  5917 raft_consensus.cc:740] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:41.524565  5917 consensus_queue.cc:260] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:41.526090  5917 raft_consensus.cc:3060] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:41.531044  5771 raft_consensus.cc:2468] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 4.
I20260709 13:08:41.532505  7067 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:41.533354  7264 raft_consensus.cc:2804] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:41.533394  5917 raft_consensus.cc:2468] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 4.
I20260709 13:08:41.534051  7264 raft_consensus.cc:697] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 4 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:41.535008  7264 consensus_queue.cc:237] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:41.549376  5595 catalog_manager.cc:5697] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 3 to 4, leader changed from 792907f253c744aa9392e8d572f88e06 (127.27.71.132) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 4 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.554211  7251 raft_consensus.cc:993] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.554778  7257 raft_consensus.cc:1081] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.562207  7108 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54206
I20260709 13:08:41.563201  7108 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:41.563616  7108 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:41.569762  7108 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:41.571389  7108 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 election: Requested vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:41.572088  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:41.572397  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e03b5d529f34213b58b61ece24ce512" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:41.572712  5917 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:41.573050  5767 raft_consensus.cc:3055] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:41.573388  5767 raft_consensus.cc:740] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:41.573975  5767 consensus_queue.cc:260] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:41.574671  5767 raft_consensus.cc:3060] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:41.578188  5917 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 5.
I20260709 13:08:41.579430  7068 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:41.580367  7264 raft_consensus.cc:2804] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:41.580447  5767 raft_consensus.cc:2468] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 5.
I20260709 13:08:41.580861  7264 raft_consensus.cc:697] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 5 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:41.581841  7264 consensus_queue.cc:237] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:41.591683  5595 catalog_manager.cc:5697] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 4 to 5, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 5 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.710175  7257 raft_consensus.cc:993] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.710755  7279 raft_consensus.cc:1081] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 7 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:41.712263  7108 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54206
I20260709 13:08:41.712788  7108 raft_consensus.cc:493] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:41.713058  7108 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:41.719815  7108 raft_consensus.cc:515] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:41.721714  7108 leader_election.cc:290] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 8 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:41.722378  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:41.723028  5767 raft_consensus.cc:3055] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:41.723320  5767 raft_consensus.cc:740] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:41.723730  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbbd1771e7664e0cb3441c84672d037d" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:41.724349  5851 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:41.724175  5767 consensus_queue.cc:260] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:41.725874  5767 raft_consensus.cc:3060] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:41.731868  5851 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 8.
I20260709 13:08:41.732892  5767 raft_consensus.cc:2468] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 8.
I20260709 13:08:41.733153  7068 leader_election.cc:304] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:41.734025  7264 raft_consensus.cc:2804] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:41.734516  7264 raft_consensus.cc:697] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 8 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:41.735244  7264 consensus_queue.cc:237] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:41.746433  5595 catalog_manager.cc:5697] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 7 to 8, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 8 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.787415  5917 raft_consensus.cc:1275] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:41.788724  7264 consensus_queue.cc:1048] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:41.800375  5767 raft_consensus.cc:1275] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:41.802544  7265 consensus_queue.cc:1048] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:41.809643  5771 raft_consensus.cc:1275] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:41.811487  7265 consensus_queue.cc:1048] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:41.824863  5851 raft_consensus.cc:1275] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:41.826534  7268 consensus_queue.cc:1048] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:41.918483  5917 raft_consensus.cc:1275] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:41.920149  7268 consensus_queue.cc:1048] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:41.929885  5851 raft_consensus.cc:1275] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:41.931898  7265 consensus_queue.cc:1048] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:41.986796  5917 raft_consensus.cc:1275] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:08:41.988269  7265 consensus_queue.cc:1048] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:41.990834  5851 raft_consensus.cc:1275] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:41.992367  7264 consensus_queue.cc:1048] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:41.999846  5767 raft_consensus.cc:1275] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:08:42.006637  5767 raft_consensus.cc:1275] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:42.007158  7264 consensus_queue.cc:1048] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:42.014344  7268 consensus_queue.cc:1048] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:42.022390  5767 raft_consensus.cc:1275] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:42.026361  7265 consensus_queue.cc:1048] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:42.066007  5917 raft_consensus.cc:1275] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:42.068416  7264 consensus_queue.cc:1048] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:42.186420  5771 raft_consensus.cc:1275] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 8 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:08:42.187851  7268 consensus_queue.cc:1048] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:42.196503  5851 raft_consensus.cc:1275] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 8 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:08:42.197727  7268 consensus_queue.cc:1048] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:42.208755 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:42.269934 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:42.306936  5844 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df"
dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
mode: GRACEFUL
new_leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
 from {username='slave'} at 127.0.0.1:57310
I20260709 13:08:42.307569  5844 raft_consensus.cc:606] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Received request to transfer leadership to TS 373ee2f4cc9d4b86b49ada724f55e16a
I20260709 13:08:42.310305  5771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb"
dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:53026
I20260709 13:08:42.310882  5771 raft_consensus.cc:606] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:42.313714  5771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93"
dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:53026
I20260709 13:08:42.314210  5771 raft_consensus.cc:606] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:42.316535  5917 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45"
dest_uuid: "792907f253c744aa9392e8d572f88e06"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:39686
I20260709 13:08:42.317143  5917 raft_consensus.cc:606] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:42.319962  5851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e"
dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:57310
I20260709 13:08:42.320521  5851 raft_consensus.cc:606] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:42.323307  5851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202"
dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:57310
I20260709 13:08:42.323908  5851 raft_consensus.cc:606] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:42.326010 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:08:42.327802 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:42.399969  7183 raft_consensus.cc:993] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:42.401268  7183 raft_consensus.cc:1081] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:42.404254  7113 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54196
I20260709 13:08:42.405493  7113 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.406174  7113 raft_consensus.cc:3060] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:42.412762  7251 raft_consensus.cc:993] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:42.413872  7257 raft_consensus.cc:1081] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:42.416893  7106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54206
I20260709 13:08:42.417904  7106 raft_consensus.cc:493] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.418447  7106 raft_consensus.cc:3060] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.431588  7113 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:42.434113  7106 raft_consensus.cc:515] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.435473  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:42.436038  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:42.436707  5844 raft_consensus.cc:3060] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:42.437762  7106 leader_election.cc:290] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:42.439529  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:42.439113  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b6cadc1b9d441983391c6e69f42beb" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:42.440557  5851 raft_consensus.cc:3060] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.440898  5771 raft_consensus.cc:3055] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:42.441417  5771 raft_consensus.cc:740] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:42.436800  5917 raft_consensus.cc:3055] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:42.443615  5917 raft_consensus.cc:740] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 792907f253c744aa9392e8d572f88e06, State: Running, Role: LEADER
I20260709 13:08:42.448137  5771 consensus_queue.cc:260] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.434180  7113 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:42.445252  5917 consensus_queue.cc:260] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:42.462843  5917 raft_consensus.cc:3060] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:42.463615  5851 raft_consensus.cc:2468] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:42.451083  5771 raft_consensus.cc:3060] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.465533  7068 leader_election.cc:304] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:42.463752  5844 raft_consensus.cc:2468] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:42.466775  7264 raft_consensus.cc:2804] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:42.467409  7264 raft_consensus.cc:697] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 2 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:42.469020  7264 consensus_queue.cc:237] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.476990  7068 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:42.496946  5917 raft_consensus.cc:2468] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:42.503477  5771 raft_consensus.cc:2468] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 2.
I20260709 13:08:42.511176  7305 raft_consensus.cc:2804] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:42.512591  7305 raft_consensus.cc:697] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:42.539731  7249 raft_consensus.cc:993] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:42.540462  7249 raft_consensus.cc:1081] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:42.542505  7113 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54190
I20260709 13:08:42.543134  7113 raft_consensus.cc:493] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.543583  7113 raft_consensus.cc:3060] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:42.545550  7305 consensus_queue.cc:237] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:42.567342  7113 raft_consensus.cc:515] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.567757  5598 catalog_manager.cc:5697] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 1 to 2, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 2 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.574859  7113 leader_election.cc:290] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:42.572556  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:42.577850  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ed8e83113241bda21a6e92ed10ac9e" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:42.578379  5767 raft_consensus.cc:3060] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:42.578683  5844 raft_consensus.cc:3055] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:42.579211  5844 raft_consensus.cc:740] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:42.580819  5844 consensus_queue.cc:260] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.583442  5844 raft_consensus.cc:3060] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:42.605909  5844 raft_consensus.cc:2468] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:42.607602  7068 leader_election.cc:304] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:42.610831  7305 raft_consensus.cc:2804] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:42.612912  7305 raft_consensus.cc:697] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:42.614082  5767 raft_consensus.cc:2468] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:42.614051  7305 consensus_queue.cc:237] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.659714  7279 raft_consensus.cc:993] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:42.660279  7257 raft_consensus.cc:1081] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:42.660780  5598 catalog_manager.cc:5697] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 3, leader changed from 792907f253c744aa9392e8d572f88e06 (127.27.71.132) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 3 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:42.661931  7106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54206
I20260709 13:08:42.662518  7106 raft_consensus.cc:493] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.662896  7106 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:42.663164  5598 catalog_manager.cc:5697] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 3, leader changed from 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 3 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.672305  7106 raft_consensus.cc:515] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.676282  7106 leader_election.cc:290] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:42.678747  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:42.679775  5844 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:42.680764  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "365d76e5c79a4b0b8b65ad7465deac93" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:42.681988  5771 raft_consensus.cc:3055] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:42.682396  5771 raft_consensus.cc:740] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:42.683271  5771 consensus_queue.cc:260] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.685949  5771 raft_consensus.cc:3060] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:42.688912  5844 raft_consensus.cc:2468] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 4.
I20260709 13:08:42.696460  5771 raft_consensus.cc:2468] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 4.
I20260709 13:08:42.697081  7068 leader_election.cc:304] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:42.698101  7305 raft_consensus.cc:2804] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:42.698629  7305 raft_consensus.cc:697] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 4 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:42.699805  7305 consensus_queue.cc:237] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.711774  5598 catalog_manager.cc:5697] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 3 to 4, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 4 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.830163  7267 raft_consensus.cc:993] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e: : Instructing follower 373ee2f4cc9d4b86b49ada724f55e16a to start an election
I20260709 13:08:42.830724  7267 raft_consensus.cc:1081] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Signalling peer 373ee2f4cc9d4b86b49ada724f55e16a to start an election
I20260709 13:08:42.832353  5771 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df"
dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
 from {username='slave'} at 127.0.0.1:60920
I20260709 13:08:42.832821  5771 raft_consensus.cc:493] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.833127  5771 raft_consensus.cc:3060] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.838991  5771 raft_consensus.cc:515] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:42.841545  5771 leader_election.cc:290] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:42.842437  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:42.842324  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098c5f84c14148eabebfe76f9ccfc1df" candidate_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:42.843153  5917 raft_consensus.cc:3060] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.843209  5844 raft_consensus.cc:3055] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:42.843643  5844 raft_consensus.cc:740] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:42.844501  5844 consensus_queue.cc:260] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:42.845842  5844 raft_consensus.cc:3060] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.850972  5917 raft_consensus.cc:2468] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:42.853538  7267 raft_consensus.cc:993] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:42.853358  5742 leader_election.cc:304] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, 792907f253c744aa9392e8d572f88e06; no voters: 
I20260709 13:08:42.853802  5844 raft_consensus.cc:2468] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373ee2f4cc9d4b86b49ada724f55e16a in term 2.
I20260709 13:08:42.854140  7267 raft_consensus.cc:1081] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:42.854621  7257 raft_consensus.cc:2804] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:42.855129  7257 raft_consensus.cc:697] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Becoming Leader. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:42.856576  7113 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54190
I20260709 13:08:42.856642  7257 consensus_queue.cc:237] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } }
I20260709 13:08:42.857259  7113 raft_consensus.cc:493] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.857802  7113 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.879281  5598 catalog_manager.cc:5697] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a reported cstate change: term changed from 1 to 2, leader changed from 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131) to 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130). New cstate: current_term: 2 leader_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.880636  7113 raft_consensus.cc:515] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.883709  7113 leader_election.cc:290] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:42.885229  5844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:42.885840  5844 raft_consensus.cc:3055] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:42.886138  5844 raft_consensus.cc:740] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:42.884547  5771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5051e38577e54ce780fad112f2fcd202" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:42.887176  5771 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.886978  5844 consensus_queue.cc:260] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.888411  5844 raft_consensus.cc:3060] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.896497  5771 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 5.
I20260709 13:08:42.896659  5844 raft_consensus.cc:2468] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 5.
I20260709 13:08:42.898955  7067 leader_election.cc:304] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:42.900211  7305 raft_consensus.cc:2804] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:42.900750  7305 raft_consensus.cc:697] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 5 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:42.901897  7305 consensus_queue.cc:237] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:42.916779  5598 catalog_manager.cc:5697] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 4 to 5, leader changed from 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 5 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.930804  5771 raft_consensus.cc:1275] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:42.932528  7305 consensus_queue.cc:1048] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:42.947201  5851 raft_consensus.cc:1275] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:42.948987  7307 consensus_queue.cc:1048] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:42.997761  5771 raft_consensus.cc:1275] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:42.999075  7307 consensus_queue.cc:1048] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:43.008886  5851 raft_consensus.cc:1275] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:43.025341  5844 raft_consensus.cc:1275] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:43.026554  7264 consensus_queue.cc:1048] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:43.029603  7305 consensus_queue.cc:1048] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:43.070611  5921 raft_consensus.cc:1275] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:43.072669  7307 consensus_queue.cc:1048] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:43.093837  5771 raft_consensus.cc:1275] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:43.095330  7307 consensus_queue.cc:1048] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:43.111303  5851 raft_consensus.cc:1275] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:43.113265  7305 consensus_queue.cc:1048] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:43.262437  5921 raft_consensus.cc:1275] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:43.263743  7257 consensus_queue.cc:1048] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:43.279848  5851 raft_consensus.cc:1275] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Refusing update from remote peer 373ee2f4cc9d4b86b49ada724f55e16a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:43.282105  7279 consensus_queue.cc:1048] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:43.328946 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:43.375456 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:43.424669  5767 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7"
dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:53026
I20260709 13:08:43.425496  5767 raft_consensus.cc:606] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:43.429100  5851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b"
dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
mode: GRACEFUL
new_leader_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:57310
I20260709 13:08:43.429630  5851 raft_consensus.cc:606] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Received request to transfer leadership to TS d9952df48cf245608db9181dedad0aeb
I20260709 13:08:43.431157 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:43.432689 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:43.466435  5851 raft_consensus.cc:1275] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:43.467756  7307 consensus_queue.cc:1048] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:43.479236  5767 raft_consensus.cc:1275] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:43.481884  7268 consensus_queue.cc:1048] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:43.630119  7267 raft_consensus.cc:993] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:43.630759  7298 raft_consensus.cc:1081] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:43.632241  7113 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54190
I20260709 13:08:43.632823  7113 raft_consensus.cc:493] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:43.633420  7113 raft_consensus.cc:3060] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:43.641433  7113 raft_consensus.cc:515] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:43.643095  7113 leader_election.cc:290] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Requested vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241)
I20260709 13:08:43.643901  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:43.644490  5851 raft_consensus.cc:3055] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:43.644491  5917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d76ed139f0d94b8484f5b5101634fb1b" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "792907f253c744aa9392e8d572f88e06"
I20260709 13:08:43.645143  5851 raft_consensus.cc:740] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52b726da57c24dd2aa72488ebe2ff21e, State: Running, Role: LEADER
I20260709 13:08:43.645370  5917 raft_consensus.cc:3060] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:43.646042  5851 consensus_queue.cc:260] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:43.647404  5851 raft_consensus.cc:3060] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:43.654073  5917 raft_consensus.cc:2468] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:43.654505  5851 raft_consensus.cc:2468] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 3.
I20260709 13:08:43.655354  7068 leader_election.cc:304] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:43.656213  7268 raft_consensus.cc:2804] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:43.656710  7268 raft_consensus.cc:697] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:43.657577  7268 consensus_queue.cc:237] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:43.668329  5598 catalog_manager.cc:5697] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 2 to 3, leader changed from 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 3 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:43.917444  7330 raft_consensus.cc:993] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a: : Instructing follower d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:43.917999  7279 raft_consensus.cc:1081] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Signalling peer d9952df48cf245608db9181dedad0aeb to start an election
I20260709 13:08:43.919744  7106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7"
dest_uuid: "d9952df48cf245608db9181dedad0aeb"
 from {username='slave'} at 127.0.0.1:54206
I20260709 13:08:43.920331  7106 raft_consensus.cc:493] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:43.920579  7106 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:43.926028  7106 raft_consensus.cc:515] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:43.928471  5767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "373ee2f4cc9d4b86b49ada724f55e16a"
I20260709 13:08:43.928920  7106 leader_election.cc:290] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 election: Requested vote from peers 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429), 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585)
I20260709 13:08:43.929062  5767 raft_consensus.cc:3055] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:43.929553  5767 raft_consensus.cc:740] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 373ee2f4cc9d4b86b49ada724f55e16a, State: Running, Role: LEADER
I20260709 13:08:43.930339  5767 consensus_queue.cc:260] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:43.931463  5767 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:43.939733  5767 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 5.
I20260709 13:08:43.945927  5851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bba78f968ad41538018aacddcc5d9d7" candidate_uuid: "d9952df48cf245608db9181dedad0aeb" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "52b726da57c24dd2aa72488ebe2ff21e"
I20260709 13:08:43.946609  5851 raft_consensus.cc:3060] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:43.948530  7067 leader_election.cc:304] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb; no voters: 
I20260709 13:08:43.949446  7264 raft_consensus.cc:2804] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:43.950018  7264 raft_consensus.cc:697] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 5 LEADER]: Becoming Leader. State: Replica: d9952df48cf245608db9181dedad0aeb, State: Running, Role: LEADER
I20260709 13:08:43.950918  7264 consensus_queue.cc:237] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } }
I20260709 13:08:43.957240  5851 raft_consensus.cc:2468] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9952df48cf245608db9181dedad0aeb in term 5.
I20260709 13:08:43.968374  5598 catalog_manager.cc:5697] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb reported cstate change: term changed from 4 to 5, leader changed from 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130) to d9952df48cf245608db9181dedad0aeb (127.27.71.129). New cstate: current_term: 5 leader_uuid: "d9952df48cf245608db9181dedad0aeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:44.099391  5921 raft_consensus.cc:1275] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:44.103188  7268 consensus_queue.cc:1048] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:44.117452  5851 raft_consensus.cc:1275] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:44.120905  7268 consensus_queue.cc:1048] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:44.433697 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:44.472606 27934 tablet_server.cc:179] TabletServer@127.27.71.129:40419 shutting down...
I20260709 13:08:44.472812  5844 raft_consensus.cc:1275] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:44.475363  7264 consensus_queue.cc:1048] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
W20260709 13:08:44.518605  5888 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 157 similar messages]
W20260709 13:08:44.552222  5888 consensus_peers.cc:597] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.583400  5767 raft_consensus.cc:1275] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Refusing update from remote peer d9952df48cf245608db9181dedad0aeb: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:08:44.585773  5888 consensus_peers.cc:597] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.571506  5742 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.132:45241, user_credentials={real_user=slave}} blocked reactor thread for 85330.9us
I20260709 13:08:44.593881  7350 consensus_queue.cc:1048] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:08:44.599502  5813 consensus_peers.cc:597] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.602562  5813 consensus_peers.cc:597] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.627317  5739 consensus_peers.cc:597] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.634131  5739 consensus_peers.cc:597] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.636677  5813 consensus_peers.cc:597] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.638659  5739 consensus_peers.cc:597] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.646219 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:08:44.647504  5739 consensus_peers.cc:597] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.649613 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.650902 27934 raft_consensus.cc:2243] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.651667 27934 raft_consensus.cc:2272] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.656052 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.657507 27934 raft_consensus.cc:2243] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.658802 27934 raft_consensus.cc:2272] T e982efc662b84721b51ef9101f0cf5a8 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.662676 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.663683 27934 raft_consensus.cc:2243] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.664831 27934 raft_consensus.cc:2272] T 40ba6b5dcd674422a95eeaad0efd1e45 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.668586 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.669469 27934 raft_consensus.cc:2243] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:44.669535  5739 consensus_peers.cc:597] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.670135 27934 raft_consensus.cc:2272] T 8cb2353dae2d41588ef00b83e6cc10d1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.673101 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.673797 27934 raft_consensus.cc:2243] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.674259 27934 raft_consensus.cc:2272] T 75595b8e97ef436992a8acba2c9724a7 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.676908 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.677666 27934 raft_consensus.cc:2243] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.678167 27934 raft_consensus.cc:2272] T 27bf52f47b87456c94c45846de53788d P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.681442 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.682322 27934 raft_consensus.cc:2243] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.683370 27934 raft_consensus.cc:2272] T 5051e38577e54ce780fad112f2fcd202 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.686618 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.687311 27934 raft_consensus.cc:2243] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.687798 27934 raft_consensus.cc:2272] T 9f08c26e76934d3c94ef7060e15caed0 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.690302 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.690905 27934 raft_consensus.cc:2243] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.691318 27934 raft_consensus.cc:2272] T 422b96951eda4c1c8b8d86612ac6c163 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.693564 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.694252 27934 raft_consensus.cc:2243] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.694726 27934 raft_consensus.cc:2272] T 759acc6b90aa41a1b6c9053bd20846c1 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.697155 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.697777 27934 raft_consensus.cc:2243] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.698199 27934 raft_consensus.cc:2272] T 106cdfafda414cf6981dbcaf1e28cc9d P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.701092 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.701704 27934 raft_consensus.cc:2243] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.702118 27934 raft_consensus.cc:2272] T de4f21d87306453b81a417014e41e4a4 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.704468 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.705078 27934 raft_consensus.cc:2243] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.705889 27934 raft_consensus.cc:2272] T d76ed139f0d94b8484f5b5101634fb1b P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.709136 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.710214 27934 raft_consensus.cc:2243] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.711205 27934 raft_consensus.cc:2272] T 32b6cadc1b9d441983391c6e69f42beb P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.713711 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.714280 27934 raft_consensus.cc:2243] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.714751 27934 raft_consensus.cc:2272] T 0ee9614d828d4ea8ab18a900dbc063c1 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.716971 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.717602 27934 raft_consensus.cc:2243] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.718026 27934 raft_consensus.cc:2272] T a9e48769916844ce8d464a5bd1bda6de P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.720320 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.720891 27934 raft_consensus.cc:2243] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.721380 27934 raft_consensus.cc:2272] T 77f29b60a32f4f79a3f1bd59278b00bd P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.723706 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.724318 27934 raft_consensus.cc:2243] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.724673 27934 raft_consensus.cc:2272] T 3e04051483df4ae3bf4c3f5ab037fda0 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.726662 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.727149 27934 raft_consensus.cc:2243] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.727587 27934 raft_consensus.cc:2272] T 11ac5c3627ca49348e39094121c135c4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.729872 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.730620 27934 raft_consensus.cc:2243] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.731089 27934 raft_consensus.cc:2272] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.733952 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.734808 27934 raft_consensus.cc:2243] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.735239 27934 raft_consensus.cc:2272] T c1280a1f51344b42b4410fc2852da38d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.738322 27934 tablet_replica.cc:333] stopping tablet replica
W20260709 13:08:44.738430  5888 consensus_peers.cc:597] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.739058 27934 raft_consensus.cc:2243] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.740059 27934 raft_consensus.cc:2272] T 2bba78f968ad41538018aacddcc5d9d7 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.742717 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.743348 27934 raft_consensus.cc:2243] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.744383 27934 raft_consensus.cc:2272] T 30f9347b11ec44d1b07c475379a51881 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.747319 27934 tablet_replica.cc:333] stopping tablet replica
W20260709 13:08:44.747546  5813 consensus_peers.cc:597] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.748054 27934 raft_consensus.cc:2243] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.748572 27934 raft_consensus.cc:2272] T dcc03c962c314adf95650bc3bf8e9d34 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.750949 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.751624 27934 raft_consensus.cc:2243] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.752120 27934 raft_consensus.cc:2272] T 3e83357d1cc74a0292e36bd8947af234 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.754596 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.755386 27934 raft_consensus.cc:2243] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.755956 27934 raft_consensus.cc:2272] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.759137 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.759967 27934 raft_consensus.cc:2243] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.760454 27934 raft_consensus.cc:2272] T a3f19012a0a24bd88cecafe95b9b461c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:44.762002  5888 consensus_peers.cc:597] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.763243 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.764276 27934 raft_consensus.cc:2243] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.764838 27934 raft_consensus.cc:2272] T a9dfeaefea774ae19834e0a619330eb5 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:44.766899  5813 consensus_peers.cc:597] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.767824 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.768733 27934 raft_consensus.cc:2243] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:08:44.769265  5739 consensus_peers.cc:597] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.769690 27934 raft_consensus.cc:2272] T 3ada8ac4d923408997f9511c6910789c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.772035 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.772662 27934 raft_consensus.cc:2243] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.773479 27934 raft_consensus.cc:2272] T 365d76e5c79a4b0b8b65ad7465deac93 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.776492 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.777277 27934 raft_consensus.cc:2243] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.777763 27934 raft_consensus.cc:2272] T 2e47f79b0ad84696b05ddc28ce9bef20 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:44.780017  5888 consensus_peers.cc:597] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.780691 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.781566 27934 raft_consensus.cc:2243] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.782536 27934 raft_consensus.cc:2272] T 43ed8e83113241bda21a6e92ed10ac9e P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:44.783589  5813 consensus_peers.cc:597] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.785332 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.786018 27934 raft_consensus.cc:2243] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.786825 27934 raft_consensus.cc:2272] T 0e03b5d529f34213b58b61ece24ce512 P d9952df48cf245608db9181dedad0aeb [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.789472 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.790333 27934 raft_consensus.cc:2243] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.790884 27934 raft_consensus.cc:2272] T aa162ebfa5094c25a3a311044956d167 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.793903 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.794553 27934 raft_consensus.cc:2243] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:44.794811  5739 consensus_peers.cc:597] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.794994 27934 raft_consensus.cc:2272] T deecf916d19d4f5ca9ac8200dc931c5c P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.797459 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.798269 27934 raft_consensus.cc:2243] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.798826 27934 raft_consensus.cc:2272] T 8842506bb6124bf9adf9b13c6300c796 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.802063 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.802995 27934 raft_consensus.cc:2243] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:44.803298  5888 consensus_peers.cc:597] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.803560 27934 raft_consensus.cc:2272] T a1b715b22a0343fbb87245b6bf0fb522 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:44.805549  5813 consensus_peers.cc:597] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.806774 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.807500 27934 raft_consensus.cc:2243] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.808113 27934 raft_consensus.cc:2272] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.810601 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.811228 27934 raft_consensus.cc:2243] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.812090 27934 raft_consensus.cc:2272] T 400faa6e9aed42f19cda604767b955c2 P d9952df48cf245608db9181dedad0aeb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.817381 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.818256 27934 raft_consensus.cc:2243] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.818861 27934 raft_consensus.cc:2272] T 7f95c9f03a2848e29a1ed53170f6bc85 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.821892 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.822481 27934 raft_consensus.cc:2243] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:44.823325 27934 raft_consensus.cc:2272] T 12df4d7a5aad4ece999b2b3722b294e5 P d9952df48cf245608db9181dedad0aeb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.825455 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.825999 27934 raft_consensus.cc:2243] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.826473 27934 raft_consensus.cc:2272] T 4890d4f381d74b37a99616c1b1dc65d4 P d9952df48cf245608db9181dedad0aeb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.829459 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.829991 27934 raft_consensus.cc:2243] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.830510 27934 raft_consensus.cc:2272] T 892918414c974de38c7050ccd7151f94 P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:44.832083  5813 consensus_peers.cc:597] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.833304 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.833962 27934 raft_consensus.cc:2243] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:44.834522 27934 raft_consensus.cc:2272] T b37c67a7062e4372b72045052c828e2d P d9952df48cf245608db9181dedad0aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:44.837538 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:44.838209 27934 raft_consensus.cc:2243] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 8 LEADER]: Raft consensus shutting down.
W20260709 13:08:44.839008  5813 consensus_peers.cc:597] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.839303 27934 raft_consensus.cc:2272] T fbbd1771e7664e0cb3441c84672d037d P d9952df48cf245608db9181dedad0aeb [term 8 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:44.840857  5888 consensus_peers.cc:597] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.857239  5888 consensus_peers.cc:597] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.898073  5739 consensus_peers.cc:597] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.899192  5813 consensus_peers.cc:597] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.907089  5813 connection.cc:573] client connection to 127.27.71.129:40419 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:08:44.907696  5813 consensus_peers.cc:597] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:44.927834 27934 tablet_server.cc:196] TabletServer@127.27.71.129:40419 shutdown complete.
W20260709 13:08:44.929088  5739 consensus_peers.cc:597] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.931432  5813 consensus_peers.cc:597] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:44.940781  5739 consensus_peers.cc:597] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.038915 27934 tablet_server.cc:179] TabletServer@127.27.71.130:0 shutting down...
W20260709 13:08:45.048055  5890 consensus_peers.cc:597] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.048771  5890 consensus_peers.cc:597] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.054258  5890 consensus_peers.cc:597] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.067588  5815 consensus_peers.cc:597] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.089871  5888 consensus_peers.cc:597] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 -> Peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Couldn't send request to peer d9952df48cf245608db9181dedad0aeb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.095754  5815 consensus_peers.cc:597] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.108809  5815 consensus_peers.cc:597] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.112327 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:45.113363 27934 tablet_replica.cc:333] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.114316 27934 raft_consensus.cc:2243] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:08:45.114969  5815 consensus_peers.cc:597] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.115511 27934 raft_consensus.cc:2272] T 3e83357d1cc74a0292e36bd8947af234 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.118098 27934 tablet_replica.cc:333] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.118770 27934 raft_consensus.cc:2243] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.119197 27934 raft_consensus.cc:2272] T 27bf52f47b87456c94c45846de53788d P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.121923 27934 tablet_replica.cc:333] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.122745 27934 raft_consensus.cc:2243] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.123306 27934 raft_consensus.cc:2272] T b1e618980cb143c69eb4b401ba701662 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.126083 27934 tablet_replica.cc:333] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.126835 27934 raft_consensus.cc:2243] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.127480 27934 raft_consensus.cc:2272] T 12df4d7a5aad4ece999b2b3722b294e5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.130215 27934 tablet_replica.cc:333] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.130761 27934 raft_consensus.cc:2243] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.131695 27934 raft_consensus.cc:2272] T 098c5f84c14148eabebfe76f9ccfc1df P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.134240 27934 tablet_replica.cc:333] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.134829 27934 raft_consensus.cc:2243] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.135286 27934 raft_consensus.cc:2272] T 5051e38577e54ce780fad112f2fcd202 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.138123 27934 tablet_replica.cc:333] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.138760 27934 raft_consensus.cc:2243] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:45.139031  5815 consensus_peers.cc:597] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.139356 27934 raft_consensus.cc:2272] T 30f9347b11ec44d1b07c475379a51881 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.141814 27934 tablet_replica.cc:333] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.142655 27934 raft_consensus.cc:2243] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.143741 27934 raft_consensus.cc:2272] T dcc03c962c314adf95650bc3bf8e9d34 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:45.146317  5815 consensus_peers.cc:597] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.146579 27934 tablet_replica.cc:333] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.147261 27934 raft_consensus.cc:2243] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.147670 27934 raft_consensus.cc:2272] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.150229 27934 tablet_replica.cc:333] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.151151 27934 raft_consensus.cc:2243] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.152489 27934 raft_consensus.cc:2272] T deecf916d19d4f5ca9ac8200dc931c5c P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.155627 27934 tablet_replica.cc:333] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.156431 27934 raft_consensus.cc:2243] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.157019 27934 raft_consensus.cc:2272] T 8842506bb6124bf9adf9b13c6300c796 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.160372 27934 tablet_replica.cc:333] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.161192 27934 raft_consensus.cc:2243] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.161749 27934 raft_consensus.cc:2272] T d8f9da7df7034a9f99e973517496cd22 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.164765 27934 tablet_replica.cc:333] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.165485 27934 raft_consensus.cc:2243] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Raft consensus shutting down.
W20260709 13:08:45.166090  5890 consensus_peers.cc:597] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.166541 27934 raft_consensus.cc:2272] T dd06784a6f3244309ebdd41cf8127741 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.168887 27934 tablet_replica.cc:333] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.169548 27934 raft_consensus.cc:2243] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.170389 27934 raft_consensus.cc:2272] T aa162ebfa5094c25a3a311044956d167 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.172549 27934 tablet_replica.cc:333] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.173161 27934 raft_consensus.cc:2243] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.173629 27934 raft_consensus.cc:2272] T 0e03b5d529f34213b58b61ece24ce512 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.176486 27934 tablet_replica.cc:333] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.177109 27934 raft_consensus.cc:2243] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.177726 27934 raft_consensus.cc:2272] T 43ed8e83113241bda21a6e92ed10ac9e P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.180354 27934 tablet_replica.cc:333] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.181143 27934 raft_consensus.cc:2243] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.182217 27934 raft_consensus.cc:2272] T 422b96951eda4c1c8b8d86612ac6c163 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.184942 27934 tablet_replica.cc:333] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.185689 27934 raft_consensus.cc:2243] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.186710 27934 raft_consensus.cc:2272] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.189586 27934 tablet_replica.cc:333] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.190382 27934 raft_consensus.cc:2243] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.190901 27934 raft_consensus.cc:2272] T 400faa6e9aed42f19cda604767b955c2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.193792 27934 tablet_replica.cc:333] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.194543 27934 raft_consensus.cc:2243] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.195120 27934 raft_consensus.cc:2272] T e982efc662b84721b51ef9101f0cf5a8 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.197393 27934 tablet_replica.cc:333] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.198012 27934 raft_consensus.cc:2243] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.198864 27934 raft_consensus.cc:2272] T 0ee9614d828d4ea8ab18a900dbc063c1 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.200917 27934 tablet_replica.cc:333] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.201532 27934 raft_consensus.cc:2243] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.202004 27934 raft_consensus.cc:2272] T 4890d4f381d74b37a99616c1b1dc65d4 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.204298 27934 tablet_replica.cc:333] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.204957 27934 raft_consensus.cc:2243] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.205873 27934 raft_consensus.cc:2272] T 892918414c974de38c7050ccd7151f94 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.208288 27934 tablet_replica.cc:333] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.208860 27934 raft_consensus.cc:2243] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.209321 27934 raft_consensus.cc:2272] T fbbd1771e7664e0cb3441c84672d037d P 373ee2f4cc9d4b86b49ada724f55e16a [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.211413 27934 tablet_replica.cc:333] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.211889 27934 raft_consensus.cc:2243] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.212422 27934 raft_consensus.cc:2272] T 8ba1e6d2a3344631b647a567e5280513 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.214691 27934 tablet_replica.cc:333] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.215445 27934 raft_consensus.cc:2243] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.216045 27934 raft_consensus.cc:2272] T 674fd655eaf14b10bc2e013c27c40557 P 373ee2f4cc9d4b86b49ada724f55e16a [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.219071 27934 tablet_replica.cc:333] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.219776 27934 raft_consensus.cc:2243] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.220345 27934 raft_consensus.cc:2272] T 2e47f79b0ad84696b05ddc28ce9bef20 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.222625 27934 tablet_replica.cc:333] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.223203 27934 raft_consensus.cc:2243] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.223618 27934 raft_consensus.cc:2272] T 365d76e5c79a4b0b8b65ad7465deac93 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.226068 27934 tablet_replica.cc:333] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.226809 27934 raft_consensus.cc:2243] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.227906 27934 raft_consensus.cc:2272] T a1b715b22a0343fbb87245b6bf0fb522 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.230736 27934 tablet_replica.cc:333] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.231482 27934 raft_consensus.cc:2243] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.232601 27934 raft_consensus.cc:2272] T 057217b46a0d4d188d28fecbdf744762 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:45.234742  5890 consensus_peers.cc:597] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.235605 27934 tablet_replica.cc:333] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.236505 27934 raft_consensus.cc:2243] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.237021 27934 raft_consensus.cc:2272] T 32b6cadc1b9d441983391c6e69f42beb P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:45.237512  5815 consensus_peers.cc:597] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.239805 27934 tablet_replica.cc:333] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.240725 27934 raft_consensus.cc:2243] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.241310 27934 raft_consensus.cc:2272] T 75595b8e97ef436992a8acba2c9724a7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.244029 27934 tablet_replica.cc:333] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.244752 27934 raft_consensus.cc:2243] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.245313 27934 raft_consensus.cc:2272] T 2bba78f968ad41538018aacddcc5d9d7 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.248641 27934 tablet_replica.cc:333] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.249439 27934 raft_consensus.cc:2243] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.250026 27934 raft_consensus.cc:2272] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 373ee2f4cc9d4b86b49ada724f55e16a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.252774 27934 tablet_replica.cc:333] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.253518 27934 raft_consensus.cc:2243] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.254410 27934 raft_consensus.cc:2272] T a9e48769916844ce8d464a5bd1bda6de P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.256805 27934 tablet_replica.cc:333] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.257416 27934 raft_consensus.cc:2243] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.258421 27934 raft_consensus.cc:2272] T ffd891d1488c409d87be1300b3e91325 P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.260725 27934 tablet_replica.cc:333] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.261399 27934 raft_consensus.cc:2243] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.261916 27934 raft_consensus.cc:2272] T a62aede00e774cabb36368e7202cdff9 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.265141 27934 tablet_replica.cc:333] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
W20260709 13:08:45.265026  5890 consensus_peers.cc:597] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.265978 27934 raft_consensus.cc:2243] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.266495 27934 raft_consensus.cc:2272] T adfec13bcbce4db48b5f0dba7a8e5345 P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.268868 27934 tablet_replica.cc:333] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.269510 27934 raft_consensus.cc:2243] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.270328 27934 raft_consensus.cc:2272] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 373ee2f4cc9d4b86b49ada724f55e16a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.273234 27934 tablet_replica.cc:333] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.273948 27934 raft_consensus.cc:2243] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.274470 27934 raft_consensus.cc:2272] T e2d7f76b05b8419783f1a8f16afaec07 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.277331 27934 tablet_replica.cc:333] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.278088 27934 raft_consensus.cc:2243] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.279429 27934 raft_consensus.cc:2272] T c7f81180a2c94fe5874f1a402e44c52f P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.282677 27934 tablet_replica.cc:333] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.283466 27934 raft_consensus.cc:2243] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.284032 27934 raft_consensus.cc:2272] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 373ee2f4cc9d4b86b49ada724f55e16a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.287273 27934 tablet_replica.cc:333] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.287997 27934 raft_consensus.cc:2243] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.288547 27934 raft_consensus.cc:2272] T 7f95c9f03a2848e29a1ed53170f6bc85 P 373ee2f4cc9d4b86b49ada724f55e16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.291363 27934 tablet_replica.cc:333] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a: stopping tablet replica
I20260709 13:08:45.292049 27934 raft_consensus.cc:2243] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.292604 27934 raft_consensus.cc:2272] T 77f29b60a32f4f79a3f1bd59278b00bd P 373ee2f4cc9d4b86b49ada724f55e16a [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:45.341668  5890 consensus_peers.cc:597] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.392118  5890 consensus_peers.cc:597] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:39429: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.414445  5815 consensus_peers.cc:597] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:39429: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.445562 27934 tablet_server.cc:196] TabletServer@127.27.71.130:0 shutdown complete.
W20260709 13:08:45.489701  5890 consensus_peers.cc:597] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:39429: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.534168 27934 tablet_server.cc:179] TabletServer@127.27.71.131:0 shutting down...
W20260709 13:08:45.574812  5891 consensus_peers.cc:597] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.588140  5891 consensus_peers.cc:597] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.609234  5890 consensus_peers.cc:597] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 -> Peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Couldn't send request to peer 373ee2f4cc9d4b86b49ada724f55e16a. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:39429: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.616477 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:45.617431 27934 tablet_replica.cc:333] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.618165 27934 raft_consensus.cc:2243] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.618729 27934 raft_consensus.cc:2272] T 30f9347b11ec44d1b07c475379a51881 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.621488 27934 tablet_replica.cc:333] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.622148 27934 raft_consensus.cc:2243] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.622598 27934 raft_consensus.cc:2272] T 098c5f84c14148eabebfe76f9ccfc1df P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.624706 27934 tablet_replica.cc:333] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.625331 27934 raft_consensus.cc:2243] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.626247 27934 raft_consensus.cc:2272] T c1280a1f51344b42b4410fc2852da38d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.628805 27934 tablet_replica.cc:333] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.629393 27934 raft_consensus.cc:2243] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.629835 27934 raft_consensus.cc:2272] T 2bba78f968ad41538018aacddcc5d9d7 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.630393  7366 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:45.630892  7366 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:45.632360 27934 tablet_replica.cc:333] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.633106  7366 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 792907f253c744aa9392e8d572f88e06 (127.27.71.132:45241), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:45.633263 27934 raft_consensus.cc:2243] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.634142 27934 raft_consensus.cc:2272] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.634037  5921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40ba6b5dcd674422a95eeaad0efd1e45" candidate_uuid: "52b726da57c24dd2aa72488ebe2ff21e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "792907f253c744aa9392e8d572f88e06" is_pre_election: true
W20260709 13:08:45.637102  5813 leader_election.cc:336] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:45.637231 27934 tablet_replica.cc:333] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.637475  5813 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52b726da57c24dd2aa72488ebe2ff21e; no voters: 792907f253c744aa9392e8d572f88e06, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:45.638031 27934 raft_consensus.cc:2243] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.638109  7366 raft_consensus.cc:2749] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:45.638571 27934 raft_consensus.cc:2272] T 5051e38577e54ce780fad112f2fcd202 P 52b726da57c24dd2aa72488ebe2ff21e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.640664 27934 tablet_replica.cc:333] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.641248 27934 raft_consensus.cc:2243] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.642031 27934 raft_consensus.cc:2272] T deecf916d19d4f5ca9ac8200dc931c5c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.644894 27934 tablet_replica.cc:333] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.645754 27934 raft_consensus.cc:2243] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:08:45.645920  5891 consensus_peers.cc:597] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.646341 27934 raft_consensus.cc:2272] T aa162ebfa5094c25a3a311044956d167 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.649170 27934 tablet_replica.cc:333] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.649821 27934 raft_consensus.cc:2243] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.650249 27934 raft_consensus.cc:2272] T 43ed8e83113241bda21a6e92ed10ac9e P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:45.651326  5891 consensus_peers.cc:597] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.652571 27934 tablet_replica.cc:333] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.653324 27934 raft_consensus.cc:2243] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.654080 27934 raft_consensus.cc:2272] T 8cb2353dae2d41588ef00b83e6cc10d1 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.656289 27934 tablet_replica.cc:333] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.656885 27934 raft_consensus.cc:2243] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.657662 27934 raft_consensus.cc:2272] T 8842506bb6124bf9adf9b13c6300c796 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.660329 27934 tablet_replica.cc:333] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.661185 27934 raft_consensus.cc:2243] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.661746 27934 raft_consensus.cc:2272] T d8f9da7df7034a9f99e973517496cd22 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.664714 27934 tablet_replica.cc:333] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.665417 27934 raft_consensus.cc:2243] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.665880 27934 raft_consensus.cc:2272] T dd06784a6f3244309ebdd41cf8127741 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.668087 27934 tablet_replica.cc:333] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.668874 27934 raft_consensus.cc:2243] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.669266 27934 raft_consensus.cc:2272] T e982efc662b84721b51ef9101f0cf5a8 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.671305 27934 tablet_replica.cc:333] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.671824 27934 raft_consensus.cc:2243] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.672250 27934 raft_consensus.cc:2272] T b37c67a7062e4372b72045052c828e2d P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.674213 27934 tablet_replica.cc:333] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.674734 27934 raft_consensus.cc:2243] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.675109 27934 raft_consensus.cc:2272] T fbbd1771e7664e0cb3441c84672d037d P 52b726da57c24dd2aa72488ebe2ff21e [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.677052 27934 tablet_replica.cc:333] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.677567 27934 raft_consensus.cc:2243] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.677951 27934 raft_consensus.cc:2272] T 674fd655eaf14b10bc2e013c27c40557 P 52b726da57c24dd2aa72488ebe2ff21e [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.679785 27934 tablet_replica.cc:333] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.680346 27934 raft_consensus.cc:2243] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.680747 27934 raft_consensus.cc:2272] T 365d76e5c79a4b0b8b65ad7465deac93 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.682547 27934 tablet_replica.cc:333] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.683020 27934 raft_consensus.cc:2243] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.683774 27934 raft_consensus.cc:2272] T 8ba1e6d2a3344631b647a567e5280513 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.686610 27934 tablet_replica.cc:333] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.687429 27934 raft_consensus.cc:2243] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.688465 27934 raft_consensus.cc:2272] T de4f21d87306453b81a417014e41e4a4 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:45.688709  5891 consensus_peers.cc:597] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.690907 27934 tablet_replica.cc:333] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.691635 27934 raft_consensus.cc:2243] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.692201 27934 raft_consensus.cc:2272] T 106cdfafda414cf6981dbcaf1e28cc9d P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.694583 27934 tablet_replica.cc:333] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.695369 27934 raft_consensus.cc:2243] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.695897 27934 raft_consensus.cc:2272] T 057217b46a0d4d188d28fecbdf744762 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.698577 27934 tablet_replica.cc:333] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.699064 27934 raft_consensus.cc:2243] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.699397 27934 raft_consensus.cc:2272] T d76ed139f0d94b8484f5b5101634fb1b P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.701277 27934 tablet_replica.cc:333] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.701777 27934 raft_consensus.cc:2243] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.702214 27934 raft_consensus.cc:2272] T 32b6cadc1b9d441983391c6e69f42beb P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.704540 27934 tablet_replica.cc:333] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.705302 27934 raft_consensus.cc:2243] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.706261 27934 raft_consensus.cc:2272] T b1e618980cb143c69eb4b401ba701662 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.708364 27934 tablet_replica.cc:333] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.708940 27934 raft_consensus.cc:2243] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.709939 27934 raft_consensus.cc:2272] T 759acc6b90aa41a1b6c9053bd20846c1 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.712334 27934 tablet_replica.cc:333] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.713083 27934 raft_consensus.cc:2243] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.713528 27934 raft_consensus.cc:2272] T 3ada8ac4d923408997f9511c6910789c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.716096 27934 tablet_replica.cc:333] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.716687 27934 raft_consensus.cc:2243] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.717648 27934 raft_consensus.cc:2272] T a9dfeaefea774ae19834e0a619330eb5 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:45.718053  5891 consensus_peers.cc:597] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.719715 27934 tablet_replica.cc:333] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.720397 27934 raft_consensus.cc:2243] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.720834 27934 raft_consensus.cc:2272] T a3f19012a0a24bd88cecafe95b9b461c P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.722888 27934 tablet_replica.cc:333] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.723454 27934 raft_consensus.cc:2243] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.724026 27934 raft_consensus.cc:2272] T 9f08c26e76934d3c94ef7060e15caed0 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.726212 27934 tablet_replica.cc:333] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.726780 27934 raft_consensus.cc:2243] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.727586 27934 raft_consensus.cc:2272] T 11ac5c3627ca49348e39094121c135c4 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.729625 27934 tablet_replica.cc:333] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.730192 27934 raft_consensus.cc:2243] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.730965 27934 raft_consensus.cc:2272] T 3e04051483df4ae3bf4c3f5ab037fda0 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.733080 27934 tablet_replica.cc:333] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.733652 27934 raft_consensus.cc:2243] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.734100 27934 raft_consensus.cc:2272] T e2d7f76b05b8419783f1a8f16afaec07 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.736388 27934 tablet_replica.cc:333] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.737356 27934 raft_consensus.cc:2243] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.737883 27934 raft_consensus.cc:2272] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.739980 27934 tablet_replica.cc:333] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.740501 27934 raft_consensus.cc:2243] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.741251 27934 raft_consensus.cc:2272] T 75595b8e97ef436992a8acba2c9724a7 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.743085 27934 tablet_replica.cc:333] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.743641 27934 raft_consensus.cc:2243] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.744143 27934 raft_consensus.cc:2272] T c7f81180a2c94fe5874f1a402e44c52f P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.746479 27934 tablet_replica.cc:333] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.747236 27934 raft_consensus.cc:2243] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.747793 27934 raft_consensus.cc:2272] T ffd891d1488c409d87be1300b3e91325 P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.749840 27934 tablet_replica.cc:333] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.750399 27934 raft_consensus.cc:2243] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.750887 27934 raft_consensus.cc:2272] T a62aede00e774cabb36368e7202cdff9 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.752889 27934 tablet_replica.cc:333] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.753414 27934 raft_consensus.cc:2243] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.753875 27934 raft_consensus.cc:2272] T adfec13bcbce4db48b5f0dba7a8e5345 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.755877 27934 tablet_replica.cc:333] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.756469 27934 raft_consensus.cc:2243] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.756882 27934 raft_consensus.cc:2272] T 40ba6b5dcd674422a95eeaad0efd1e45 P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.758797 27934 tablet_replica.cc:333] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.759276 27934 raft_consensus.cc:2243] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:45.759806 27934 raft_consensus.cc:2272] T 2ddbd585b0264150a1fbd1b5cb8b5ebb P 52b726da57c24dd2aa72488ebe2ff21e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.762563 27934 tablet_replica.cc:333] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.763139 27934 raft_consensus.cc:2243] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.764092 27934 raft_consensus.cc:2272] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 52b726da57c24dd2aa72488ebe2ff21e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.766950 27934 tablet_replica.cc:333] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.767624 27934 raft_consensus.cc:2243] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.768591 27934 raft_consensus.cc:2272] T 7f95c9f03a2848e29a1ed53170f6bc85 P 52b726da57c24dd2aa72488ebe2ff21e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:45.771490 27934 tablet_replica.cc:333] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e: stopping tablet replica
I20260709 13:08:45.772100 27934 raft_consensus.cc:2243] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:45.772955 27934 raft_consensus.cc:2272] T 77f29b60a32f4f79a3f1bd59278b00bd P 52b726da57c24dd2aa72488ebe2ff21e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:45.815689  5891 consensus_peers.cc:597] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.867343  5891 consensus_peers.cc:597] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.131:37585: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:45.874037  5891 consensus_peers.cc:597] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.131:37585: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.890097  7351 raft_consensus.cc:493] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:45.890681  7351 raft_consensus.cc:515] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:45.893190  7351 leader_election.cc:290] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:45.895071 27934 tablet_server.cc:196] TabletServer@127.27.71.131:0 shutdown complete.
W20260709 13:08:45.900404  5891 leader_election.cc:336] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Network error: Client connection negotiation failed: client connection to 127.27.71.131:37585: connect: Connection refused (error 111)
W20260709 13:08:45.906626  5888 leader_election.cc:336] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:45.907025  5888 leader_election.cc:304] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:45.907666  7351 raft_consensus.cc:2749] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:08:45.913182  5891 consensus_peers.cc:597] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.131:37585: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.944756  7338 raft_consensus.cc:493] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:45.945415  7338 raft_consensus.cc:515] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:45.947819  7338 leader_election.cc:290] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:45.953533  5890 leader_election.cc:336] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Network error: Client connection negotiation failed: client connection to 127.27.71.130:39429: connect: Connection refused (error 111)
W20260709 13:08:45.955696  5888 leader_election.cc:336] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:45.956220  5888 leader_election.cc:304] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:45.957001  7338 raft_consensus.cc:2749] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:08:45.968271  5891 consensus_peers.cc:597] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 -> Peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Couldn't send request to peer 52b726da57c24dd2aa72488ebe2ff21e. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.131:37585: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:45.981477  7351 raft_consensus.cc:493] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:45.982131  7351 raft_consensus.cc:515] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:45.983731 27934 tablet_server.cc:179] TabletServer@127.27.71.132:0 shutting down...
I20260709 13:08:45.986500  7351 leader_election.cc:290] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
W20260709 13:08:45.991027  5888 leader_election.cc:336] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
W20260709 13:08:45.992091  5890 leader_election.cc:336] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Network error: Client connection negotiation failed: client connection to 127.27.71.130:39429: connect: Connection refused (error 111)
I20260709 13:08:45.992429  5890 leader_election.cc:304] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:45.993075  7351 raft_consensus.cc:2749] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:46.034260 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:46.036485 27934 tablet_replica.cc:333] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.037572 27934 raft_consensus.cc:2243] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.038187 27934 raft_consensus.cc:2272] T 3e83357d1cc74a0292e36bd8947af234 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.041281 27934 tablet_replica.cc:333] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.042132 27934 raft_consensus.cc:2243] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.043135 27934 raft_consensus.cc:2272] T 27bf52f47b87456c94c45846de53788d P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.045869 27934 tablet_replica.cc:333] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.046541 27934 raft_consensus.cc:2243] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.046999 27934 raft_consensus.cc:2272] T 098c5f84c14148eabebfe76f9ccfc1df P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.049055 27934 tablet_replica.cc:333] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.049707 27934 raft_consensus.cc:2243] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.050184 27934 raft_consensus.cc:2272] T 12df4d7a5aad4ece999b2b3722b294e5 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.052529 27934 tablet_replica.cc:333] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.052488  7362 raft_consensus.cc:493] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:46.053104  7362 raft_consensus.cc:515] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } } peers { permanent_uuid: "373ee2f4cc9d4b86b49ada724f55e16a" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 39429 } }
I20260709 13:08:46.053994 27934 raft_consensus.cc:2243] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.054828 27934 raft_consensus.cc:2272] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.055773  7362 leader_election.cc:290] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419), 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429)
I20260709 13:08:46.059820 27934 tablet_replica.cc:333] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
W20260709 13:08:46.060351  5888 leader_election.cc:336] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:46.061287 27934 raft_consensus.cc:2243] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:08:46.061870  5890 leader_election.cc:336] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 373ee2f4cc9d4b86b49ada724f55e16a (127.27.71.130:39429): Network error: Client connection negotiation failed: client connection to 127.27.71.130:39429: connect: Connection refused (error 111)
I20260709 13:08:46.062338  5890 leader_election.cc:304] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 373ee2f4cc9d4b86b49ada724f55e16a, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:46.062474 27934 raft_consensus.cc:2272] T 440b21f4e5f94cf2a96ffb8b2a631d60 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:46.063005  5890 raft_consensus.cc:2686] T 0e03b5d529f34213b58b61ece24ce512 P 792907f253c744aa9392e8d572f88e06: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:08:46.065474 27934 tablet_replica.cc:333] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.066473 27934 raft_consensus.cc:2243] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.067070 27934 raft_consensus.cc:2272] T dcc03c962c314adf95650bc3bf8e9d34 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.069655 27934 tablet_replica.cc:333] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.070297 27934 raft_consensus.cc:2243] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.070747 27934 raft_consensus.cc:2272] T 057217b46a0d4d188d28fecbdf744762 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.073051 27934 tablet_replica.cc:333] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.073776 27934 raft_consensus.cc:2243] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.074273 27934 raft_consensus.cc:2272] T d76ed139f0d94b8484f5b5101634fb1b P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.076643 27934 tablet_replica.cc:333] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.077210 27934 raft_consensus.cc:2243] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.078114 27934 raft_consensus.cc:2272] T d8f9da7df7034a9f99e973517496cd22 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.080766 27934 tablet_replica.cc:333] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.081365 27934 raft_consensus.cc:2243] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.081844 27934 raft_consensus.cc:2272] T dd06784a6f3244309ebdd41cf8127741 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.084069 27934 tablet_replica.cc:333] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.084640 27934 raft_consensus.cc:2243] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.085044 27934 raft_consensus.cc:2272] T b1e618980cb143c69eb4b401ba701662 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.087144 27934 tablet_replica.cc:333] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.087668 27934 raft_consensus.cc:2243] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.088093 27934 raft_consensus.cc:2272] T 759acc6b90aa41a1b6c9053bd20846c1 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.090036 27934 tablet_replica.cc:333] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.090641 27934 raft_consensus.cc:2243] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.091037 27934 raft_consensus.cc:2272] T 8aa6b3cf215a40a5b6bc25bd3dddc408 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.093211 27934 tablet_replica.cc:333] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.093729 27934 raft_consensus.cc:2243] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.094107 27934 raft_consensus.cc:2272] T 400faa6e9aed42f19cda604767b955c2 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.096449 27934 tablet_replica.cc:333] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.097060 27934 raft_consensus.cc:2243] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.097595 27934 raft_consensus.cc:2272] T 0ee9614d828d4ea8ab18a900dbc063c1 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.100339 27934 tablet_replica.cc:333] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.101145 27934 raft_consensus.cc:2243] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.101621 27934 raft_consensus.cc:2272] T 8cb2353dae2d41588ef00b83e6cc10d1 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.104223 27934 tablet_replica.cc:333] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.104743 27934 raft_consensus.cc:2243] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.105118 27934 raft_consensus.cc:2272] T de4f21d87306453b81a417014e41e4a4 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.105633  7362 raft_consensus.cc:493] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9952df48cf245608db9181dedad0aeb)
I20260709 13:08:46.106078  7362 raft_consensus.cc:515] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b726da57c24dd2aa72488ebe2ff21e" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37585 } } peers { permanent_uuid: "792907f253c744aa9392e8d572f88e06" member_type: VOTER last_known_addr { host: "127.27.71.132" port: 45241 } } peers { permanent_uuid: "d9952df48cf245608db9181dedad0aeb" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 40419 } }
I20260709 13:08:46.107357 27934 tablet_replica.cc:333] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.107903  7362 leader_election.cc:290] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585), d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419)
I20260709 13:08:46.108276 27934 raft_consensus.cc:2243] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.109475 27934 raft_consensus.cc:2272] T 106cdfafda414cf6981dbcaf1e28cc9d P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:46.111869  5888 leader_election.cc:336] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9952df48cf245608db9181dedad0aeb (127.27.71.129:40419): Network error: Client connection negotiation failed: client connection to 127.27.71.129:40419: connect: Connection refused (error 111)
I20260709 13:08:46.112049 27934 tablet_replica.cc:333] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
W20260709 13:08:46.112426  5891 leader_election.cc:336] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 52b726da57c24dd2aa72488ebe2ff21e (127.27.71.131:37585): Network error: Client connection negotiation failed: client connection to 127.27.71.131:37585: connect: Connection refused (error 111)
I20260709 13:08:46.112896  5891 leader_election.cc:304] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792907f253c744aa9392e8d572f88e06; no voters: 52b726da57c24dd2aa72488ebe2ff21e, d9952df48cf245608db9181dedad0aeb
I20260709 13:08:46.112986 27934 raft_consensus.cc:2243] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.113687 27934 raft_consensus.cc:2272] T 8ba1e6d2a3344631b647a567e5280513 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.113698  7338 raft_consensus.cc:2749] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:46.116200 27934 tablet_replica.cc:333] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.117120 27934 raft_consensus.cc:2243] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.117702 27934 raft_consensus.cc:2272] T 3ada8ac4d923408997f9511c6910789c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.119686 27934 tablet_replica.cc:333] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.120260 27934 raft_consensus.cc:2243] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.120726 27934 raft_consensus.cc:2272] T a9dfeaefea774ae19834e0a619330eb5 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.122613 27934 tablet_replica.cc:333] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.123138 27934 raft_consensus.cc:2243] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.123900 27934 raft_consensus.cc:2272] T a3f19012a0a24bd88cecafe95b9b461c P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.125819 27934 tablet_replica.cc:333] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.126349 27934 raft_consensus.cc:2243] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 6 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.127113 27934 raft_consensus.cc:2272] T 674fd655eaf14b10bc2e013c27c40557 P 792907f253c744aa9392e8d572f88e06 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.128959 27934 tablet_replica.cc:333] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.129544 27934 raft_consensus.cc:2243] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.129972 27934 raft_consensus.cc:2272] T a1b715b22a0343fbb87245b6bf0fb522 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.131829 27934 tablet_replica.cc:333] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.132421 27934 raft_consensus.cc:2243] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.133180 27934 raft_consensus.cc:2272] T 9f08c26e76934d3c94ef7060e15caed0 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.135135 27934 tablet_replica.cc:333] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.135697 27934 raft_consensus.cc:2243] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.136130 27934 raft_consensus.cc:2272] T c1280a1f51344b42b4410fc2852da38d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.137974 27934 tablet_replica.cc:333] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.138522 27934 raft_consensus.cc:2243] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.139292 27934 raft_consensus.cc:2272] T 3c979e51cdc44dac9ae6b2117e8ac0c5 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.141340 27934 tablet_replica.cc:333] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.141878 27934 raft_consensus.cc:2243] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.142704 27934 raft_consensus.cc:2272] T 2e47f79b0ad84696b05ddc28ce9bef20 P 792907f253c744aa9392e8d572f88e06 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.144743 27934 tablet_replica.cc:333] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.145287 27934 raft_consensus.cc:2243] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.145782 27934 raft_consensus.cc:2272] T 11ac5c3627ca49348e39094121c135c4 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.147840 27934 tablet_replica.cc:333] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.148445 27934 raft_consensus.cc:2243] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.148880 27934 raft_consensus.cc:2272] T 3e04051483df4ae3bf4c3f5ab037fda0 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.150761 27934 tablet_replica.cc:333] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.151289 27934 raft_consensus.cc:2243] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.152241 27934 raft_consensus.cc:2272] T e2d7f76b05b8419783f1a8f16afaec07 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.154897 27934 tablet_replica.cc:333] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.155617 27934 raft_consensus.cc:2243] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.156492 27934 raft_consensus.cc:2272] T 96bcc183bf584d4e8fbd4d0a87c2e2ea P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.158492 27934 tablet_replica.cc:333] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.159047 27934 raft_consensus.cc:2243] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.159497 27934 raft_consensus.cc:2272] T 422b96951eda4c1c8b8d86612ac6c163 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.161348 27934 tablet_replica.cc:333] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.161976 27934 raft_consensus.cc:2243] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.162431 27934 raft_consensus.cc:2272] T ffd891d1488c409d87be1300b3e91325 P 792907f253c744aa9392e8d572f88e06 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.164412 27934 tablet_replica.cc:333] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.164914 27934 raft_consensus.cc:2243] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.165678 27934 raft_consensus.cc:2272] T a62aede00e774cabb36368e7202cdff9 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.167419 27934 tablet_replica.cc:333] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.167989 27934 raft_consensus.cc:2243] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.168390 27934 raft_consensus.cc:2272] T 40ba6b5dcd674422a95eeaad0efd1e45 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.170220 27934 tablet_replica.cc:333] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.170724 27934 raft_consensus.cc:2243] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.171593 27934 raft_consensus.cc:2272] T adfec13bcbce4db48b5f0dba7a8e5345 P 792907f253c744aa9392e8d572f88e06 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.173609 27934 tablet_replica.cc:333] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.174139 27934 raft_consensus.cc:2243] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.174912 27934 raft_consensus.cc:2272] T 4890d4f381d74b37a99616c1b1dc65d4 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.176994 27934 tablet_replica.cc:333] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.177546 27934 raft_consensus.cc:2243] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.178337 27934 raft_consensus.cc:2272] T b37c67a7062e4372b72045052c828e2d P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.180393 27934 tablet_replica.cc:333] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.180909 27934 raft_consensus.cc:2243] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.181308 27934 raft_consensus.cc:2272] T 892918414c974de38c7050ccd7151f94 P 792907f253c744aa9392e8d572f88e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.183180 27934 tablet_replica.cc:333] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.183694 27934 raft_consensus.cc:2243] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.184193 27934 raft_consensus.cc:2272] T c7f81180a2c94fe5874f1a402e44c52f P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.186101 27934 tablet_replica.cc:333] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.186642 27934 raft_consensus.cc:2243] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.187038 27934 raft_consensus.cc:2272] T a9e48769916844ce8d464a5bd1bda6de P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.189023 27934 tablet_replica.cc:333] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06: stopping tablet replica
I20260709 13:08:46.189455 27934 raft_consensus.cc:2243] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:46.189878 27934 raft_consensus.cc:2272] T e78f9ad8be2a4dd6bf2aa1d922e471a2 P 792907f253c744aa9392e8d572f88e06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.264258 27934 tablet_server.cc:196] TabletServer@127.27.71.132:0 shutdown complete.
I20260709 13:08:46.332767 27934 master.cc:562] Master@127.27.71.190:45203 shutting down...
W20260709 13:08:46.360141  5646 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 13:08:46.361343 27934 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:46.361891 27934 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:46.362218 27934 tablet_replica.cc:333] T 00000000000000000000000000000000 P c783fb2b5e494764af12cd3d23090672: stopping tablet replica
I20260709 13:08:46.403262 27934 master.cc:584] Master@127.27.71.190:45203 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (54365 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:08:46.516920 27934 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.71.190:43555
I20260709 13:08:46.518152 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:46.524355  7369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:46.524461  7370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:46.525516  7372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:46.526136 27934 server_base.cc:1061] running on GCE node
I20260709 13:08:46.528283 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:46.528569 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:46.528753 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602526528731 us; error 0 us; skew 500 ppm
I20260709 13:08:46.529752 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:46.533360 27934 webserver.cc:533] Webserver started at http://127.27.71.190:37469/ using document root <none> and password file <none>
I20260709 13:08:46.534103 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:46.534353 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:46.534693 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:46.536398 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/master-0-root/instance:
uuid: "e4ef72a1896142cdadf26f51300d52a4"
format_stamp: "Formatted at 2026-07-09 13:08:46 on dist-test-slave-mlt7"
I20260709 13:08:46.543103 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:08:46.548548  7377 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:46.549624 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:08:46.550040 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/master-0-root
uuid: "e4ef72a1896142cdadf26f51300d52a4"
format_stamp: "Formatted at 2026-07-09 13:08:46 on dist-test-slave-mlt7"
I20260709 13:08:46.550436 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:46.584462 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:46.586364 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:46.632766 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.190:43555
I20260709 13:08:46.632900  7428 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.190:43555 every 8 connection(s)
I20260709 13:08:46.637388  7429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:46.649997  7429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4: Bootstrap starting.
I20260709 13:08:46.655596  7429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:46.661031  7429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4: No bootstrap required, opened a new log
I20260709 13:08:46.663549  7429 raft_consensus.cc:359] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef72a1896142cdadf26f51300d52a4" member_type: VOTER }
I20260709 13:08:46.664095  7429 raft_consensus.cc:385] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:46.664323  7429 raft_consensus.cc:740] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4ef72a1896142cdadf26f51300d52a4, State: Initialized, Role: FOLLOWER
I20260709 13:08:46.665231  7429 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef72a1896142cdadf26f51300d52a4" member_type: VOTER }
I20260709 13:08:46.665946  7429 raft_consensus.cc:399] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:08:46.666181  7429 raft_consensus.cc:493] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:08:46.666500  7429 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:46.673195  7429 raft_consensus.cc:515] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef72a1896142cdadf26f51300d52a4" member_type: VOTER }
I20260709 13:08:46.673799  7429 leader_election.cc:304] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4ef72a1896142cdadf26f51300d52a4; no voters: 
I20260709 13:08:46.675042  7429 leader_election.cc:290] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:08:46.675592  7432 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:46.677667  7432 raft_consensus.cc:697] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [term 1 LEADER]: Becoming Leader. State: Replica: e4ef72a1896142cdadf26f51300d52a4, State: Running, Role: LEADER
I20260709 13:08:46.678355  7432 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef72a1896142cdadf26f51300d52a4" member_type: VOTER }
I20260709 13:08:46.678578  7429 sys_catalog.cc:565] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:08:46.684842  7433 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4ef72a1896142cdadf26f51300d52a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef72a1896142cdadf26f51300d52a4" member_type: VOTER } }
I20260709 13:08:46.685115  7434 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4ef72a1896142cdadf26f51300d52a4. Latest consensus state: current_term: 1 leader_uuid: "e4ef72a1896142cdadf26f51300d52a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ef72a1896142cdadf26f51300d52a4" member_type: VOTER } }
I20260709 13:08:46.686017  7434 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:46.685534  7433 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:46.689157  7439 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:08:46.695685  7439 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:08:46.700007 27934 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:08:46.706727  7439 catalog_manager.cc:1383] Generated new cluster ID: af365ba6472b4d67bf24df09e3995095
I20260709 13:08:46.707053  7439 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:08:46.735670  7439 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:08:46.737115  7439 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:08:46.763700  7439 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4: Generated new TSK 0
I20260709 13:08:46.764429  7439 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:08:46.832824 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:46.839668  7450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:46.843290  7451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:46.849872 27934 server_base.cc:1061] running on GCE node
W20260709 13:08:46.852824  7453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:46.854489 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:46.854775 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:46.854964 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602526854948 us; error 0 us; skew 500 ppm
I20260709 13:08:46.855510 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:46.858295 27934 webserver.cc:533] Webserver started at http://127.27.71.129:43115/ using document root <none> and password file <none>
I20260709 13:08:46.858870 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:46.859102 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:46.859411 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:46.860657 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-0-root/instance:
uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
format_stamp: "Formatted at 2026-07-09 13:08:46 on dist-test-slave-mlt7"
I20260709 13:08:46.865617 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:08:46.869349  7458 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:46.870285 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:08:46.870637 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-0-root
uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
format_stamp: "Formatted at 2026-07-09 13:08:46 on dist-test-slave-mlt7"
I20260709 13:08:46.870906 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:46.890064 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:46.891289 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:46.892897 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:46.895304 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:46.895521 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:46.895774 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:46.895998 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:08:46.939348 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.129:42053
I20260709 13:08:46.939455  7520 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.129:42053 every 8 connection(s)
I20260709 13:08:46.944689 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:46.952699  7526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:46.954092  7525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:46.955716  7528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:46.956787 27934 server_base.cc:1061] running on GCE node
I20260709 13:08:46.958560 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:46.958810 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:46.959000 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602526958983 us; error 0 us; skew 500 ppm
I20260709 13:08:46.959578 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:46.962466 27934 webserver.cc:533] Webserver started at http://127.27.71.130:44787/ using document root <none> and password file <none>
I20260709 13:08:46.963119 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:46.963357 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:46.963694 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:46.965193  7521 heartbeater.cc:344] Connected to a master server at 127.27.71.190:43555
I20260709 13:08:46.965276 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-1-root/instance:
uuid: "55cabad661424707b66c6ddd198939b1"
format_stamp: "Formatted at 2026-07-09 13:08:46 on dist-test-slave-mlt7"
I20260709 13:08:46.965781  7521 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:46.966598  7521 heartbeater.cc:507] Master 127.27.71.190:43555 requested a full tablet report, sending...
I20260709 13:08:46.969082  7394 ts_manager.cc:194] Registered new tserver with Master: f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053)
I20260709 13:08:46.970984 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:08:46.971551  7394 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46644
I20260709 13:08:46.975459  7533 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:46.976387 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:08:46.976696 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-1-root
uuid: "55cabad661424707b66c6ddd198939b1"
format_stamp: "Formatted at 2026-07-09 13:08:46 on dist-test-slave-mlt7"
I20260709 13:08:46.976981 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:46.988270 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:46.989549 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:46.991236 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:46.995077 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:46.995262 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:46.995445 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:46.995574 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:47.039721 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.130:46359
I20260709 13:08:47.039858  7595 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.130:46359 every 8 connection(s)
I20260709 13:08:47.045131 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:47.057206  7599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:47.058427  7600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:47.060770  7596 heartbeater.cc:344] Connected to a master server at 127.27.71.190:43555
I20260709 13:08:47.061393  7596 heartbeater.cc:461] Registering TS with master...
W20260709 13:08:47.062050  7602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:47.062094 27934 server_base.cc:1061] running on GCE node
I20260709 13:08:47.062600  7596 heartbeater.cc:507] Master 127.27.71.190:43555 requested a full tablet report, sending...
I20260709 13:08:47.063524 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:47.063815 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:47.064128 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602527064106 us; error 0 us; skew 500 ppm
I20260709 13:08:47.064917 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:47.065299  7394 ts_manager.cc:194] Registered new tserver with Master: 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:47.067144  7394 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46658
I20260709 13:08:47.067755 27934 webserver.cc:533] Webserver started at http://127.27.71.131:39569/ using document root <none> and password file <none>
I20260709 13:08:47.068498 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:47.068746 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:47.069082 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:47.070441 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-2-root/instance:
uuid: "69c9355bd3b449d088b37517aed3a2a4"
format_stamp: "Formatted at 2026-07-09 13:08:47 on dist-test-slave-mlt7"
I20260709 13:08:47.075515 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:08:47.079473  7607 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:47.080572 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:08:47.080875 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-2-root
uuid: "69c9355bd3b449d088b37517aed3a2a4"
format_stamp: "Formatted at 2026-07-09 13:08:47 on dist-test-slave-mlt7"
I20260709 13:08:47.081151 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:47.097864 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:47.099077 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:47.100557 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:47.103150 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:47.103344 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:47.103789 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:47.104043 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.001s
I20260709 13:08:47.149374 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.131:45237
I20260709 13:08:47.149515  7669 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.131:45237 every 8 connection(s)
I20260709 13:08:47.167198  7670 heartbeater.cc:344] Connected to a master server at 127.27.71.190:43555
I20260709 13:08:47.167693  7670 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:47.168520  7670 heartbeater.cc:507] Master 127.27.71.190:43555 requested a full tablet report, sending...
I20260709 13:08:47.170589  7394 ts_manager.cc:194] Registered new tserver with Master: 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:47.170724 27934 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01728866s
I20260709 13:08:47.172097  7394 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46664
I20260709 13:08:47.198714  7394 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46668:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:08:47.204008  7394 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:08:47.290460  7635 tablet_service.cc:1511] Processing CreateTablet for tablet 23bbc33e47fb45e38236cc96db1d262f (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:08:47.292097  7635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23bbc33e47fb45e38236cc96db1d262f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.295483  7486 tablet_service.cc:1511] Processing CreateTablet for tablet 23bbc33e47fb45e38236cc96db1d262f (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:08:47.297046  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23bbc33e47fb45e38236cc96db1d262f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.301766  7634 tablet_service.cc:1511] Processing CreateTablet for tablet ee14275e88d648c6982fe9fc771ed0b7 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:08:47.302184  7633 tablet_service.cc:1511] Processing CreateTablet for tablet a34c585c5de24649b746f9cd53cc8e57 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:08:47.303336  7634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee14275e88d648c6982fe9fc771ed0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.304921  7632 tablet_service.cc:1511] Processing CreateTablet for tablet d7aca33050e64b7595613a13182a0738 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:08:47.306453  7632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7aca33050e64b7595613a13182a0738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.308959  7633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a34c585c5de24649b746f9cd53cc8e57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.318794  7631 tablet_service.cc:1511] Processing CreateTablet for tablet 16926f7e3623437c9c6d389bb22ecb59 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:08:47.320962  7631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16926f7e3623437c9c6d389bb22ecb59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.322273  7485 tablet_service.cc:1511] Processing CreateTablet for tablet ee14275e88d648c6982fe9fc771ed0b7 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:08:47.323746  7485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee14275e88d648c6982fe9fc771ed0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.325557  7630 tablet_service.cc:1511] Processing CreateTablet for tablet ad99cf019e854bea806a64aaabd61f70 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:08:47.327494  7630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad99cf019e854bea806a64aaabd61f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.330101  7629 tablet_service.cc:1511] Processing CreateTablet for tablet 68a7ef75b1e34928b01a24437f9ab01f (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:08:47.332958  7484 tablet_service.cc:1511] Processing CreateTablet for tablet a34c585c5de24649b746f9cd53cc8e57 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:08:47.334373  7484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a34c585c5de24649b746f9cd53cc8e57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.334035  7628 tablet_service.cc:1511] Processing CreateTablet for tablet 15cc71669e814555aa12d1e2eb776c95 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:08:47.338094  7627 tablet_service.cc:1511] Processing CreateTablet for tablet 3f39053c868b4252acca47e18d123fb0 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:08:47.344030  7483 tablet_service.cc:1511] Processing CreateTablet for tablet d7aca33050e64b7595613a13182a0738 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:08:47.345541  7483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7aca33050e64b7595613a13182a0738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.346967  7626 tablet_service.cc:1511] Processing CreateTablet for tablet 7bc65eac152241638209dea1a9158863 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:08:47.349218  7482 tablet_service.cc:1511] Processing CreateTablet for tablet 16926f7e3623437c9c6d389bb22ecb59 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:08:47.350677  7482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16926f7e3623437c9c6d389bb22ecb59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.352192  7481 tablet_service.cc:1511] Processing CreateTablet for tablet ad99cf019e854bea806a64aaabd61f70 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:08:47.353691  7555 tablet_service.cc:1511] Processing CreateTablet for tablet 68a7ef75b1e34928b01a24437f9ab01f (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:08:47.354573  7561 tablet_service.cc:1511] Processing CreateTablet for tablet 23bbc33e47fb45e38236cc96db1d262f (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:08:47.355151  7555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a7ef75b1e34928b01a24437f9ab01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.357173  7560 tablet_service.cc:1511] Processing CreateTablet for tablet ee14275e88d648c6982fe9fc771ed0b7 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:08:47.357473  7554 tablet_service.cc:1511] Processing CreateTablet for tablet 15cc71669e814555aa12d1e2eb776c95 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:08:47.359567  7559 tablet_service.cc:1511] Processing CreateTablet for tablet a34c585c5de24649b746f9cd53cc8e57 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:08:47.360414  7553 tablet_service.cc:1511] Processing CreateTablet for tablet 3f39053c868b4252acca47e18d123fb0 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:08:47.362025  7557 tablet_service.cc:1511] Processing CreateTablet for tablet 16926f7e3623437c9c6d389bb22ecb59 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:08:47.363413  7552 tablet_service.cc:1511] Processing CreateTablet for tablet 7bc65eac152241638209dea1a9158863 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:08:47.364270  7556 tablet_service.cc:1511] Processing CreateTablet for tablet ad99cf019e854bea806a64aaabd61f70 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:08:47.367033  7558 tablet_service.cc:1511] Processing CreateTablet for tablet d7aca33050e64b7595613a13182a0738 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:08:47.370561  7480 tablet_service.cc:1511] Processing CreateTablet for tablet 68a7ef75b1e34928b01a24437f9ab01f (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:08:47.372985  7479 tablet_service.cc:1511] Processing CreateTablet for tablet 15cc71669e814555aa12d1e2eb776c95 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:08:47.374241  7685 tablet_bootstrap.cc:492] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4: Bootstrap starting.
I20260709 13:08:47.375384  7478 tablet_service.cc:1511] Processing CreateTablet for tablet 3f39053c868b4252acca47e18d123fb0 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:08:47.378118  7561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23bbc33e47fb45e38236cc96db1d262f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.378451  7477 tablet_service.cc:1511] Processing CreateTablet for tablet 7bc65eac152241638209dea1a9158863 (DEFAULT_TABLE table=test-workload [id=837fb0986ef842c9a0c1e00d89b01eb7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:08:47.379488  7480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a7ef75b1e34928b01a24437f9ab01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.380800  7477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bc65eac152241638209dea1a9158863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.381479  7478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f39053c868b4252acca47e18d123fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.378832  7556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad99cf019e854bea806a64aaabd61f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.383991  7552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bc65eac152241638209dea1a9158863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.344435  7627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f39053c868b4252acca47e18d123fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.388278  7479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15cc71669e814555aa12d1e2eb776c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.391441  7558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7aca33050e64b7595613a13182a0738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.396291  7481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad99cf019e854bea806a64aaabd61f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.400560  7557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16926f7e3623437c9c6d389bb22ecb59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.404320  7559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a34c585c5de24649b746f9cd53cc8e57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.406134  7560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee14275e88d648c6982fe9fc771ed0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.407629  7626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bc65eac152241638209dea1a9158863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.410504  7554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15cc71669e814555aa12d1e2eb776c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.416584  7628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15cc71669e814555aa12d1e2eb776c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.424211  7553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f39053c868b4252acca47e18d123fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.425733  7629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a7ef75b1e34928b01a24437f9ab01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:47.482466  7685 tablet_bootstrap.cc:654] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.499946  7686 tablet_bootstrap.cc:492] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:47.512247  7685 tablet_bootstrap.cc:492] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4: No bootstrap required, opened a new log
I20260709 13:08:47.515872  7685 ts_tablet_manager.cc:1403] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4: Time spent bootstrapping tablet: real 0.142s	user 0.016s	sys 0.000s
I20260709 13:08:47.517372  7688 tablet_bootstrap.cc:492] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1: Bootstrap starting.
I20260709 13:08:47.520959  7685 raft_consensus.cc:359] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:47.521737  7685 raft_consensus.cc:385] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.522061  7685 raft_consensus.cc:740] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.522907  7685 consensus_queue.cc:260] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:47.523801  7688 tablet_bootstrap.cc:654] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.525887  7685 ts_tablet_manager.cc:1434] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:08:47.526324  7686 tablet_bootstrap.cc:654] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.528370  7670 heartbeater.cc:499] Master 127.27.71.190:43555 was elected leader, sending a full tablet report...
I20260709 13:08:47.530496  7685 tablet_bootstrap.cc:492] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4: Bootstrap starting.
I20260709 13:08:47.532269  7688 tablet_bootstrap.cc:492] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1: No bootstrap required, opened a new log
I20260709 13:08:47.532275  7686 tablet_bootstrap.cc:492] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d: No bootstrap required, opened a new log
I20260709 13:08:47.532833  7688 ts_tablet_manager.cc:1403] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 13:08:47.533257  7686 ts_tablet_manager.cc:1403] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.003s
I20260709 13:08:47.535847  7686 raft_consensus.cc:359] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:47.536105  7688 raft_consensus.cc:359] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.536711  7686 raft_consensus.cc:385] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.536962  7688 raft_consensus.cc:385] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.537073  7686 raft_consensus.cc:740] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.537482  7688 raft_consensus.cc:740] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.537969  7686 consensus_queue.cc:260] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:47.538326  7688 consensus_queue.cc:260] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.539314  7685 tablet_bootstrap.cc:654] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.548671  7686 ts_tablet_manager.cc:1434] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.005s
I20260709 13:08:47.550734  7521 heartbeater.cc:499] Master 127.27.71.190:43555 was elected leader, sending a full tablet report...
I20260709 13:08:47.551342  7688 ts_tablet_manager.cc:1434] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.009s
I20260709 13:08:47.552762  7686 tablet_bootstrap.cc:492] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:47.557379  7596 heartbeater.cc:499] Master 127.27.71.190:43555 was elected leader, sending a full tablet report...
I20260709 13:08:47.559784  7686 tablet_bootstrap.cc:654] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.562335  7688 tablet_bootstrap.cc:492] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1: Bootstrap starting.
I20260709 13:08:47.566129  7685 tablet_bootstrap.cc:492] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4: No bootstrap required, opened a new log
I20260709 13:08:47.566766  7685 ts_tablet_manager.cc:1403] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.007s
I20260709 13:08:47.569962  7685 raft_consensus.cc:359] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.570739  7685 raft_consensus.cc:385] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.571035  7685 raft_consensus.cc:740] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.571995  7685 consensus_queue.cc:260] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.573567  7688 tablet_bootstrap.cc:654] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.574489  7685 ts_tablet_manager.cc:1434] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:08:47.580528  7685 tablet_bootstrap.cc:492] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4: Bootstrap starting.
I20260709 13:08:47.581602  7686 tablet_bootstrap.cc:492] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d: No bootstrap required, opened a new log
I20260709 13:08:47.582146  7686 ts_tablet_manager.cc:1403] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.017s
I20260709 13:08:47.585342  7686 raft_consensus.cc:359] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.586108  7686 raft_consensus.cc:385] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.586408  7686 raft_consensus.cc:740] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.587108  7686 consensus_queue.cc:260] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.588433  7685 tablet_bootstrap.cc:654] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.590152  7686 ts_tablet_manager.cc:1434] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:08:47.591024  7688 tablet_bootstrap.cc:492] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1: No bootstrap required, opened a new log
I20260709 13:08:47.591367  7686 tablet_bootstrap.cc:492] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:47.591567  7688 ts_tablet_manager.cc:1403] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.004s
I20260709 13:08:47.595491  7689 raft_consensus.cc:493] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.595429  7688 raft_consensus.cc:359] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.596496  7688 raft_consensus.cc:385] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.596144  7689 raft_consensus.cc:515] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:47.596748  7685 tablet_bootstrap.cc:492] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4: No bootstrap required, opened a new log
I20260709 13:08:47.596961  7688 raft_consensus.cc:740] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.597573  7685 ts_tablet_manager.cc:1403] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.001s
I20260709 13:08:47.598302  7688 consensus_queue.cc:260] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.600754  7688 ts_tablet_manager.cc:1434] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:08:47.601100  7685 raft_consensus.cc:359] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.601615  7693 raft_consensus.cc:493] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.601926  7688 tablet_bootstrap.cc:492] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1: Bootstrap starting.
I20260709 13:08:47.602180  7693 raft_consensus.cc:515] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.604948  7693 leader_election.cc:290] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:47.606001  7689 leader_election.cc:290] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:47.601959  7685 raft_consensus.cc:385] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.611943  7685 raft_consensus.cc:740] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.619577  7685 consensus_queue.cc:260] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.628028  7688 tablet_bootstrap.cc:654] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.628361  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bbc33e47fb45e38236cc96db1d262f" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" is_pre_election: true
I20260709 13:08:47.629222  7496 raft_consensus.cc:2468] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 0.
I20260709 13:08:47.630705  7610 leader_election.cc:304] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:47.630203  7686 tablet_bootstrap.cc:654] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.631485  7685 ts_tablet_manager.cc:1434] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.000s
I20260709 13:08:47.636905  7689 raft_consensus.cc:2804] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:47.637379  7689 raft_consensus.cc:493] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:47.637733  7689 raft_consensus.cc:3060] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:47.637846  7685 tablet_bootstrap.cc:492] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4: Bootstrap starting.
I20260709 13:08:47.648729  7685 tablet_bootstrap.cc:654] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.649181  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bbc33e47fb45e38236cc96db1d262f" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
W20260709 13:08:47.651096  7610 leader_election.cc:343] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:47.654596  7671 tablet.cc:2406] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:08:47.668361  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34c585c5de24649b746f9cd53cc8e57" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
I20260709 13:08:47.670217  7685 tablet_bootstrap.cc:492] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4: No bootstrap required, opened a new log
I20260709 13:08:47.670809  7685 ts_tablet_manager.cc:1403] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.008s
W20260709 13:08:47.670909  7534 leader_election.cc:343] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.675767  7689 raft_consensus.cc:515] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:47.676440  7686 tablet_bootstrap.cc:492] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d: No bootstrap required, opened a new log
I20260709 13:08:47.676466  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34c585c5de24649b746f9cd53cc8e57" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" is_pre_election: true
I20260709 13:08:47.677168  7686 ts_tablet_manager.cc:1403] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.086s	user 0.014s	sys 0.011s
I20260709 13:08:47.678671  7689 leader_election.cc:290] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
W20260709 13:08:47.679054  7536 leader_election.cc:343] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.679783  7536 leader_election.cc:304] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55cabad661424707b66c6ddd198939b1; no voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:47.681149  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bbc33e47fb45e38236cc96db1d262f" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:47.681483  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bbc33e47fb45e38236cc96db1d262f" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:47.682056  7686 raft_consensus.cc:359] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.682986  7686 raft_consensus.cc:385] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.683713  7686 raft_consensus.cc:740] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.683848  7688 tablet_bootstrap.cc:492] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1: No bootstrap required, opened a new log
I20260709 13:08:47.682961  7698 raft_consensus.cc:2749] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:08:47.684144  7610 leader_election.cc:343] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.684489  7688 ts_tablet_manager.cc:1403] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1: Time spent bootstrapping tablet: real 0.083s	user 0.024s	sys 0.026s
I20260709 13:08:47.683136  7685 raft_consensus.cc:359] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.681773  7496 raft_consensus.cc:3060] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:47.684684  7686 consensus_queue.cc:260] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.685137  7685 raft_consensus.cc:385] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.685657  7685 raft_consensus.cc:740] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.686728  7685 consensus_queue.cc:260] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.689165  7686 ts_tablet_manager.cc:1434] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:08:47.690623  7686 tablet_bootstrap.cc:492] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:47.692838  7685 ts_tablet_manager.cc:1434] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4: Time spent starting tablet: real 0.022s	user 0.011s	sys 0.000s
I20260709 13:08:47.693153  7688 raft_consensus.cc:359] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.694052  7688 raft_consensus.cc:385] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.694151  7685 tablet_bootstrap.cc:492] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4: Bootstrap starting.
I20260709 13:08:47.694384  7688 raft_consensus.cc:740] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Initialized, Role: FOLLOWER
W20260709 13:08:47.695293  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:47.695325  7688 consensus_queue.cc:260] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.697082  7496 raft_consensus.cc:2468] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 1.
I20260709 13:08:47.698329  7688 ts_tablet_manager.cc:1434] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
I20260709 13:08:47.699429  7610 leader_election.cc:304] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 55cabad661424707b66c6ddd198939b1
I20260709 13:08:47.700747  7689 raft_consensus.cc:2804] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:47.703217  7686 tablet_bootstrap.cc:654] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.713704  7689 raft_consensus.cc:697] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:47.715003  7689 consensus_queue.cc:237] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:47.716854  7685 tablet_bootstrap.cc:654] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.722654  7688 tablet_bootstrap.cc:492] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1: Bootstrap starting.
I20260709 13:08:47.726148  7685 tablet_bootstrap.cc:492] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4: No bootstrap required, opened a new log
I20260709 13:08:47.726788  7685 ts_tablet_manager.cc:1403] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.000s
I20260709 13:08:47.730521  7685 raft_consensus.cc:359] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.731340  7685 raft_consensus.cc:385] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.731689  7685 raft_consensus.cc:740] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.732890  7698 raft_consensus.cc:493] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.733529  7698 raft_consensus.cc:515] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.734951  7692 raft_consensus.cc:493] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.734505  7394 catalog_manager.cc:5697] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:47.735462  7692 raft_consensus.cc:515] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.736625  7698 leader_election.cc:290] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:47.736956  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a7ef75b1e34928b01a24437f9ab01f" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" is_pre_election: true
I20260709 13:08:47.737342  7685 consensus_queue.cc:260] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
W20260709 13:08:47.738881  7536 leader_election.cc:343] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.740100  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a7ef75b1e34928b01a24437f9ab01f" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
I20260709 13:08:47.740506  7685 ts_tablet_manager.cc:1434] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.003s
W20260709 13:08:47.741509  7534 leader_election.cc:343] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.741772  7685 tablet_bootstrap.cc:492] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4: Bootstrap starting.
I20260709 13:08:47.742053  7534 leader_election.cc:304] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55cabad661424707b66c6ddd198939b1; no voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:47.743350  7698 raft_consensus.cc:2749] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:47.748839  7692 leader_election.cc:290] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:47.749003  7685 tablet_bootstrap.cc:654] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.749085  7688 tablet_bootstrap.cc:654] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.760100  7685 tablet_bootstrap.cc:492] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4: No bootstrap required, opened a new log
I20260709 13:08:47.760129  7686 tablet_bootstrap.cc:492] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d: No bootstrap required, opened a new log
I20260709 13:08:47.760869  7685 ts_tablet_manager.cc:1403] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4: Time spent bootstrapping tablet: real 0.019s	user 0.004s	sys 0.013s
I20260709 13:08:47.760959  7686 ts_tablet_manager.cc:1403] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.071s	user 0.037s	sys 0.005s
I20260709 13:08:47.764832  7686 raft_consensus.cc:359] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.765072  7685 raft_consensus.cc:359] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.765626  7686 raft_consensus.cc:385] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.765975  7685 raft_consensus.cc:385] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.766251  7686 raft_consensus.cc:740] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.766388  7685 raft_consensus.cc:740] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.766201  7688 tablet_bootstrap.cc:492] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1: No bootstrap required, opened a new log
I20260709 13:08:47.768218  7688 ts_tablet_manager.cc:1403] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.000s
I20260709 13:08:47.767637  7685 consensus_queue.cc:260] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.768378  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aca33050e64b7595613a13182a0738" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:47.769099  7571 raft_consensus.cc:2468] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 0.
I20260709 13:08:47.770526  7685 ts_tablet_manager.cc:1434] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 13:08:47.770539  7461 leader_election.cc:304] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:47.767179  7686 consensus_queue.cc:260] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.771816  7685 tablet_bootstrap.cc:492] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4: Bootstrap starting.
I20260709 13:08:47.772230  7692 raft_consensus.cc:2804] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:47.772708  7692 raft_consensus.cc:493] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:47.773059  7692 raft_consensus.cc:3060] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:47.774004  7688 raft_consensus.cc:359] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.774801  7688 raft_consensus.cc:385] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.775085  7688 raft_consensus.cc:740] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.775429  7689 raft_consensus.cc:493] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.775962  7689 raft_consensus.cc:515] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.775876  7688 consensus_queue.cc:260] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.779793  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" is_pre_election: true
I20260709 13:08:47.780453  7685 tablet_bootstrap.cc:654] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.781350  7688 ts_tablet_manager.cc:1434] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
W20260709 13:08:47.781493  7610 leader_election.cc:343] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.782612  7688 tablet_bootstrap.cc:492] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1: Bootstrap starting.
I20260709 13:08:47.784821  7692 raft_consensus.cc:515] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.787813  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aca33050e64b7595613a13182a0738" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:47.788444  7571 raft_consensus.cc:3060] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:47.789119  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aca33050e64b7595613a13182a0738" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
I20260709 13:08:47.789825  7645 raft_consensus.cc:2468] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 0.
I20260709 13:08:47.791802  7688 tablet_bootstrap.cc:654] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.797017  7571 raft_consensus.cc:2468] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 1.
I20260709 13:08:47.798401  7461 leader_election.cc:304] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:47.802037  7689 leader_election.cc:290] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:47.806360  7717 raft_consensus.cc:2804] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:47.806931  7717 raft_consensus.cc:697] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 LEADER]: Becoming Leader. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Running, Role: LEADER
I20260709 13:08:47.809448  7686 ts_tablet_manager.cc:1434] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.003s
I20260709 13:08:47.810521  7686 tablet_bootstrap.cc:492] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:47.813732  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:47.807801  7717 consensus_queue.cc:237] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.816839  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aca33050e64b7595613a13182a0738" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:47.817477  7645 raft_consensus.cc:3060] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:47.819164  7689 raft_consensus.cc:493] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.819693  7689 raft_consensus.cc:515] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
W20260709 13:08:47.820387  7610 leader_election.cc:343] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.820858  7610 leader_election.cc:304] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4; no voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:47.821905  7689 leader_election.cc:290] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:47.829679  7692 leader_election.cc:290] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 election: Requested vote from peers 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:47.833891  7688 tablet_bootstrap.cc:492] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1: No bootstrap required, opened a new log
I20260709 13:08:47.834939  7688 ts_tablet_manager.cc:1403] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1: Time spent bootstrapping tablet: real 0.053s	user 0.016s	sys 0.033s
I20260709 13:08:47.835060  7706 raft_consensus.cc:2749] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:47.834357  7686 tablet_bootstrap.cc:654] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.835166  7685 tablet_bootstrap.cc:492] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4: No bootstrap required, opened a new log
I20260709 13:08:47.836102  7685 ts_tablet_manager.cc:1403] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4: Time spent bootstrapping tablet: real 0.065s	user 0.019s	sys 0.041s
I20260709 13:08:47.840039  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16926f7e3623437c9c6d389bb22ecb59" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" is_pre_election: true
I20260709 13:08:47.841696  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16926f7e3623437c9c6d389bb22ecb59" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:47.842417  7571 raft_consensus.cc:2468] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 0.
I20260709 13:08:47.845220  7688 raft_consensus.cc:359] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.845954  7645 raft_consensus.cc:2468] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 1.
I20260709 13:08:47.846426  7688 raft_consensus.cc:385] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.847286  7688 raft_consensus.cc:740] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Initialized, Role: FOLLOWER
W20260709 13:08:47.846481  7610 leader_election.cc:343] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.848433  7688 consensus_queue.cc:260] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.849951  7610 leader_election.cc:304] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:47.850956  7706 raft_consensus.cc:2804] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:47.851329  7706 raft_consensus.cc:493] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:47.851677  7706 raft_consensus.cc:3060] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:47.852193  7688 ts_tablet_manager.cc:1434] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 13:08:47.854024  7688 tablet_bootstrap.cc:492] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1: Bootstrap starting.
I20260709 13:08:47.855798  7394 catalog_manager.cc:5697] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d reported cstate change: term changed from 0 to 1, leader changed from <none> to f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129). New cstate: current_term: 1 leader_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:47.860215  7685 raft_consensus.cc:359] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.861223  7685 raft_consensus.cc:385] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.861701  7685 raft_consensus.cc:740] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.863512  7686 tablet_bootstrap.cc:492] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: No bootstrap required, opened a new log
I20260709 13:08:47.864149  7686 ts_tablet_manager.cc:1403] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.014s
I20260709 13:08:47.863734  7685 consensus_queue.cc:260] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.866225  7688 tablet_bootstrap.cc:654] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.866349  7685 ts_tablet_manager.cc:1434] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:08:47.867663  7685 tablet_bootstrap.cc:492] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4: Bootstrap starting.
I20260709 13:08:47.870411  7686 raft_consensus.cc:359] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.871426  7686 raft_consensus.cc:385] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.871965  7686 raft_consensus.cc:740] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.873207  7686 consensus_queue.cc:260] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.874811  7685 tablet_bootstrap.cc:654] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.861301  7706 raft_consensus.cc:515] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.879071  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16926f7e3623437c9c6d389bb22ecb59" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:47.883839  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16926f7e3623437c9c6d389bb22ecb59" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:47.884827  7571 raft_consensus.cc:3060] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:47.887423  7610 leader_election.cc:343] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.891278  7686 ts_tablet_manager.cc:1434] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.004s
I20260709 13:08:47.897171  7698 raft_consensus.cc:493] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.897848  7698 raft_consensus.cc:515] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.898870  7706 leader_election.cc:290] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:47.900213  7686 tablet_bootstrap.cc:492] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:47.900115  7698 leader_election.cc:290] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:47.904090  7685 tablet_bootstrap.cc:492] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4: No bootstrap required, opened a new log
I20260709 13:08:47.904752  7685 ts_tablet_manager.cc:1403] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.016s
I20260709 13:08:47.903560  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
I20260709 13:08:47.901904  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" is_pre_election: true
I20260709 13:08:47.905390  7645 raft_consensus.cc:2468] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 0.
W20260709 13:08:47.906349  7536 leader_election.cc:343] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.902966  7571 raft_consensus.cc:2468] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 1.
I20260709 13:08:47.904919  7688 tablet_bootstrap.cc:492] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1: No bootstrap required, opened a new log
I20260709 13:08:47.908053  7688 ts_tablet_manager.cc:1403] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1: Time spent bootstrapping tablet: real 0.054s	user 0.026s	sys 0.014s
I20260709 13:08:47.908380  7685 raft_consensus.cc:359] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.909195  7685 raft_consensus.cc:385] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.909536  7685 raft_consensus.cc:740] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.910411  7685 consensus_queue.cc:260] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.911366  7686 tablet_bootstrap.cc:654] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.911464  7689 raft_consensus.cc:493] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.910444  7610 leader_election.cc:304] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:47.912312  7689 raft_consensus.cc:515] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.912381  7534 leader_election.cc:304] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:47.914757  7706 raft_consensus.cc:2804] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:47.915833  7698 raft_consensus.cc:2804] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:47.916349  7698 raft_consensus.cc:493] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:47.916811  7698 raft_consensus.cc:3060] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:47.911404  7688 raft_consensus.cc:359] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.914753  7689 leader_election.cc:290] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:47.920607  7688 raft_consensus.cc:385] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.920994  7688 raft_consensus.cc:740] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.922080  7688 consensus_queue.cc:260] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.921135  7689 raft_consensus.cc:493] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.923635  7689 raft_consensus.cc:515] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.929195  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" is_pre_election: true
I20260709 13:08:47.929488  7689 leader_election.cc:290] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:47.931509  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:47.931748  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee14275e88d648c6982fe9fc771ed0b7" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" is_pre_election: true
I20260709 13:08:47.932493  7496 raft_consensus.cc:2468] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 0.
W20260709 13:08:47.933900  7610 leader_election.cc:343] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.934444  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee14275e88d648c6982fe9fc771ed0b7" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:47.934667  7610 leader_election.cc:304] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
W20260709 13:08:47.935688  7610 leader_election.cc:343] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.936148  7610 leader_election.cc:304] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4; no voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:47.937157  7725 raft_consensus.cc:2804] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:08:47.937492  7610 leader_election.cc:343] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.937644  7725 raft_consensus.cc:493] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:47.938201  7725 raft_consensus.cc:3060] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:47.938585  7689 raft_consensus.cc:2749] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:47.940002  7688 ts_tablet_manager.cc:1434] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:08:47.940120  7685 ts_tablet_manager.cc:1434] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:08:47.941248  7688 tablet_bootstrap.cc:492] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1: Bootstrap starting.
I20260709 13:08:47.941636  7685 tablet_bootstrap.cc:492] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4: Bootstrap starting.
I20260709 13:08:47.947887  7706 raft_consensus.cc:697] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:47.949002  7706 consensus_queue.cc:237] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.950366  7725 raft_consensus.cc:515] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.953135  7725 leader_election.cc:290] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:47.954051  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee14275e88d648c6982fe9fc771ed0b7" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:47.954645  7496 raft_consensus.cc:3060] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:47.955382  7716 raft_consensus.cc:493] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.956249  7716 raft_consensus.cc:515] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.958678  7716 leader_election.cc:290] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:47.961838  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a7ef75b1e34928b01a24437f9ab01f" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:47.962531  7496 raft_consensus.cc:2468] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 1.
I20260709 13:08:47.962565  7571 raft_consensus.cc:2468] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 0.
I20260709 13:08:47.963430  7685 tablet_bootstrap.cc:654] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.955345  7698 raft_consensus.cc:515] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.964795  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a7ef75b1e34928b01a24437f9ab01f" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
I20260709 13:08:47.965541  7645 raft_consensus.cc:2468] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 0.
I20260709 13:08:47.966948  7461 leader_election.cc:304] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:47.968518  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee14275e88d648c6982fe9fc771ed0b7" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:47.969131  7698 leader_election.cc:290] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:47.970638  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:47.971266  7645 raft_consensus.cc:3060] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:47.971387  7686 tablet_bootstrap.cc:492] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d: No bootstrap required, opened a new log
I20260709 13:08:47.972054  7686 ts_tablet_manager.cc:1403] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.072s	user 0.024s	sys 0.044s
I20260709 13:08:47.973682  7394 catalog_manager.cc:5697] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:47.975597  7716 raft_consensus.cc:2804] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:47.976111  7716 raft_consensus.cc:493] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:47.976534  7716 raft_consensus.cc:3060] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:47.976786  7686 raft_consensus.cc:359] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.977842  7686 raft_consensus.cc:385] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.978235  7686 raft_consensus.cc:740] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.979651  7686 consensus_queue.cc:260] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.982821  7686 ts_tablet_manager.cc:1434] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:08:47.984753  7686 tablet_bootstrap.cc:492] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:47.986558  7685 tablet_bootstrap.cc:492] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4: No bootstrap required, opened a new log
I20260709 13:08:47.987108  7685 ts_tablet_manager.cc:1403] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4: Time spent bootstrapping tablet: real 0.046s	user 0.024s	sys 0.001s
I20260709 13:08:47.988027  7645 raft_consensus.cc:2468] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 1.
I20260709 13:08:47.990082  7534 leader_election.cc:304] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: 
W20260709 13:08:47.990607  7610 leader_election.cc:343] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.991008  7698 raft_consensus.cc:2804] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:47.991503  7610 leader_election.cc:304] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 55cabad661424707b66c6ddd198939b1
I20260709 13:08:47.991592  7686 tablet_bootstrap.cc:654] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:47.991586  7698 raft_consensus.cc:697] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 1 LEADER]: Becoming Leader. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:47.988828  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:47.993530  7725 raft_consensus.cc:493] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.994643  7725 raft_consensus.cc:515] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.995466  7685 raft_consensus.cc:359] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:47.996234  7685 raft_consensus.cc:385] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:47.996531  7685 raft_consensus.cc:740] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Initialized, Role: FOLLOWER
I20260709 13:08:47.997592  7725 leader_election.cc:290] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:47.994134  7706 raft_consensus.cc:2804] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:47.997411  7685 consensus_queue.cc:260] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
W20260709 13:08:47.998373  7536 leader_election.cc:343] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:47.993224  7698 consensus_queue.cc:237] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.000913  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34c585c5de24649b746f9cd53cc8e57" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:48.001753  7686 tablet_bootstrap.cc:492] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d: No bootstrap required, opened a new log
I20260709 13:08:48.002170  7571 raft_consensus.cc:2468] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 0.
I20260709 13:08:48.002329  7686 ts_tablet_manager.cc:1403] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.002s
I20260709 13:08:48.001384  7706 raft_consensus.cc:697] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:48.004364  7706 consensus_queue.cc:237] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.003281  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34c585c5de24649b746f9cd53cc8e57" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" is_pre_election: true
I20260709 13:08:48.005770  7686 raft_consensus.cc:359] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.006510  7686 raft_consensus.cc:385] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:48.006820  7686 raft_consensus.cc:740] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:48.007227  7610 leader_election.cc:304] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: 
I20260709 13:08:48.007679  7686 consensus_queue.cc:260] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.008679  7685 ts_tablet_manager.cc:1434] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 13:08:48.011430  7496 raft_consensus.cc:2468] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 0.
I20260709 13:08:48.011813  7685 tablet_bootstrap.cc:492] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4: Bootstrap starting.
I20260709 13:08:48.012178  7686 ts_tablet_manager.cc:1434] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:08:48.013365  7686 tablet_bootstrap.cc:492] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:48.010419  7725 raft_consensus.cc:2804] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:48.016755  7725 raft_consensus.cc:493] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.018425  7725 raft_consensus.cc:3060] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.019363  7688 tablet_bootstrap.cc:654] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:48.000066  7716 raft_consensus.cc:515] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.026986  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a7ef75b1e34928b01a24437f9ab01f" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:48.027894  7571 raft_consensus.cc:3060] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.019848  7686 tablet_bootstrap.cc:654] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:48.017716  7685 tablet_bootstrap.cc:654] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:48.040576  7716 leader_election.cc:290] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 election: Requested vote from peers 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:48.042517  7685 tablet_bootstrap.cc:492] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4: No bootstrap required, opened a new log
I20260709 13:08:48.043044  7685 ts_tablet_manager.cc:1403] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.005s
I20260709 13:08:48.043776  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a7ef75b1e34928b01a24437f9ab01f" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:48.044523  7645 raft_consensus.cc:3060] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.046008  7685 raft_consensus.cc:359] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.049434  7685 raft_consensus.cc:385] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:48.048590  7391 catalog_manager.cc:5697] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55cabad661424707b66c6ddd198939b1 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "55cabad661424707b66c6ddd198939b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:48.049726  7685 raft_consensus.cc:740] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Initialized, Role: FOLLOWER
I20260709 13:08:48.050823  7685 consensus_queue.cc:260] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.051379  7571 raft_consensus.cc:2468] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 1.
I20260709 13:08:48.051677  7394 catalog_manager.cc:5697] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:48.053391  7685 ts_tablet_manager.cc:1434] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:08:48.052728  7461 leader_election.cc:304] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:48.040272  7689 raft_consensus.cc:493] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.064254  7645 raft_consensus.cc:2468] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 1.
I20260709 13:08:48.064389  7689 raft_consensus.cc:515] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.066079  7688 tablet_bootstrap.cc:492] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1: No bootstrap required, opened a new log
I20260709 13:08:48.066567  7688 ts_tablet_manager.cc:1403] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1: Time spent bootstrapping tablet: real 0.126s	user 0.023s	sys 0.004s
I20260709 13:08:48.066756  7689 leader_election.cc:290] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:48.056708  7725 raft_consensus.cc:515] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.068971  7725 leader_election.cc:290] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:48.069005  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" is_pre_election: true
I20260709 13:08:48.069458  7686 tablet_bootstrap.cc:492] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: No bootstrap required, opened a new log
I20260709 13:08:48.069694  7716 raft_consensus.cc:2804] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:48.069772  7688 raft_consensus.cc:359] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.070315  7716 raft_consensus.cc:697] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 LEADER]: Becoming Leader. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Running, Role: LEADER
I20260709 13:08:48.070676  7688 raft_consensus.cc:385] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:48.070137  7686 ts_tablet_manager.cc:1403] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.005s
I20260709 13:08:48.071023  7688 raft_consensus.cc:740] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Initialized, Role: FOLLOWER
I20260709 13:08:48.071231  7716 consensus_queue.cc:237] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.072451  7570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34c585c5de24649b746f9cd53cc8e57" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:48.072345  7688 consensus_queue.cc:260] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.073134  7570 raft_consensus.cc:3060] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.074462  7686 raft_consensus.cc:359] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.075219  7686 raft_consensus.cc:385] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:48.075598  7686 raft_consensus.cc:740] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:48.077019  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a34c585c5de24649b746f9cd53cc8e57" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:48.077659  7496 raft_consensus.cc:3060] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.078048  7688 ts_tablet_manager.cc:1434] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 13:08:48.076493  7686 consensus_queue.cc:260] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.079414  7688 tablet_bootstrap.cc:492] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1: Bootstrap starting.
W20260709 13:08:48.080294  7610 leader_election.cc:343] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:48.071005  7717 raft_consensus.cc:493] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.081385  7717 raft_consensus.cc:515] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.086630  7686 ts_tablet_manager.cc:1434] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.008s
I20260709 13:08:48.088107  7686 tablet_bootstrap.cc:492] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:48.095810  7688 tablet_bootstrap.cc:654] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:48.096285  7686 tablet_bootstrap.cc:654] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:48.098786  7717 leader_election.cc:290] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:48.099220  7569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16926f7e3623437c9c6d389bb22ecb59" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:48.100188  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:48.100392  7569 raft_consensus.cc:2393] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f7f82d1c7a294db588c3fc4e48c6012d in current term 1: Already voted for candidate 69c9355bd3b449d088b37517aed3a2a4 in this term.
I20260709 13:08:48.100896  7571 raft_consensus.cc:2468] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 0.
I20260709 13:08:48.106187  7570 raft_consensus.cc:2468] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 1.
W20260709 13:08:48.108309  7610 consensus_peers.cc:597] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 -> Peer 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359): Couldn't send request to peer 55cabad661424707b66c6ddd198939b1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:48.109263  7610 leader_election.cc:304] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:48.113322  7725 raft_consensus.cc:2804] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:48.113852  7725 raft_consensus.cc:493] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.114287  7725 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.104995  7393 catalog_manager.cc:5697] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d reported cstate change: term changed from 0 to 1, leader changed from <none> to f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129). New cstate: current_term: 1 leader_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:48.090955  7496 raft_consensus.cc:2468] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 1.
I20260709 13:08:48.121312  7689 consensus_queue.cc:1048] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:48.121479  7610 leader_election.cc:304] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: 
I20260709 13:08:48.123085  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16926f7e3623437c9c6d389bb22ecb59" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
I20260709 13:08:48.123996  7706 raft_consensus.cc:2804] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:48.126118  7706 raft_consensus.cc:697] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:48.127308  7706 consensus_queue.cc:237] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.129355  7717 raft_consensus.cc:493] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.129854  7717 raft_consensus.cc:515] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.132421  7717 leader_election.cc:290] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:48.132881  7725 raft_consensus.cc:515] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.134279  7570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:48.134959  7570 raft_consensus.cc:2468] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 0.
I20260709 13:08:48.136245  7461 leader_election.cc:304] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:48.137200  7717 raft_consensus.cc:2804] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:48.137583  7717 raft_consensus.cc:493] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.137917  7717 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.138319  7725 leader_election.cc:290] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:48.141108  7459 leader_election.cc:304] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7f82d1c7a294db588c3fc4e48c6012d; no voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4
I20260709 13:08:48.143067  7716 raft_consensus.cc:3060] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.143728  7570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:48.144676  7570 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.157542  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
I20260709 13:08:48.158906  7645 raft_consensus.cc:2468] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 0.
I20260709 13:08:48.166298  7570 raft_consensus.cc:2468] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 1.
I20260709 13:08:48.168812  7725 raft_consensus.cc:493] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.169086  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:48.169565  7725 raft_consensus.cc:515] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.173792  7725 leader_election.cc:290] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:48.168118  7610 leader_election.cc:304] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: 
I20260709 13:08:48.176355  7725 raft_consensus.cc:2804] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:48.177091  7725 raft_consensus.cc:697] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:48.177592  7716 raft_consensus.cc:2749] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:48.178635  7725 consensus_queue.cc:237] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.183359  7569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
W20260709 13:08:48.183408  7610 leader_election.cc:343] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:48.186304  7610 leader_election.cc:343] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:48.187774  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" is_pre_election: true
I20260709 13:08:48.188850  7495 raft_consensus.cc:2468] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 0.
I20260709 13:08:48.193224  7610 leader_election.cc:304] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 55cabad661424707b66c6ddd198939b1
I20260709 13:08:48.193825  7688 tablet_bootstrap.cc:492] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1: No bootstrap required, opened a new log
I20260709 13:08:48.194204  7689 raft_consensus.cc:2804] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:48.194401  7688 ts_tablet_manager.cc:1403] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1: Time spent bootstrapping tablet: real 0.115s	user 0.043s	sys 0.049s
I20260709 13:08:48.194391  7686 tablet_bootstrap.cc:492] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d: No bootstrap required, opened a new log
I20260709 13:08:48.194811  7689 raft_consensus.cc:493] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.195263  7689 raft_consensus.cc:3060] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.201233  7688 raft_consensus.cc:359] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.202054  7688 raft_consensus.cc:385] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:48.202390  7688 raft_consensus.cc:740] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Initialized, Role: FOLLOWER
I20260709 13:08:48.203986  7686 ts_tablet_manager.cc:1403] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.116s	user 0.022s	sys 0.057s
I20260709 13:08:48.203435  7688 consensus_queue.cc:260] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.201086  7393 catalog_manager.cc:5697] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:48.205744  7689 raft_consensus.cc:515] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.207626  7686 raft_consensus.cc:359] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.208382  7686 raft_consensus.cc:385] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:48.208662  7686 raft_consensus.cc:740] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:48.209817  7686 consensus_queue.cc:260] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.210989  7688 ts_tablet_manager.cc:1434] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:08:48.212280  7688 tablet_bootstrap.cc:492] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1: Bootstrap starting.
I20260709 13:08:48.210320  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:48.213488  7686 ts_tablet_manager.cc:1434] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:08:48.213613  7495 raft_consensus.cc:3060] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.214636  7686 tablet_bootstrap.cc:492] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:48.216089  7689 leader_election.cc:290] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:48.217844  7717 raft_consensus.cc:515] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.220248  7717 leader_election.cc:290] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 election: Requested vote from peers 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:48.218158  7570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:48.222779  7688 tablet_bootstrap.cc:654] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:48.223623  7569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:48.224349  7569 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:48.226178  7610 leader_election.cc:343] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:48.227512  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:48.228156  7645 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.230710  7495 raft_consensus.cc:2468] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 1.
I20260709 13:08:48.233028  7610 leader_election.cc:304] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 55cabad661424707b66c6ddd198939b1
I20260709 13:08:48.234545  7689 raft_consensus.cc:2804] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:48.235069  7689 raft_consensus.cc:697] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:48.235898  7689 consensus_queue.cc:237] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.238281  7686 tablet_bootstrap.cc:654] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:48.241953  7391 catalog_manager.cc:5697] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:48.251513  7645 raft_consensus.cc:2468] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 1.
I20260709 13:08:48.252203  7569 raft_consensus.cc:2468] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7f82d1c7a294db588c3fc4e48c6012d in term 1.
I20260709 13:08:48.252216  7717 consensus_queue.cc:1048] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:48.253460  7686 tablet_bootstrap.cc:492] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d: No bootstrap required, opened a new log
I20260709 13:08:48.253973  7686 ts_tablet_manager.cc:1403] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.000s
I20260709 13:08:48.255007  7461 leader_election.cc:304] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:48.255858  7717 raft_consensus.cc:2804] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:48.256822  7717 raft_consensus.cc:697] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 LEADER]: Becoming Leader. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Running, Role: LEADER
I20260709 13:08:48.256702  7686 raft_consensus.cc:359] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.258571  7686 raft_consensus.cc:385] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:48.258855  7686 raft_consensus.cc:740] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:48.258256  7717 consensus_queue.cc:237] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.259601  7686 consensus_queue.cc:260] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.261924  7686 ts_tablet_manager.cc:1434] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:08:48.263800  7688 tablet_bootstrap.cc:492] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1: No bootstrap required, opened a new log
I20260709 13:08:48.264379  7688 ts_tablet_manager.cc:1403] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.011s
I20260709 13:08:48.272189  7688 raft_consensus.cc:359] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:48.273702  7688 raft_consensus.cc:385] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:48.274085  7688 raft_consensus.cc:740] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Initialized, Role: FOLLOWER
I20260709 13:08:48.275032  7688 consensus_queue.cc:260] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:48.277207  7688 ts_tablet_manager.cc:1434] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:08:48.278574  7688 tablet_bootstrap.cc:492] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1: Bootstrap starting.
I20260709 13:08:48.278398  7391 catalog_manager.cc:5697] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d reported cstate change: term changed from 0 to 1, leader changed from <none> to f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129). New cstate: current_term: 1 leader_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:48.286198  7688 tablet_bootstrap.cc:654] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:48.297258  7692 consensus_queue.cc:1048] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:48.296521  7393 catalog_manager.cc:5697] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:48.307454  7688 tablet_bootstrap.cc:492] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1: No bootstrap required, opened a new log
I20260709 13:08:48.308300  7688 ts_tablet_manager.cc:1403] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1: Time spent bootstrapping tablet: real 0.030s	user 0.024s	sys 0.003s
I20260709 13:08:48.311798  7688 raft_consensus.cc:359] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.312577  7688 raft_consensus.cc:385] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:48.312834  7688 raft_consensus.cc:740] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Initialized, Role: FOLLOWER
I20260709 13:08:48.313557  7688 consensus_queue.cc:260] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.315433  7688 ts_tablet_manager.cc:1434] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:08:48.412323 27934 ts_manager.cc:295] Set tserver state for f7f82d1c7a294db588c3fc4e48c6012d to MAINTENANCE_MODE
I20260709 13:08:48.413148 27934 tablet_server.cc:179] TabletServer@127.27.71.129:0 shutting down...
I20260709 13:08:48.438573 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:48.439536 27934 tablet_replica.cc:333] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d: stopping tablet replica
I20260709 13:08:48.440284 27934 raft_consensus.cc:2243] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:48.440680 27934 raft_consensus.cc:2272] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:48.442848  7692 raft_consensus.cc:493] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.443156 27934 tablet_replica.cc:333] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: stopping tablet replica
I20260709 13:08:48.443346  7692 raft_consensus.cc:515] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.444119 27934 raft_consensus.cc:2243] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:48.444929 27934 pending_rounds.cc:70] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Trying to abort 1 pending ops.
I20260709 13:08:48.445185 27934 pending_rounds.cc:77] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635955758194688 op_type: NO_OP noop_request { }
I20260709 13:08:48.445561 27934 raft_consensus.cc:2889] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:48.445904 27934 raft_consensus.cc:2272] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:48.446393  7570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:48.447335  7570 raft_consensus.cc:2393] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f7f82d1c7a294db588c3fc4e48c6012d in current term 1: Already voted for candidate 69c9355bd3b449d088b37517aed3a2a4 in this term.
I20260709 13:08:48.447320  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
I20260709 13:08:48.447788  7692 leader_election.cc:290] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:48.449119  7459 leader_election.cc:304] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7f82d1c7a294db588c3fc4e48c6012d; no voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4
I20260709 13:08:48.449820  7692 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.450392  7698 consensus_queue.cc:1048] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:48.450815 27934 tablet_replica.cc:333] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d: stopping tablet replica
I20260709 13:08:48.451624 27934 raft_consensus.cc:2243] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:48.452121 27934 raft_consensus.cc:2272] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:48.454823 27934 tablet_replica.cc:333] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d: stopping tablet replica
I20260709 13:08:48.455713 27934 raft_consensus.cc:2243] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:48.456193 27934 raft_consensus.cc:2272] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:48.457820  7692 raft_consensus.cc:2749] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:48.458514 27934 tablet_replica.cc:333] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: stopping tablet replica
I20260709 13:08:48.459370 27934 raft_consensus.cc:2243] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:48.460212 27934 pending_rounds.cc:70] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Trying to abort 1 pending ops.
I20260709 13:08:48.460467 27934 pending_rounds.cc:77] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305635954990620672 op_type: NO_OP noop_request { }
I20260709 13:08:48.460822 27934 raft_consensus.cc:2889] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:08:48.461200 27934 raft_consensus.cc:2272] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:48.464052 27934 tablet_replica.cc:333] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d: stopping tablet replica
I20260709 13:08:48.464833 27934 raft_consensus.cc:2243] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:48.465302 27934 raft_consensus.cc:2272] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:48.468094 27934 tablet_replica.cc:333] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d: stopping tablet replica
I20260709 13:08:48.468834 27934 raft_consensus.cc:2243] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:48.469913 27934 raft_consensus.cc:2272] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:48.471643  7536 consensus_peers.cc:597] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:48.472991 27934 tablet_replica.cc:333] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d: stopping tablet replica
I20260709 13:08:48.473850 27934 raft_consensus.cc:2243] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:48.474321 27934 raft_consensus.cc:2272] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:48.476816 27934 tablet_replica.cc:333] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d: stopping tablet replica
I20260709 13:08:48.477280 27934 raft_consensus.cc:2243] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:48.477599 27934 raft_consensus.cc:2272] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:48.479640 27934 tablet_replica.cc:333] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d: stopping tablet replica
I20260709 13:08:48.480186 27934 raft_consensus.cc:2243] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:48.480643 27934 raft_consensus.cc:2272] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:48.506587  7610 consensus_peers.cc:597] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:48.511679 27934 tablet_server.cc:196] TabletServer@127.27.71.129:0 shutdown complete.
W20260709 13:08:48.529738  7610 consensus_peers.cc:597] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:48.551605  7698 raft_consensus.cc:493] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.552320  7698 raft_consensus.cc:515] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.553942  7570 raft_consensus.cc:3060] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.555562  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee14275e88d648c6982fe9fc771ed0b7" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
I20260709 13:08:48.557484  7698 leader_election.cc:290] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
W20260709 13:08:48.562040  7536 leader_election.cc:336] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111)
I20260709 13:08:48.562522  7536 leader_election.cc:304] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55cabad661424707b66c6ddd198939b1; no voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:48.563274  7698 raft_consensus.cc:2749] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:48.563688  7706 consensus_queue.cc:1048] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:48.568578  7706 consensus_queue.cc:1048] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:08:48.648108  7610 consensus_peers.cc:597] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:48.650169  7610 consensus_peers.cc:597] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:48.656953  7570 raft_consensus.cc:3060] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:48.657434  7610 consensus_peers.cc:597] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:48.684824  7698 raft_consensus.cc:493] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.685431  7698 raft_consensus.cc:515] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:48.687336  7698 leader_election.cc:290] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:48.688472  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
W20260709 13:08:48.690898  7536 leader_election.cc:336] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111)
I20260709 13:08:48.691301  7536 leader_election.cc:304] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55cabad661424707b66c6ddd198939b1; no voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:48.692015  7698 raft_consensus.cc:2749] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:08:48.696501  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:48.745324  7570 raft_consensus.cc:3060] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:48.749115  7725 consensus_queue.cc:1048] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:48.753217  7725 consensus_queue.cc:1048] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:08:48.766635  7610 consensus_peers.cc:597] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:48.795935  7706 consensus_queue.cc:1048] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:08:49.534253  7610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111) [suppressed 151 similar messages]
W20260709 13:08:49.697686  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:49.762478  7706 raft_consensus.cc:493] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.763159  7706 raft_consensus.cc:515] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:49.765391  7706 leader_election.cc:290] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:49.767243  7570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:49.768078  7570 raft_consensus.cc:2468] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 1.
I20260709 13:08:49.769299  7610 leader_election.cc:304] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: 
W20260709 13:08:49.770066  7610 leader_election.cc:336] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111)
I20260709 13:08:49.770095  7706 raft_consensus.cc:2804] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:49.770642  7706 raft_consensus.cc:493] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.770924  7706 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:49.777194  7706 raft_consensus.cc:515] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:49.779072  7706 leader_election.cc:290] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:49.780746  7570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:49.781418  7570 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:49.783357  7610 leader_election.cc:336] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111)
I20260709 13:08:49.787539  7570 raft_consensus.cc:2468] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 2.
I20260709 13:08:49.788664  7610 leader_election.cc:304] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:49.789387  7706 raft_consensus.cc:2804] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:49.790189  7706 raft_consensus.cc:697] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:49.790902  7706 consensus_queue.cc:237] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:49.797827  7706 raft_consensus.cc:493] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f7f82d1c7a294db588c3fc4e48c6012d)
I20260709 13:08:49.798238  7706 raft_consensus.cc:515] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:49.799176  7394 catalog_manager.cc:5697] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 1 to 2, leader changed from f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129) to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:49.800117  7706 leader_election.cc:290] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:49.801102  7698 raft_consensus.cc:493] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f7f82d1c7a294db588c3fc4e48c6012d)
I20260709 13:08:49.801676  7570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aca33050e64b7595613a13182a0738" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:49.801641  7698 raft_consensus.cc:515] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:49.803170  7570 raft_consensus.cc:2468] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 1.
W20260709 13:08:49.805007  7610 leader_election.cc:336] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111)
I20260709 13:08:49.805949  7610 leader_election.cc:304] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:49.806105  7698 leader_election.cc:290] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:49.806819  7706 raft_consensus.cc:2804] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:49.807202  7706 raft_consensus.cc:493] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f7f82d1c7a294db588c3fc4e48c6012d)
I20260709 13:08:49.807626  7706 raft_consensus.cc:3060] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:49.808175  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aca33050e64b7595613a13182a0738" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
W20260709 13:08:49.810765  7536 leader_election.cc:336] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111)
I20260709 13:08:49.818712  7706 raft_consensus.cc:515] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:49.819783  7644 raft_consensus.cc:2393] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 55cabad661424707b66c6ddd198939b1 in current term 2: Already voted for candidate 69c9355bd3b449d088b37517aed3a2a4 in this term.
I20260709 13:08:49.820868  7706 leader_election.cc:290] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:49.821774  7534 leader_election.cc:304] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55cabad661424707b66c6ddd198939b1; no voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:49.822394  7570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aca33050e64b7595613a13182a0738" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:49.822619  7698 raft_consensus.cc:3060] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:49.826226  7610 leader_election.cc:336] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111)
I20260709 13:08:49.830857  7698 raft_consensus.cc:2749] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:49.836828  7570 raft_consensus.cc:2468] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 2.
I20260709 13:08:49.838487  7610 leader_election.cc:304] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:49.839293  7706 raft_consensus.cc:2804] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:49.840790  7706 raft_consensus.cc:697] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:49.841738  7706 consensus_queue.cc:237] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:49.849936  7391 catalog_manager.cc:5697] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 1 to 2, leader changed from f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129) to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:49.925609  7698 raft_consensus.cc:493] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.926249  7698 raft_consensus.cc:515] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:49.928340  7698 leader_election.cc:290] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:49.929647  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a7ef75b1e34928b01a24437f9ab01f" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
I20260709 13:08:49.930395  7644 raft_consensus.cc:2468] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 1.
I20260709 13:08:49.931517  7534 leader_election.cc:304] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: 
I20260709 13:08:49.932421  7698 raft_consensus.cc:2804] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:08:49.932423  7536 leader_election.cc:336] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111)
I20260709 13:08:49.932972  7698 raft_consensus.cc:493] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.933344  7698 raft_consensus.cc:3060] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:49.940467  7698 raft_consensus.cc:515] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:49.942468  7698 leader_election.cc:290] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:49.943909  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a7ef75b1e34928b01a24437f9ab01f" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:49.944682  7644 raft_consensus.cc:3060] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:49.946256  7536 leader_election.cc:336] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111)
I20260709 13:08:49.951109  7644 raft_consensus.cc:2468] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 2.
I20260709 13:08:49.952219  7534 leader_election.cc:304] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4; no voters: f7f82d1c7a294db588c3fc4e48c6012d
I20260709 13:08:49.952914  7698 raft_consensus.cc:2804] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:49.953372  7698 raft_consensus.cc:697] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Becoming Leader. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:49.954144  7698 consensus_queue.cc:237] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:49.961833  7391 catalog_manager.cc:5697] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 reported cstate change: term changed from 1 to 2, leader changed from f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129) to 55cabad661424707b66c6ddd198939b1 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "55cabad661424707b66c6ddd198939b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:50.246699  7570 raft_consensus.cc:1275] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Refusing update from remote peer 69c9355bd3b449d088b37517aed3a2a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:50.248446  7706 consensus_queue.cc:1048] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:50.266731  7610 consensus_peers.cc:597] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.366979  7570 raft_consensus.cc:1275] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Refusing update from remote peer 69c9355bd3b449d088b37517aed3a2a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:50.368585  7706 consensus_queue.cc:1048] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:08:50.371009  7610 consensus_peers.cc:597] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.407307  7644 raft_consensus.cc:1275] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:50.409044  7698 consensus_queue.cc:1048] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:08:50.421998  7536 consensus_peers.cc:597] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:50.702837  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:08:50.851702  7610 consensus_peers.cc:597] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:50.963824  7536 consensus_peers.cc:597] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:51.024663  7610 consensus_peers.cc:597] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:51.115584  7610 consensus_peers.cc:597] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:51.137297  7610 consensus_peers.cc:597] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:51.236101  7610 consensus_peers.cc:597] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:51.276880  7610 consensus_peers.cc:597] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:51.704135  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:08:52.641088  7610 consensus_peers.cc:597] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:52.705432  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:08:52.826390  7610 consensus_peers.cc:597] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:52.883214  7536 consensus_peers.cc:597] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:08:53.494132  7610 consensus_peers.cc:597] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:53.498981  7610 consensus_peers.cc:597] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:53.533847 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:53.543031  7610 consensus_peers.cc:597] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:53.543226  7783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:53.545945  7610 consensus_peers.cc:597] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:08:53.548375  7784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:53.548560 27934 server_base.cc:1061] running on GCE node
W20260709 13:08:53.551389  7787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:53.553701 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:53.553925 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:53.554077 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602533554065 us; error 0 us; skew 500 ppm
I20260709 13:08:53.554637 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:53.558044 27934 webserver.cc:533] Webserver started at http://127.27.71.129:43115/ using document root <none> and password file <none>
I20260709 13:08:53.558921 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:53.559195 27934 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:08:53.565486 27934 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:08:53.569926  7791 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:53.570789 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:08:53.571118 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-0-root
uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
format_stamp: "Formatted at 2026-07-09 13:08:46 on dist-test-slave-mlt7"
I20260709 13:08:53.571545 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602179482400-27934-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:53.589215 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:53.590572 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:53.592502 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:53.596629  7798 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:08:53.644656  7610 consensus_peers.cc:597] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:53.653239 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:08:53.653601 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.058s	user 0.003s	sys 0.000s
I20260709 13:08:53.653872 27934 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:08:53.660192  7798 tablet_bootstrap.cc:492] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:53.676432  7798 tablet_bootstrap.cc:492] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:53.677337  7798 tablet_bootstrap.cc:492] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap complete.
I20260709 13:08:53.678099  7798 ts_tablet_manager.cc:1403] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 13:08:53.681308  7798 raft_consensus.cc:359] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.682394  7798 raft_consensus.cc:740] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.683243  7798 consensus_queue.cc:260] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.686498  7798 ts_tablet_manager.cc:1434] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:08:53.687364  7798 tablet_bootstrap.cc:492] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:53.702742  7798 tablet_bootstrap.cc:492] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:53.703682  7798 tablet_bootstrap.cc:492] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap complete.
I20260709 13:08:53.704476  7798 ts_tablet_manager.cc:1403] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.007s
W20260709 13:08:53.706763  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:53.707377  7798 raft_consensus.cc:359] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.708189  7798 raft_consensus.cc:740] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.709025  7798 consensus_queue.cc:260] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.711387  7798 ts_tablet_manager.cc:1434] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 13:08:53.712329  7798 tablet_bootstrap.cc:492] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:53.721149 27934 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:08:53.721555 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.068s	user 0.067s	sys 0.000s
I20260709 13:08:53.727555  7798 tablet_bootstrap.cc:492] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:53.728613  7798 tablet_bootstrap.cc:492] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap complete.
I20260709 13:08:53.729403  7798 ts_tablet_manager.cc:1403] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:08:53.732543  7798 raft_consensus.cc:359] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.733582  7798 raft_consensus.cc:740] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.734774  7798 consensus_queue.cc:260] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.737692  7798 ts_tablet_manager.cc:1434] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:08:53.738865  7798 tablet_bootstrap.cc:492] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:53.756865  7798 tablet_bootstrap.cc:492] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:53.757612  7798 tablet_bootstrap.cc:492] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap complete.
I20260709 13:08:53.758423  7798 ts_tablet_manager.cc:1403] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:08:53.760623  7798 raft_consensus.cc:359] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.761176  7798 raft_consensus.cc:385] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.761430  7798 raft_consensus.cc:740] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.762320  7798 consensus_queue.cc:260] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.764570  7798 ts_tablet_manager.cc:1434] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:08:53.765463  7798 tablet_bootstrap.cc:492] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
W20260709 13:08:53.768428  7536 consensus_peers.cc:597] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:08:53.782717  7798 tablet_bootstrap.cc:492] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:53.783659  7798 tablet_bootstrap.cc:492] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap complete.
I20260709 13:08:53.784281  7798 ts_tablet_manager.cc:1403] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 13:08:53.788645 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.129:42053
I20260709 13:08:53.788313  7798 raft_consensus.cc:359] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:53.788766  7860 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.129:42053 every 8 connection(s)
I20260709 13:08:53.789184  7798 raft_consensus.cc:740] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.790359  7798 consensus_queue.cc:260] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:53.792421  7798 ts_tablet_manager.cc:1434] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:08:53.793891  7798 tablet_bootstrap.cc:492] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:53.801088 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:53.813189  7798 tablet_bootstrap.cc:492] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:53.814374  7798 tablet_bootstrap.cc:492] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap complete.
I20260709 13:08:53.815279  7798 ts_tablet_manager.cc:1403] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 13:08:53.818415  7798 raft_consensus.cc:359] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.819231  7798 raft_consensus.cc:740] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.820210  7861 heartbeater.cc:344] Connected to a master server at 127.27.71.190:43555
I20260709 13:08:53.820125  7798 consensus_queue.cc:260] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.820751  7861 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:53.821909  7861 heartbeater.cc:507] Master 127.27.71.190:43555 requested a full tablet report, sending...
I20260709 13:08:53.822082  7798 ts_tablet_manager.cc:1434] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:08:53.823050  7798 tablet_bootstrap.cc:492] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:53.828137  7394 ts_manager.cc:194] Re-registered known tserver with Master: f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053)
I20260709 13:08:53.832367  7644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
mode: GRACEFUL
new_leader_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:59130
I20260709 13:08:53.833247  7644 raft_consensus.cc:606] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Received request to transfer leadership to TS 55cabad661424707b66c6ddd198939b1
I20260709 13:08:53.836465  7644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16926f7e3623437c9c6d389bb22ecb59"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
mode: GRACEFUL
new_leader_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:59130
I20260709 13:08:53.837198  7798 tablet_bootstrap.cc:492] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:08:53.837090  7644 raft_consensus.cc:606] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Received request to transfer leadership to TS 55cabad661424707b66c6ddd198939b1
I20260709 13:08:53.838313  7798 tablet_bootstrap.cc:492] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap complete.
I20260709 13:08:53.838933  7798 ts_tablet_manager.cc:1403] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:08:53.840550  7394 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35726
I20260709 13:08:53.840446  7644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7bc65eac152241638209dea1a9158863"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
mode: GRACEFUL
new_leader_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:59130
I20260709 13:08:53.841293  7644 raft_consensus.cc:606] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Received request to transfer leadership to TS 55cabad661424707b66c6ddd198939b1
I20260709 13:08:53.841782  7798 raft_consensus.cc:359] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.842792  7798 raft_consensus.cc:740] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.843632  7798 consensus_queue.cc:260] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.845132  7861 heartbeater.cc:499] Master 127.27.71.190:43555 was elected leader, sending a full tablet report...
I20260709 13:08:53.845985  7798 ts_tablet_manager.cc:1434] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:08:53.846882  7798 tablet_bootstrap.cc:492] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:53.848273  7644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee14275e88d648c6982fe9fc771ed0b7"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
mode: GRACEFUL
new_leader_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:59130
I20260709 13:08:53.848915  7644 raft_consensus.cc:606] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Received request to transfer leadership to TS 55cabad661424707b66c6ddd198939b1
I20260709 13:08:53.850751 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:08:53.851558 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:53.852500 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:53.868729  7798 tablet_bootstrap.cc:492] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:53.869848  7798 tablet_bootstrap.cc:492] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap complete.
I20260709 13:08:53.870747  7798 ts_tablet_manager.cc:1403] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.024s	user 0.021s	sys 0.000s
I20260709 13:08:53.873675  7798 raft_consensus.cc:359] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.874485  7798 raft_consensus.cc:740] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.875294  7798 consensus_queue.cc:260] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.877086  7798 ts_tablet_manager.cc:1434] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:08:53.878422  7798 tablet_bootstrap.cc:492] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:53.882632  7835 raft_consensus.cc:3060] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:53.896979  7798 tablet_bootstrap.cc:492] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:08:53.898195  7798 tablet_bootstrap.cc:492] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap complete.
I20260709 13:08:53.898970  7798 ts_tablet_manager.cc:1403] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 13:08:53.901912  7835 pending_rounds.cc:85] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Aborting all ops after (but not including) 0
I20260709 13:08:53.902549  7835 pending_rounds.cc:107] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:08:53.901999  7798 raft_consensus.cc:359] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.902940  7835 raft_consensus.cc:2889] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:53.903285  7798 raft_consensus.cc:740] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.904255  7798 consensus_queue.cc:260] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.906818  7798 ts_tablet_manager.cc:1434] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:08:53.907712  7798 tablet_bootstrap.cc:492] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap starting.
I20260709 13:08:53.923686  7798 tablet_bootstrap.cc:492] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:53.924904  7798 tablet_bootstrap.cc:492] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d: Bootstrap complete.
I20260709 13:08:53.925753  7798 ts_tablet_manager.cc:1403] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.008s
I20260709 13:08:53.925956  7706 raft_consensus.cc:993] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4: : Instructing follower 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:53.926668  7877 raft_consensus.cc:1081] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Signalling peer 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:53.928584  7798 raft_consensus.cc:359] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.929234  7798 raft_consensus.cc:740] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f82d1c7a294db588c3fc4e48c6012d, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.929112  7570 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee14275e88d648c6982fe9fc771ed0b7"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:39244
I20260709 13:08:53.930141  7570 raft_consensus.cc:493] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:53.930352  7798 consensus_queue.cc:260] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.930691  7570 raft_consensus.cc:3060] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:53.932299  7798 ts_tablet_manager.cc:1434] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:08:53.940182  7877 raft_consensus.cc:993] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4: : Instructing follower 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:53.942171  7877 raft_consensus.cc:1081] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Signalling peer 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:53.942723  7570 raft_consensus.cc:515] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.944952  7569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7bc65eac152241638209dea1a9158863"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:39244
I20260709 13:08:53.945673  7569 raft_consensus.cc:493] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:53.946064  7569 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:53.946475  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee14275e88d648c6982fe9fc771ed0b7" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:53.946678  7834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee14275e88d648c6982fe9fc771ed0b7" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:53.947273  7644 raft_consensus.cc:3055] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:53.947382  7834 raft_consensus.cc:3060] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:53.947731  7644 raft_consensus.cc:740] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:53.948885  7644 consensus_queue.cc:260] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.950383  7644 raft_consensus.cc:3060] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:53.957876  7569 raft_consensus.cc:515] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.958848  7570 leader_election.cc:290] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:53.958343  7834 raft_consensus.cc:2468] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 2.
I20260709 13:08:53.961407  7835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:53.962085  7835 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:53.961905  7536 leader_election.cc:304] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:53.961947  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:53.962986  7645 raft_consensus.cc:3055] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:53.963330  7645 raft_consensus.cc:740] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:53.963439  7770 raft_consensus.cc:2804] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:53.964766  7645 consensus_queue.cc:260] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.966024  7645 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:53.971735  7835 raft_consensus.cc:2468] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 2.
I20260709 13:08:53.973246  7536 leader_election.cc:304] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:53.973248  7644 raft_consensus.cc:2468] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 2.
I20260709 13:08:53.976024  7569 leader_election.cc:290] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:53.978263  7880 raft_consensus.cc:2804] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:53.978597  7770 raft_consensus.cc:697] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Becoming Leader. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:53.980329  7880 raft_consensus.cc:697] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Becoming Leader. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:53.981014  7770 consensus_queue.cc:237] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.981745  7880 consensus_queue.cc:237] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:53.984990  7645 raft_consensus.cc:2468] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 2.
I20260709 13:08:53.986533  7835 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:53.995751  7835 pending_rounds.cc:85] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Aborting all ops after (but not including) 0
I20260709 13:08:53.996133  7835 pending_rounds.cc:107] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:08:53.996402  7835 raft_consensus.cc:2889] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:53.994856  7394 catalog_manager.cc:5697] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 reported cstate change: term changed from 1 to 2, leader changed from 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131) to 55cabad661424707b66c6ddd198939b1 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "55cabad661424707b66c6ddd198939b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.021589  7835 raft_consensus.cc:3060] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.021860  7391 catalog_manager.cc:5697] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 reported cstate change: term changed from 1 to 2, leader changed from 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131) to 55cabad661424707b66c6ddd198939b1 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "55cabad661424707b66c6ddd198939b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.071696  7800 raft_consensus.cc:493] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.072270  7800 raft_consensus.cc:515] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:54.074952  7800 leader_election.cc:290] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:54.086264  7569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55cabad661424707b66c6ddd198939b1" is_pre_election: true
I20260709 13:08:54.086838  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70" candidate_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c9355bd3b449d088b37517aed3a2a4" is_pre_election: true
I20260709 13:08:54.088543  7792 leader_election.cc:304] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7f82d1c7a294db588c3fc4e48c6012d; no voters: 55cabad661424707b66c6ddd198939b1, 69c9355bd3b449d088b37517aed3a2a4
I20260709 13:08:54.089215  7800 raft_consensus.cc:2749] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:54.187489  7878 raft_consensus.cc:993] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4: : Instructing follower 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:54.188078  7877 raft_consensus.cc:1081] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Signalling peer 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:54.188271  7835 raft_consensus.cc:3060] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.189602  7569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16926f7e3623437c9c6d389bb22ecb59"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:39244
I20260709 13:08:54.190240  7569 raft_consensus.cc:493] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:54.190624  7569 raft_consensus.cc:3060] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.199348  7569 raft_consensus.cc:515] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:54.202235  7569 leader_election.cc:290] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:54.202356  7835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16926f7e3623437c9c6d389bb22ecb59" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:54.203074  7835 raft_consensus.cc:3060] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.203382  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16926f7e3623437c9c6d389bb22ecb59" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:54.204030  7645 raft_consensus.cc:3055] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:54.204375  7645 raft_consensus.cc:740] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:54.205248  7645 consensus_queue.cc:260] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:54.206470  7645 raft_consensus.cc:3060] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:54.210307  7835 raft_consensus.cc:2468] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 2.
I20260709 13:08:54.211477  7536 leader_election.cc:304] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:54.212427  7880 raft_consensus.cc:2804] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:54.212932  7880 raft_consensus.cc:697] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Becoming Leader. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:54.213701  7645 raft_consensus.cc:2468] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 2.
I20260709 13:08:54.213982  7880 consensus_queue.cc:237] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:54.221843  7391 catalog_manager.cc:5697] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 reported cstate change: term changed from 1 to 2, leader changed from 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131) to 55cabad661424707b66c6ddd198939b1 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "55cabad661424707b66c6ddd198939b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.235447  7878 raft_consensus.cc:993] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4: : Instructing follower 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:54.235984  7877 raft_consensus.cc:1081] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Signalling peer 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:54.237725  7569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:39244
I20260709 13:08:54.238224  7569 raft_consensus.cc:493] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:54.238504  7569 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:54.244061  7569 raft_consensus.cc:515] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:54.245744  7569 leader_election.cc:290] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 3 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:54.246876  7835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:54.246948  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:54.247536  7835 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:54.247608  7645 raft_consensus.cc:3055] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:54.248175  7645 raft_consensus.cc:740] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:54.248958  7645 consensus_queue.cc:260] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:54.250064  7645 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:54.253429  7835 raft_consensus.cc:2468] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 3.
I20260709 13:08:54.254634  7536 leader_election.cc:304] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:54.255584  7881 raft_consensus.cc:2804] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:54.256184  7881 raft_consensus.cc:697] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 3 LEADER]: Becoming Leader. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:54.256264  7645 raft_consensus.cc:2468] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 3.
I20260709 13:08:54.257368  7881 consensus_queue.cc:237] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:54.266536  7391 catalog_manager.cc:5697] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 reported cstate change: term changed from 2 to 3, leader changed from 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131) to 55cabad661424707b66c6ddd198939b1 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "55cabad661424707b66c6ddd198939b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.374089  7644 raft_consensus.cc:1275] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:54.375638  7770 consensus_queue.cc:1048] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:54.379992  7835 raft_consensus.cc:1275] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:54.381709  7770 consensus_queue.cc:1048] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:54.450464  7835 raft_consensus.cc:1275] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:54.451867  7770 consensus_queue.cc:1048] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:54.462415  7644 raft_consensus.cc:1275] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:54.465579  7770 consensus_queue.cc:1048] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:54.651314  7835 raft_consensus.cc:1275] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:54.652746  7770 consensus_queue.cc:1048] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:54.660771  7644 raft_consensus.cc:1275] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:54.662428  7881 consensus_queue.cc:1048] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:54.668883  7835 raft_consensus.cc:1275] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:54.670390  7881 consensus_queue.cc:1048] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:54.680684  7644 raft_consensus.cc:1275] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:54.682358  7881 consensus_queue.cc:1048] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:08:54.707996  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:54.866684 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:54.884083  7569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
mode: GRACEFUL
new_leader_uuid: "69c9355bd3b449d088b37517aed3a2a4"
 from {username='slave'} at 127.0.0.1:45382
I20260709 13:08:54.884644  7569 raft_consensus.cc:606] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 3 LEADER]: Received request to transfer leadership to TS 69c9355bd3b449d088b37517aed3a2a4
I20260709 13:08:54.887153  7569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7bc65eac152241638209dea1a9158863"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
mode: GRACEFUL
new_leader_uuid: "69c9355bd3b449d088b37517aed3a2a4"
 from {username='slave'} at 127.0.0.1:45382
I20260709 13:08:54.887637  7569 raft_consensus.cc:606] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Received request to transfer leadership to TS 69c9355bd3b449d088b37517aed3a2a4
I20260709 13:08:54.888762 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:54.889264 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:54.889855 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:55.084889  7770 raft_consensus.cc:993] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1: : Instructing follower 69c9355bd3b449d088b37517aed3a2a4 to start an election
I20260709 13:08:55.085474  7881 raft_consensus.cc:1081] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Signalling peer 69c9355bd3b449d088b37517aed3a2a4 to start an election
I20260709 13:08:55.087208  7644 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7bc65eac152241638209dea1a9158863"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
 from {username='slave'} at 127.0.0.1:38200
I20260709 13:08:55.087759  7644 raft_consensus.cc:493] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:55.088186  7644 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:55.094631  7644 raft_consensus.cc:515] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:55.096477  7644 leader_election.cc:290] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 3 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:55.097283  7835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:55.097501  7569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:55.097981  7835 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:55.098134  7569 raft_consensus.cc:3055] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:55.098585  7569 raft_consensus.cc:740] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:55.099334  7569 consensus_queue.cc:260] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:55.100482  7569 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:55.105268  7902 raft_consensus.cc:993] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1: : Instructing follower 69c9355bd3b449d088b37517aed3a2a4 to start an election
I20260709 13:08:55.105907  7902 raft_consensus.cc:1081] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 3 LEADER]: Signalling peer 69c9355bd3b449d088b37517aed3a2a4 to start an election
I20260709 13:08:55.107540  7835 raft_consensus.cc:2468] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 3.
I20260709 13:08:55.108219  7644 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
 from {username='slave'} at 127.0.0.1:38200
I20260709 13:08:55.108906  7644 raft_consensus.cc:493] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:55.108625  7569 raft_consensus.cc:2468] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 3.
I20260709 13:08:55.109134  7610 leader_election.cc:304] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:55.109383  7644 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:55.110042  7877 raft_consensus.cc:2804] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:55.111271  7877 raft_consensus.cc:697] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:55.112224  7877 consensus_queue.cc:237] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:55.115365  7644 raft_consensus.cc:515] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:55.117290  7644 leader_election.cc:290] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 4 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:55.118000  7835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:55.118410  7569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15cc71669e814555aa12d1e2eb776c95" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:55.118714  7835 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:55.119324  7569 raft_consensus.cc:3055] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:55.119719  7569 raft_consensus.cc:740] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:55.120441  7569 consensus_queue.cc:260] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:55.121755  7569 raft_consensus.cc:3060] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:55.120041  7392 catalog_manager.cc:5697] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 2 to 3, leader changed from 55cabad661424707b66c6ddd198939b1 (127.27.71.130) to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 3 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:55.126178  7835 raft_consensus.cc:2468] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 4.
I20260709 13:08:55.127802  7610 leader_election.cc:304] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:55.128705  7877 raft_consensus.cc:2804] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:55.129613  7569 raft_consensus.cc:2468] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 4.
I20260709 13:08:55.129199  7877 raft_consensus.cc:697] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 4 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:55.130795  7877 consensus_queue.cc:237] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:55.141372  7392 catalog_manager.cc:5697] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 3 to 4, leader changed from 55cabad661424707b66c6ddd198939b1 (127.27.71.130) to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 4 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:55.561230  7569 raft_consensus.cc:1275] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Refusing update from remote peer 69c9355bd3b449d088b37517aed3a2a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:55.562402  7835 raft_consensus.cc:1275] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 4 FOLLOWER]: Refusing update from remote peer 69c9355bd3b449d088b37517aed3a2a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:55.563380  7907 consensus_queue.cc:1048] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:55.564157  7877 consensus_queue.cc:1048] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:55.588559  7833 raft_consensus.cc:1275] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Refusing update from remote peer 69c9355bd3b449d088b37517aed3a2a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:55.595956  7569 raft_consensus.cc:1275] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 4 FOLLOWER]: Refusing update from remote peer 69c9355bd3b449d088b37517aed3a2a4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:55.598044  7915 consensus_queue.cc:1048] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:08:55.603495  7877 consensus_queue.cc:1048] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:08:55.709152  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:55.897502 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:55.905606  7644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
mode: GRACEFUL
new_leader_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:59130
I20260709 13:08:55.906190  7644 raft_consensus.cc:606] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Received request to transfer leadership to TS 55cabad661424707b66c6ddd198939b1
I20260709 13:08:55.909186  7644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7bc65eac152241638209dea1a9158863"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
mode: GRACEFUL
new_leader_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:59130
I20260709 13:08:55.909754  7644 raft_consensus.cc:606] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 LEADER]: Received request to transfer leadership to TS 55cabad661424707b66c6ddd198939b1
I20260709 13:08:55.911031 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:55.911571 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:55.912151 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:56.113727  7915 raft_consensus.cc:993] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4: : Instructing follower 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:56.114199  7877 raft_consensus.cc:1081] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 LEADER]: Signalling peer 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:56.115723  7569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7bc65eac152241638209dea1a9158863"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:39244
I20260709 13:08:56.116314  7569 raft_consensus.cc:493] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:56.116636  7569 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:56.122258  7569 raft_consensus.cc:515] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:56.124126  7569 leader_election.cc:290] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 4 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:56.125223  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:56.125753  7644 raft_consensus.cc:3055] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:56.125890  7835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bc65eac152241638209dea1a9158863" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:56.125978  7644 raft_consensus.cc:740] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:56.126562  7835 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:56.127156  7644 consensus_queue.cc:260] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:56.128310  7644 raft_consensus.cc:3060] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:56.131985  7835 raft_consensus.cc:2468] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 4.
I20260709 13:08:56.133193  7536 leader_election.cc:304] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:56.133917  7770 raft_consensus.cc:2804] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:56.134152  7644 raft_consensus.cc:2468] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 4.
I20260709 13:08:56.134888  7770 raft_consensus.cc:697] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 4 LEADER]: Becoming Leader. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:56.135844  7770 consensus_queue.cc:237] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:56.142999  7392 catalog_manager.cc:5697] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 reported cstate change: term changed from 3 to 4, leader changed from 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131) to 55cabad661424707b66c6ddd198939b1 (127.27.71.130). New cstate: current_term: 4 leader_uuid: "55cabad661424707b66c6ddd198939b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.264444  7915 raft_consensus.cc:993] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4: : Instructing follower 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:56.264958  7877 raft_consensus.cc:1081] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Signalling peer 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:56.266404  7569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:39244
I20260709 13:08:56.266897  7569 raft_consensus.cc:493] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:56.267184  7569 raft_consensus.cc:3060] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:56.273677  7569 raft_consensus.cc:515] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:56.275859  7569 leader_election.cc:290] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:56.276712  7835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:56.276961  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:56.277386  7835 raft_consensus.cc:3060] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:56.277670  7644 raft_consensus.cc:3055] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:56.278013  7644 raft_consensus.cc:740] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:56.278837  7644 consensus_queue.cc:260] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:56.280447  7644 raft_consensus.cc:3060] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:56.283831  7835 raft_consensus.cc:2468] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 2.
I20260709 13:08:56.285161  7536 leader_election.cc:304] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:56.286276  7770 raft_consensus.cc:2804] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:56.286799  7770 raft_consensus.cc:697] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Becoming Leader. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:56.287566  7644 raft_consensus.cc:2468] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 2.
I20260709 13:08:56.287853  7770 consensus_queue.cc:237] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:56.296281  7392 catalog_manager.cc:5697] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 reported cstate change: term changed from 1 to 2, leader changed from 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131) to 55cabad661424707b66c6ddd198939b1 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "55cabad661424707b66c6ddd198939b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.573913  7833 raft_consensus.cc:1275] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 4 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:56.575564  7770 consensus_queue.cc:1048] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:56.586694  7644 raft_consensus.cc:1275] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 4 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:56.588481  7927 consensus_queue.cc:1048] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:08:56.710460  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:56.738767  7833 raft_consensus.cc:1275] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:56.740680  7770 consensus_queue.cc:1048] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:56.752410  7644 raft_consensus.cc:1275] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:56.755633  7921 consensus_queue.cc:1048] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:56.918725 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:56.927192  7569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
mode: GRACEFUL
new_leader_uuid: "69c9355bd3b449d088b37517aed3a2a4"
 from {username='slave'} at 127.0.0.1:45382
I20260709 13:08:56.927772  7569 raft_consensus.cc:606] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Received request to transfer leadership to TS 69c9355bd3b449d088b37517aed3a2a4
I20260709 13:08:56.930735  7569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
mode: GRACEFUL
new_leader_uuid: "69c9355bd3b449d088b37517aed3a2a4"
 from {username='slave'} at 127.0.0.1:45382
I20260709 13:08:56.931291  7569 raft_consensus.cc:606] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 1 LEADER]: Received request to transfer leadership to TS 69c9355bd3b449d088b37517aed3a2a4
I20260709 13:08:56.932518 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:56.933106 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:56.933768 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:57.227131  7770 raft_consensus.cc:993] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1: : Instructing follower 69c9355bd3b449d088b37517aed3a2a4 to start an election
I20260709 13:08:57.227941  7921 raft_consensus.cc:1081] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Signalling peer 69c9355bd3b449d088b37517aed3a2a4 to start an election
I20260709 13:08:57.229777  7644 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
 from {username='slave'} at 127.0.0.1:38200
I20260709 13:08:57.230399  7644 raft_consensus.cc:493] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:57.230795  7644 raft_consensus.cc:3060] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:57.236653  7644 raft_consensus.cc:515] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:57.238346  7644 leader_election.cc:290] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 3 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:57.239303  7833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:57.239418  7569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f39053c868b4252acca47e18d123fb0" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:57.240394  7833 raft_consensus.cc:3060] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:57.240482  7569 raft_consensus.cc:3055] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:57.240957  7569 raft_consensus.cc:740] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:57.241766  7569 consensus_queue.cc:260] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:57.243089  7569 raft_consensus.cc:3060] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:57.246969  7833 raft_consensus.cc:2468] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 3.
I20260709 13:08:57.248291  7610 leader_election.cc:304] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:57.249029  7915 raft_consensus.cc:2804] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:57.250785  7915 raft_consensus.cc:697] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:57.251370  7569 raft_consensus.cc:2468] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 3.
I20260709 13:08:57.251854  7915 consensus_queue.cc:237] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:57.260143  7392 catalog_manager.cc:5697] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 2 to 3, leader changed from 55cabad661424707b66c6ddd198939b1 (127.27.71.130) to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 3 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:57.269670  7770 raft_consensus.cc:993] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1: : Instructing follower 69c9355bd3b449d088b37517aed3a2a4 to start an election
I20260709 13:08:57.270349  7770 raft_consensus.cc:1081] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 1 LEADER]: Signalling peer 69c9355bd3b449d088b37517aed3a2a4 to start an election
I20260709 13:08:57.272646  7644 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
 from {username='slave'} at 127.0.0.1:38200
I20260709 13:08:57.273211  7644 raft_consensus.cc:493] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:57.273699  7644 raft_consensus.cc:3060] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.284219  7644 raft_consensus.cc:515] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:57.286243  7644 leader_election.cc:290] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359)
I20260709 13:08:57.287451  7833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:57.289083  7569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad99cf019e854bea806a64aaabd61f70" candidate_uuid: "69c9355bd3b449d088b37517aed3a2a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55cabad661424707b66c6ddd198939b1"
I20260709 13:08:57.289476  7833 raft_consensus.cc:3060] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.289966  7569 raft_consensus.cc:3055] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:57.290431  7569 raft_consensus.cc:740] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:57.291613  7569 consensus_queue.cc:260] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:57.294414  7569 raft_consensus.cc:3060] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:57.300113  7833 raft_consensus.cc:2468] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 2.
I20260709 13:08:57.301848  7610 leader_election.cc:304] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69c9355bd3b449d088b37517aed3a2a4, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:57.302920  7915 raft_consensus.cc:2804] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:57.303081  7569 raft_consensus.cc:2468] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69c9355bd3b449d088b37517aed3a2a4 in term 2.
I20260709 13:08:57.303464  7915 raft_consensus.cc:697] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Becoming Leader. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:57.304930  7915 consensus_queue.cc:237] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:57.313768  7392 catalog_manager.cc:5697] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 reported cstate change: term changed from 1 to 2, leader changed from 55cabad661424707b66c6ddd198939b1 (127.27.71.130) to 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "69c9355bd3b449d088b37517aed3a2a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:57.647487  7569 raft_consensus.cc:1275] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Refusing update from remote peer 69c9355bd3b449d088b37517aed3a2a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:57.648860  7915 consensus_queue.cc:1048] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:57.658789  7833 raft_consensus.cc:1275] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Refusing update from remote peer 69c9355bd3b449d088b37517aed3a2a4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:57.660347  7915 consensus_queue.cc:1048] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:08:57.711798  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:57.717686  7569 raft_consensus.cc:1275] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Refusing update from remote peer 69c9355bd3b449d088b37517aed3a2a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:57.719120  7915 consensus_queue.cc:1048] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:57.729140  7835 raft_consensus.cc:1275] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Refusing update from remote peer 69c9355bd3b449d088b37517aed3a2a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:57.730631  7945 consensus_queue.cc:1048] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:57.940666 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:57.949841  7644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23bbc33e47fb45e38236cc96db1d262f"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
mode: GRACEFUL
new_leader_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:59130
I20260709 13:08:57.950523  7644 raft_consensus.cc:606] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Received request to transfer leadership to TS 55cabad661424707b66c6ddd198939b1
I20260709 13:08:57.953781  7644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7aca33050e64b7595613a13182a0738"
dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
mode: GRACEFUL
new_leader_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:59130
I20260709 13:08:57.954303  7644 raft_consensus.cc:606] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Received request to transfer leadership to TS 55cabad661424707b66c6ddd198939b1
I20260709 13:08:57.955327 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:57.956035 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:57.956786 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:58.229619  7940 raft_consensus.cc:993] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4: : Instructing follower 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:58.230122  7945 raft_consensus.cc:1081] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Signalling peer 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:58.231566  7569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7aca33050e64b7595613a13182a0738"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:39244
I20260709 13:08:58.232079  7569 raft_consensus.cc:493] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:58.232360  7569 raft_consensus.cc:3060] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:58.238617  7569 raft_consensus.cc:515] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:58.240491  7569 leader_election.cc:290] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 3 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:58.241371  7835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aca33050e64b7595613a13182a0738" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:58.242061  7835 raft_consensus.cc:3060] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:58.241926  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7aca33050e64b7595613a13182a0738" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:58.242631  7644 raft_consensus.cc:3055] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:58.242926  7644 raft_consensus.cc:740] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:58.243681  7644 consensus_queue.cc:260] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:58.244961  7644 raft_consensus.cc:3060] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:58.249714  7835 raft_consensus.cc:2468] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 3.
I20260709 13:08:58.251092  7536 leader_election.cc:304] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:58.252123  7770 raft_consensus.cc:2804] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:58.252146  7644 raft_consensus.cc:2468] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 3.
I20260709 13:08:58.253355  7770 raft_consensus.cc:697] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 3 LEADER]: Becoming Leader. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:58.254343  7770 consensus_queue.cc:237] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } }
I20260709 13:08:58.256171  7940 raft_consensus.cc:993] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4: : Instructing follower 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:58.256716  7945 raft_consensus.cc:1081] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Signalling peer 55cabad661424707b66c6ddd198939b1 to start an election
I20260709 13:08:58.258354  7569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23bbc33e47fb45e38236cc96db1d262f"
dest_uuid: "55cabad661424707b66c6ddd198939b1"
 from {username='slave'} at 127.0.0.1:39244
I20260709 13:08:58.259034  7569 raft_consensus.cc:493] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:58.259423  7569 raft_consensus.cc:3060] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:58.262714  7394 catalog_manager.cc:5697] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 reported cstate change: term changed from 2 to 3, leader changed from 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131) to 55cabad661424707b66c6ddd198939b1 (127.27.71.130). New cstate: current_term: 3 leader_uuid: "55cabad661424707b66c6ddd198939b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:58.265509  7569 raft_consensus.cc:515] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:58.267165  7569 leader_election.cc:290] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Requested vote from peers f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053), 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131:45237)
I20260709 13:08:58.268059  7835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bbc33e47fb45e38236cc96db1d262f" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7f82d1c7a294db588c3fc4e48c6012d"
I20260709 13:08:58.268230  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23bbc33e47fb45e38236cc96db1d262f" candidate_uuid: "55cabad661424707b66c6ddd198939b1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "69c9355bd3b449d088b37517aed3a2a4"
I20260709 13:08:58.268638  7835 raft_consensus.cc:3060] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:58.268864  7644 raft_consensus.cc:3055] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:58.269202  7644 raft_consensus.cc:740] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 69c9355bd3b449d088b37517aed3a2a4, State: Running, Role: LEADER
I20260709 13:08:58.270068  7644 consensus_queue.cc:260] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:58.271214  7644 raft_consensus.cc:3060] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:58.275848  7835 raft_consensus.cc:2468] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 2.
I20260709 13:08:58.277057  7536 leader_election.cc:304] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cabad661424707b66c6ddd198939b1, f7f82d1c7a294db588c3fc4e48c6012d; no voters: 
I20260709 13:08:58.277838  7770 raft_consensus.cc:2804] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:58.278288  7644 raft_consensus.cc:2468] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cabad661424707b66c6ddd198939b1 in term 2.
I20260709 13:08:58.278401  7770 raft_consensus.cc:697] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Becoming Leader. State: Replica: 55cabad661424707b66c6ddd198939b1, State: Running, Role: LEADER
I20260709 13:08:58.279399  7770 consensus_queue.cc:237] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } }
I20260709 13:08:58.286998  7394 catalog_manager.cc:5697] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 reported cstate change: term changed from 1 to 2, leader changed from 69c9355bd3b449d088b37517aed3a2a4 (127.27.71.131) to 55cabad661424707b66c6ddd198939b1 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "55cabad661424707b66c6ddd198939b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cabad661424707b66c6ddd198939b1" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 46359 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:58.710505  7835 raft_consensus.cc:1275] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:58.712080  7770 consensus_queue.cc:1048] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:08:58.712906  7442 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:08:58.723839  7644 raft_consensus.cc:1275] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:58.725617  7947 consensus_queue.cc:1048] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:58.764150  7645 raft_consensus.cc:1275] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.765479  7770 consensus_queue.cc:1048] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c9355bd3b449d088b37517aed3a2a4" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:58.776121  7835 raft_consensus.cc:1275] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Refusing update from remote peer 55cabad661424707b66c6ddd198939b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:58.777995  7770 consensus_queue.cc:1048] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f82d1c7a294db588c3fc4e48c6012d" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:58.964598 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:58.999540 27934 tablet_server.cc:179] TabletServer@127.27.71.129:42053 shutting down...
I20260709 13:08:59.024015 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:59.024894 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:59.025982 27934 raft_consensus.cc:2243] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.026791 27934 raft_consensus.cc:2272] T 3f39053c868b4252acca47e18d123fb0 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.030014 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:59.030722 27934 raft_consensus.cc:2243] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.031213 27934 raft_consensus.cc:2272] T 68a7ef75b1e34928b01a24437f9ab01f P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.033586 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:59.034204 27934 raft_consensus.cc:2243] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.034662 27934 raft_consensus.cc:2272] T a34c585c5de24649b746f9cd53cc8e57 P f7f82d1c7a294db588c3fc4e48c6012d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.037106 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:59.038139 27934 raft_consensus.cc:2243] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.038760 27934 raft_consensus.cc:2272] T 7bc65eac152241638209dea1a9158863 P f7f82d1c7a294db588c3fc4e48c6012d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.041780 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:59.042604 27934 raft_consensus.cc:2243] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.043185 27934 raft_consensus.cc:2272] T 23bbc33e47fb45e38236cc96db1d262f P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.046321 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:59.046977 27934 raft_consensus.cc:2243] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.047475 27934 raft_consensus.cc:2272] T 15cc71669e814555aa12d1e2eb776c95 P f7f82d1c7a294db588c3fc4e48c6012d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.049708 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:59.050314 27934 raft_consensus.cc:2243] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.050769 27934 raft_consensus.cc:2272] T ad99cf019e854bea806a64aaabd61f70 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.053107 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:59.053715 27934 raft_consensus.cc:2243] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.054179 27934 raft_consensus.cc:2272] T d7aca33050e64b7595613a13182a0738 P f7f82d1c7a294db588c3fc4e48c6012d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.057403 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:59.058013 27934 raft_consensus.cc:2243] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.058511 27934 raft_consensus.cc:2272] T 16926f7e3623437c9c6d389bb22ecb59 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.060731 27934 tablet_replica.cc:333] stopping tablet replica
I20260709 13:08:59.061259 27934 raft_consensus.cc:2243] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.061731 27934 raft_consensus.cc:2272] T ee14275e88d648c6982fe9fc771ed0b7 P f7f82d1c7a294db588c3fc4e48c6012d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.087092 27934 tablet_server.cc:196] TabletServer@127.27.71.129:42053 shutdown complete.
I20260709 13:08:59.114475 27934 tablet_server.cc:179] TabletServer@127.27.71.130:0 shutting down...
W20260709 13:08:59.138414  7536 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111) [suppressed 88 similar messages]
W20260709 13:08:59.144984  7536 consensus_peers.cc:597] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.148762 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:59.150360 27934 tablet_replica.cc:333] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1: stopping tablet replica
I20260709 13:08:59.151736 27934 raft_consensus.cc:2243] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.152684 27934 raft_consensus.cc:2272] T 3f39053c868b4252acca47e18d123fb0 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:59.155145  7610 consensus_peers.cc:597] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.155827 27934 tablet_replica.cc:333] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1: stopping tablet replica
I20260709 13:08:59.156754 27934 raft_consensus.cc:2243] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.157656 27934 raft_consensus.cc:2272] T ee14275e88d648c6982fe9fc771ed0b7 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.160676 27934 tablet_replica.cc:333] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1: stopping tablet replica
I20260709 13:08:59.161342 27934 raft_consensus.cc:2243] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.162437 27934 raft_consensus.cc:2272] T 23bbc33e47fb45e38236cc96db1d262f P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:59.164731  7610 consensus_peers.cc:597] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.166463 27934 tablet_replica.cc:333] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1: stopping tablet replica
I20260709 13:08:59.167562 27934 raft_consensus.cc:2243] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.168510 27934 raft_consensus.cc:2272] T 7bc65eac152241638209dea1a9158863 P 55cabad661424707b66c6ddd198939b1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.171264 27934 tablet_replica.cc:333] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1: stopping tablet replica
I20260709 13:08:59.171962 27934 raft_consensus.cc:2243] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.172443 27934 raft_consensus.cc:2272] T 15cc71669e814555aa12d1e2eb776c95 P 55cabad661424707b66c6ddd198939b1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.174917 27934 tablet_replica.cc:333] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1: stopping tablet replica
I20260709 13:08:59.175554 27934 raft_consensus.cc:2243] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.176059 27934 raft_consensus.cc:2272] T ad99cf019e854bea806a64aaabd61f70 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.178503 27934 tablet_replica.cc:333] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1: stopping tablet replica
I20260709 13:08:59.179189 27934 raft_consensus.cc:2243] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.180127 27934 raft_consensus.cc:2272] T 16926f7e3623437c9c6d389bb22ecb59 P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.183179 27934 tablet_replica.cc:333] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1: stopping tablet replica
I20260709 13:08:59.183820 27934 raft_consensus.cc:2243] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.185238 27934 raft_consensus.cc:2272] T d7aca33050e64b7595613a13182a0738 P 55cabad661424707b66c6ddd198939b1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.188985 27934 tablet_replica.cc:333] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1: stopping tablet replica
I20260709 13:08:59.189749 27934 raft_consensus.cc:2243] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.190712 27934 raft_consensus.cc:2272] T 68a7ef75b1e34928b01a24437f9ab01f P 55cabad661424707b66c6ddd198939b1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:59.191025  7610 consensus_peers.cc:597] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359): Couldn't send request to peer 55cabad661424707b66c6ddd198939b1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.193465 27934 tablet_replica.cc:333] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1: stopping tablet replica
I20260709 13:08:59.194133 27934 raft_consensus.cc:2243] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.194662 27934 raft_consensus.cc:2272] T a34c585c5de24649b746f9cd53cc8e57 P 55cabad661424707b66c6ddd198939b1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:59.221395  7610 consensus_peers.cc:597] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer 55cabad661424707b66c6ddd198939b1 (127.27.71.130:46359): Couldn't send request to peer 55cabad661424707b66c6ddd198939b1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:46359: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.233083 27934 tablet_server.cc:196] TabletServer@127.27.71.130:0 shutdown complete.
I20260709 13:08:59.276777 27934 tablet_server.cc:179] TabletServer@127.27.71.131:0 shutting down...
W20260709 13:08:59.307341  7610 consensus_peers.cc:597] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 -> Peer f7f82d1c7a294db588c3fc4e48c6012d (127.27.71.129:42053): Couldn't send request to peer f7f82d1c7a294db588c3fc4e48c6012d. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.327183 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:59.328245 27934 tablet_replica.cc:333] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4: stopping tablet replica
I20260709 13:08:59.329062 27934 raft_consensus.cc:2243] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.330019 27934 raft_consensus.cc:2272] T 3f39053c868b4252acca47e18d123fb0 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.332799 27934 tablet_replica.cc:333] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4: stopping tablet replica
I20260709 13:08:59.333408 27934 raft_consensus.cc:2243] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.333884 27934 raft_consensus.cc:2272] T 68a7ef75b1e34928b01a24437f9ab01f P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.336440 27934 tablet_replica.cc:333] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4: stopping tablet replica
I20260709 13:08:59.337024 27934 raft_consensus.cc:2243] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.337946 27934 raft_consensus.cc:2272] T a34c585c5de24649b746f9cd53cc8e57 P 69c9355bd3b449d088b37517aed3a2a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.340466 27934 tablet_replica.cc:333] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4: stopping tablet replica
I20260709 13:08:59.341065 27934 raft_consensus.cc:2243] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.341938 27934 raft_consensus.cc:2272] T 15cc71669e814555aa12d1e2eb776c95 P 69c9355bd3b449d088b37517aed3a2a4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.344483 27934 tablet_replica.cc:333] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4: stopping tablet replica
I20260709 13:08:59.345055 27934 raft_consensus.cc:2243] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.345860 27934 raft_consensus.cc:2272] T ad99cf019e854bea806a64aaabd61f70 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.348191 27934 tablet_replica.cc:333] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4: stopping tablet replica
I20260709 13:08:59.348747 27934 raft_consensus.cc:2243] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.349149 27934 raft_consensus.cc:2272] T d7aca33050e64b7595613a13182a0738 P 69c9355bd3b449d088b37517aed3a2a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.352979 27934 tablet_replica.cc:333] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4: stopping tablet replica
I20260709 13:08:59.353610 27934 raft_consensus.cc:2243] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.354050 27934 raft_consensus.cc:2272] T 16926f7e3623437c9c6d389bb22ecb59 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.356554 27934 tablet_replica.cc:333] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4: stopping tablet replica
I20260709 13:08:59.357065 27934 raft_consensus.cc:2243] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.357482 27934 raft_consensus.cc:2272] T ee14275e88d648c6982fe9fc771ed0b7 P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.359735 27934 tablet_replica.cc:333] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4: stopping tablet replica
I20260709 13:08:59.360301 27934 raft_consensus.cc:2243] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.360713 27934 raft_consensus.cc:2272] T 7bc65eac152241638209dea1a9158863 P 69c9355bd3b449d088b37517aed3a2a4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.362906 27934 tablet_replica.cc:333] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4: stopping tablet replica
I20260709 13:08:59.363415 27934 raft_consensus.cc:2243] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:59.363822 27934 raft_consensus.cc:2272] T 23bbc33e47fb45e38236cc96db1d262f P 69c9355bd3b449d088b37517aed3a2a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.402760 27934 tablet_server.cc:196] TabletServer@127.27.71.131:0 shutdown complete.
I20260709 13:08:59.436264 27934 master.cc:562] Master@127.27.71.190:43555 shutting down...
I20260709 13:08:59.467003 27934 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:59.467702 27934 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:59.468112 27934 tablet_replica.cc:333] T 00000000000000000000000000000000 P e4ef72a1896142cdadf26f51300d52a4: stopping tablet replica
I20260709 13:08:59.494155 27934 master.cc:584] Master@127.27.71.190:43555 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13029 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:08:59.547189 27934 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.71.190:45945
I20260709 13:08:59.548729 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:59.556322  7969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:59.556418  7968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:59.557533  7971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:59.558970 27934 server_base.cc:1061] running on GCE node
I20260709 13:08:59.560290 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:59.560494 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:59.560689 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602539560626 us; error 0 us; skew 500 ppm
I20260709 13:08:59.561538 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:59.565305 27934 webserver.cc:533] Webserver started at http://127.27.71.190:35053/ using document root <none> and password file <none>
I20260709 13:08:59.566052 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:59.566308 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:59.566681 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:59.568464 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/master-0-root/instance:
uuid: "bdb6446b6b9d48ed9a5cb824df954423"
format_stamp: "Formatted at 2026-07-09 13:08:59 on dist-test-slave-mlt7"
I20260709 13:08:59.574179 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 13:08:59.578101  7976 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:59.578909 27934 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:08:59.579233 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/master-0-root
uuid: "bdb6446b6b9d48ed9a5cb824df954423"
format_stamp: "Formatted at 2026-07-09 13:08:59 on dist-test-slave-mlt7"
I20260709 13:08:59.579602 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:59.607435 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:59.608762 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:59.651901 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.190:45945
I20260709 13:08:59.652109  8027 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.190:45945 every 8 connection(s)
I20260709 13:08:59.656402  8028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:59.671293  8028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423: Bootstrap starting.
I20260709 13:08:59.676846  8028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.681846  8028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423: No bootstrap required, opened a new log
I20260709 13:08:59.684146  8028 raft_consensus.cc:359] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb6446b6b9d48ed9a5cb824df954423" member_type: VOTER }
I20260709 13:08:59.684568  8028 raft_consensus.cc:385] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.684824  8028 raft_consensus.cc:740] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb6446b6b9d48ed9a5cb824df954423, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.685456  8028 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb6446b6b9d48ed9a5cb824df954423" member_type: VOTER }
I20260709 13:08:59.685915  8028 raft_consensus.cc:399] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:08:59.686143  8028 raft_consensus.cc:493] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:08:59.686406  8028 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.691629  8028 raft_consensus.cc:515] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb6446b6b9d48ed9a5cb824df954423" member_type: VOTER }
I20260709 13:08:59.692224  8028 leader_election.cc:304] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdb6446b6b9d48ed9a5cb824df954423; no voters: 
I20260709 13:08:59.693450  8028 leader_election.cc:290] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:08:59.693852  8031 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:59.695436  8031 raft_consensus.cc:697] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [term 1 LEADER]: Becoming Leader. State: Replica: bdb6446b6b9d48ed9a5cb824df954423, State: Running, Role: LEADER
I20260709 13:08:59.696177  8031 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb6446b6b9d48ed9a5cb824df954423" member_type: VOTER }
I20260709 13:08:59.696980  8028 sys_catalog.cc:565] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:08:59.701341  8033 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdb6446b6b9d48ed9a5cb824df954423. Latest consensus state: current_term: 1 leader_uuid: "bdb6446b6b9d48ed9a5cb824df954423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb6446b6b9d48ed9a5cb824df954423" member_type: VOTER } }
I20260709 13:08:59.702289  8033 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:59.701259  8032 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdb6446b6b9d48ed9a5cb824df954423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb6446b6b9d48ed9a5cb824df954423" member_type: VOTER } }
I20260709 13:08:59.703325  8032 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:59.704739  8038 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:08:59.713210  8038 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:08:59.714349 27934 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:08:59.722934  8038 catalog_manager.cc:1383] Generated new cluster ID: a78772abe2944a178b198b52960bddbd
I20260709 13:08:59.723555  8038 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:08:59.738334  8038 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:08:59.739794  8038 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:08:59.756213  8038 catalog_manager.cc:6070] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423: Generated new TSK 0
I20260709 13:08:59.756945  8038 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:08:59.781404 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:59.788105  8049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:59.789345  8050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:59.792650 27934 server_base.cc:1061] running on GCE node
W20260709 13:08:59.793889  8052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:59.794911 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:59.795115 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:59.795240 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602539795229 us; error 0 us; skew 500 ppm
I20260709 13:08:59.795746 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:59.798205 27934 webserver.cc:533] Webserver started at http://127.27.71.129:43159/ using document root <none> and password file <none>
I20260709 13:08:59.798694 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:59.798934 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:59.799214 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:59.800457 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-0-root/instance:
uuid: "617b49f5079e45d3934f23cfd7130d93"
format_stamp: "Formatted at 2026-07-09 13:08:59 on dist-test-slave-mlt7"
I20260709 13:08:59.805944 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:08:59.809896  8057 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:59.810870 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:08:59.811210 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-0-root
uuid: "617b49f5079e45d3934f23cfd7130d93"
format_stamp: "Formatted at 2026-07-09 13:08:59 on dist-test-slave-mlt7"
I20260709 13:08:59.811470 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:59.835444 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:59.836781 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:59.838337 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:59.840831 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:59.841009 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:59.841324 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:59.841482 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:59.884356 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.129:41207
I20260709 13:08:59.884474  8119 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.129:41207 every 8 connection(s)
I20260709 13:08:59.889259 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:59.897195  8124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:59.899014  8125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:59.901939 27934 server_base.cc:1061] running on GCE node
W20260709 13:08:59.903075  8127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:59.903533  8120 heartbeater.cc:344] Connected to a master server at 127.27.71.190:45945
I20260709 13:08:59.904064  8120 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:59.904225 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:59.904515 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:59.904706 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602539904690 us; error 0 us; skew 500 ppm
I20260709 13:08:59.904933  8120 heartbeater.cc:507] Master 127.27.71.190:45945 requested a full tablet report, sending...
I20260709 13:08:59.905364 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:59.907269  7993 ts_manager.cc:194] Registered new tserver with Master: 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129:41207)
I20260709 13:08:59.908413 27934 webserver.cc:533] Webserver started at http://127.27.71.130:41355/ using document root <none> and password file <none>
I20260709 13:08:59.909022 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:59.909236 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:59.909305  7993 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39326
I20260709 13:08:59.909498 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:59.910691 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-1-root/instance:
uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
format_stamp: "Formatted at 2026-07-09 13:08:59 on dist-test-slave-mlt7"
I20260709 13:08:59.915297 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:08:59.918952  8132 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:59.920019 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:08:59.920307 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-1-root
uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
format_stamp: "Formatted at 2026-07-09 13:08:59 on dist-test-slave-mlt7"
I20260709 13:08:59.920650 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:59.943143 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:59.944573 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:59.946192 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:59.949067 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:59.949337 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:59.949617 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:59.949800 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:59.992137 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.130:38887
I20260709 13:08:59.992264  8194 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.130:38887 every 8 connection(s)
I20260709 13:08:59.997506 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:00.005513  8198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:00.007323  8199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:00.010027  8201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:00.010305 27934 server_base.cc:1061] running on GCE node
I20260709 13:09:00.011451  8195 heartbeater.cc:344] Connected to a master server at 127.27.71.190:45945
I20260709 13:09:00.011554 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:09:00.011994  8195 heartbeater.cc:461] Registering TS with master...
W20260709 13:09:00.011996 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:00.012673 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602540012647 us; error 0 us; skew 500 ppm
I20260709 13:09:00.013324 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:00.013314  8195 heartbeater.cc:507] Master 127.27.71.190:45945 requested a full tablet report, sending...
I20260709 13:09:00.015849  7993 ts_manager.cc:194] Registered new tserver with Master: 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130:38887)
I20260709 13:09:00.016208 27934 webserver.cc:533] Webserver started at http://127.27.71.131:33553/ using document root <none> and password file <none>
I20260709 13:09:00.016821 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:00.017045 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:00.017345 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:00.017598  7993 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39328
I20260709 13:09:00.018772 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-2-root/instance:
uuid: "a07045b945684233bc871fbb82844387"
format_stamp: "Formatted at 2026-07-09 13:09:00 on dist-test-slave-mlt7"
I20260709 13:09:00.023761 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.007s
I20260709 13:09:00.027494  8206 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:00.028520 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:09:00.028880 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-2-root
uuid: "a07045b945684233bc871fbb82844387"
format_stamp: "Formatted at 2026-07-09 13:09:00 on dist-test-slave-mlt7"
I20260709 13:09:00.029251 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602179482400-27934-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:00.042311 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:00.043828 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:00.046214 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:00.050406 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:00.050772 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:00.051097 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:00.051316 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:00.095650 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.131:37367
I20260709 13:09:00.095767  8268 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.131:37367 every 8 connection(s)
I20260709 13:09:00.110158  8269 heartbeater.cc:344] Connected to a master server at 127.27.71.190:45945
I20260709 13:09:00.110661  8269 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:00.111429  8269 heartbeater.cc:507] Master 127.27.71.190:45945 requested a full tablet report, sending...
I20260709 13:09:00.113682  7993 ts_manager.cc:194] Registered new tserver with Master: a07045b945684233bc871fbb82844387 (127.27.71.131:37367)
I20260709 13:09:00.113902 27934 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014965012s
I20260709 13:09:00.115149  7993 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39332
I20260709 13:09:00.141708  7993 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39344:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:00.144083  7993 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:00.210369  8234 tablet_service.cc:1511] Processing CreateTablet for tablet 3fa6cb1236584b8b8b545409fe2fb7c6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=898ea6330ceb4b7481abeba9a7265162]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:00.211892  8234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fa6cb1236584b8b8b545409fe2fb7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.211948  8233 tablet_service.cc:1511] Processing CreateTablet for tablet d6b4038d8cb9459e94488278937c9e30 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=898ea6330ceb4b7481abeba9a7265162]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:00.213653  8233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6b4038d8cb9459e94488278937c9e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.214112  8085 tablet_service.cc:1511] Processing CreateTablet for tablet 3fa6cb1236584b8b8b545409fe2fb7c6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=898ea6330ceb4b7481abeba9a7265162]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:00.215590  8085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fa6cb1236584b8b8b545409fe2fb7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.217856  8160 tablet_service.cc:1511] Processing CreateTablet for tablet 3fa6cb1236584b8b8b545409fe2fb7c6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=898ea6330ceb4b7481abeba9a7265162]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:00.219318  8160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fa6cb1236584b8b8b545409fe2fb7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.222700  8084 tablet_service.cc:1511] Processing CreateTablet for tablet d6b4038d8cb9459e94488278937c9e30 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=898ea6330ceb4b7481abeba9a7265162]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:00.223591  8159 tablet_service.cc:1511] Processing CreateTablet for tablet d6b4038d8cb9459e94488278937c9e30 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=898ea6330ceb4b7481abeba9a7265162]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:00.224206  8084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6b4038d8cb9459e94488278937c9e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.225008  8159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6b4038d8cb9459e94488278937c9e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.247875  8284 tablet_bootstrap.cc:492] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387: Bootstrap starting.
I20260709 13:09:00.257905  8286 tablet_bootstrap.cc:492] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2: Bootstrap starting.
I20260709 13:09:00.259903  8284 tablet_bootstrap.cc:654] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.259869  8285 tablet_bootstrap.cc:492] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93: Bootstrap starting.
I20260709 13:09:00.268092  8286 tablet_bootstrap.cc:654] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.269629  8284 tablet_bootstrap.cc:492] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387: No bootstrap required, opened a new log
I20260709 13:09:00.270300  8284 ts_tablet_manager.cc:1403] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.006s
I20260709 13:09:00.270516  8285 tablet_bootstrap.cc:654] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.273761  8286 tablet_bootstrap.cc:492] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2: No bootstrap required, opened a new log
I20260709 13:09:00.274318  8286 ts_tablet_manager.cc:1403] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.004s
I20260709 13:09:00.274075  8284 raft_consensus.cc:359] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:00.275158  8284 raft_consensus.cc:385] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.275454  8284 raft_consensus.cc:740] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07045b945684233bc871fbb82844387, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.276295  8284 consensus_queue.cc:260] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:00.277588  8285 tablet_bootstrap.cc:492] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93: No bootstrap required, opened a new log
I20260709 13:09:00.277558  8286 raft_consensus.cc:359] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:00.278594  8286 raft_consensus.cc:385] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.278689  8285 ts_tablet_manager.cc:1403] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.002s
I20260709 13:09:00.278966  8286 raft_consensus.cc:740] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67bdf3c8792c41f4993f9f63ebe059b2, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.279453  8269 heartbeater.cc:499] Master 127.27.71.190:45945 was elected leader, sending a full tablet report...
I20260709 13:09:00.279850  8284 ts_tablet_manager.cc:1434] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 13:09:00.280010  8286 consensus_queue.cc:260] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:00.280977  8284 tablet_bootstrap.cc:492] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387: Bootstrap starting.
I20260709 13:09:00.282101  8285 raft_consensus.cc:359] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:00.282868  8285 raft_consensus.cc:385] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.283174  8285 raft_consensus.cc:740] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 617b49f5079e45d3934f23cfd7130d93, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.284099  8285 consensus_queue.cc:260] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:00.289783  8284 tablet_bootstrap.cc:654] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.294006  8285 ts_tablet_manager.cc:1434] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93: Time spent starting tablet: real 0.015s	user 0.013s	sys 0.002s
I20260709 13:09:00.295120  8120 heartbeater.cc:499] Master 127.27.71.190:45945 was elected leader, sending a full tablet report...
I20260709 13:09:00.305214  8285 tablet_bootstrap.cc:492] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93: Bootstrap starting.
I20260709 13:09:00.314455  8195 heartbeater.cc:499] Master 127.27.71.190:45945 was elected leader, sending a full tablet report...
I20260709 13:09:00.317176  8286 ts_tablet_manager.cc:1434] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2: Time spent starting tablet: real 0.042s	user 0.032s	sys 0.007s
I20260709 13:09:00.318276  8286 tablet_bootstrap.cc:492] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2: Bootstrap starting.
I20260709 13:09:00.319247  8285 tablet_bootstrap.cc:654] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.322110  8284 tablet_bootstrap.cc:492] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387: No bootstrap required, opened a new log
I20260709 13:09:00.322803  8284 ts_tablet_manager.cc:1403] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.023s
I20260709 13:09:00.325930  8284 raft_consensus.cc:359] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:00.326457  8286 tablet_bootstrap.cc:654] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.326881  8284 raft_consensus.cc:385] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.327308  8284 raft_consensus.cc:740] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07045b945684233bc871fbb82844387, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.328346  8284 consensus_queue.cc:260] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:00.331653  8285 tablet_bootstrap.cc:492] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93: No bootstrap required, opened a new log
I20260709 13:09:00.332285  8285 ts_tablet_manager.cc:1403] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.002s
I20260709 13:09:00.332418  8284 ts_tablet_manager.cc:1434] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:09:00.334829  8286 tablet_bootstrap.cc:492] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2: No bootstrap required, opened a new log
I20260709 13:09:00.335418  8286 ts_tablet_manager.cc:1403] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 13:09:00.335675  8285 raft_consensus.cc:359] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:00.336570  8285 raft_consensus.cc:385] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.336875  8285 raft_consensus.cc:740] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 617b49f5079e45d3934f23cfd7130d93, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.337672  8285 consensus_queue.cc:260] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:00.338829  8286 raft_consensus.cc:359] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:00.339668  8286 raft_consensus.cc:385] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.340027  8285 ts_tablet_manager.cc:1434] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.005s
I20260709 13:09:00.339988  8286 raft_consensus.cc:740] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67bdf3c8792c41f4993f9f63ebe059b2, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.341001  8286 consensus_queue.cc:260] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:00.342658  8286 ts_tablet_manager.cc:1434] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:00.359520  8290 raft_consensus.cc:493] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.360173  8290 raft_consensus.cc:515] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:00.362498  8290 leader_election.cc:290] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129:41207), 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130:38887)
I20260709 13:09:00.371874  8095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6b4038d8cb9459e94488278937c9e30" candidate_uuid: "a07045b945684233bc871fbb82844387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617b49f5079e45d3934f23cfd7130d93" is_pre_election: true
I20260709 13:09:00.372797  8095 raft_consensus.cc:2468] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07045b945684233bc871fbb82844387 in term 0.
I20260709 13:09:00.373540  8170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6b4038d8cb9459e94488278937c9e30" candidate_uuid: "a07045b945684233bc871fbb82844387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" is_pre_election: true
I20260709 13:09:00.374359  8170 raft_consensus.cc:2468] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07045b945684233bc871fbb82844387 in term 0.
I20260709 13:09:00.374374  8209 leader_election.cc:304] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b49f5079e45d3934f23cfd7130d93, a07045b945684233bc871fbb82844387; no voters: 
I20260709 13:09:00.375371  8290 raft_consensus.cc:2804] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:00.375778  8290 raft_consensus.cc:493] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.376132  8290 raft_consensus.cc:3060] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.383502  8290 raft_consensus.cc:515] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:00.385432  8290 leader_election.cc:290] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [CANDIDATE]: Term 1 election: Requested vote from peers 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129:41207), 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130:38887)
I20260709 13:09:00.386483  8095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6b4038d8cb9459e94488278937c9e30" candidate_uuid: "a07045b945684233bc871fbb82844387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617b49f5079e45d3934f23cfd7130d93"
I20260709 13:09:00.387019  8170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6b4038d8cb9459e94488278937c9e30" candidate_uuid: "a07045b945684233bc871fbb82844387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
I20260709 13:09:00.387054  8095 raft_consensus.cc:3060] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.387609  8170 raft_consensus.cc:3060] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.393406  8095 raft_consensus.cc:2468] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07045b945684233bc871fbb82844387 in term 1.
I20260709 13:09:00.393756  8170 raft_consensus.cc:2468] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07045b945684233bc871fbb82844387 in term 1.
I20260709 13:09:00.394670  8209 leader_election.cc:304] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b49f5079e45d3934f23cfd7130d93, a07045b945684233bc871fbb82844387; no voters: 
I20260709 13:09:00.395738  8290 raft_consensus.cc:2804] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:00.396963  8290 raft_consensus.cc:697] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 1 LEADER]: Becoming Leader. State: Replica: a07045b945684233bc871fbb82844387, State: Running, Role: LEADER
I20260709 13:09:00.397854  8290 consensus_queue.cc:237] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:00.407233  7993 catalog_manager.cc:5697] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 reported cstate change: term changed from 0 to 1, leader changed from <none> to a07045b945684233bc871fbb82844387 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "a07045b945684233bc871fbb82844387" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:00.698596  8291 raft_consensus.cc:493] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.699409  8291 raft_consensus.cc:515] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:00.701833  8291 leader_election.cc:290] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a07045b945684233bc871fbb82844387 (127.27.71.131:37367), 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129:41207)
W20260709 13:09:00.707880  8041 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:00.713562  8095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa6cb1236584b8b8b545409fe2fb7c6" candidate_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617b49f5079e45d3934f23cfd7130d93" is_pre_election: true
I20260709 13:09:00.713582  8244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa6cb1236584b8b8b545409fe2fb7c6" candidate_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07045b945684233bc871fbb82844387" is_pre_election: true
I20260709 13:09:00.714529  8095 raft_consensus.cc:2468] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67bdf3c8792c41f4993f9f63ebe059b2 in term 0.
I20260709 13:09:00.714632  8244 raft_consensus.cc:2468] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67bdf3c8792c41f4993f9f63ebe059b2 in term 0.
I20260709 13:09:00.715692  8135 leader_election.cc:304] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b49f5079e45d3934f23cfd7130d93, 67bdf3c8792c41f4993f9f63ebe059b2; no voters: 
I20260709 13:09:00.716550  8291 raft_consensus.cc:2804] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:00.716974  8291 raft_consensus.cc:493] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.717264  8291 raft_consensus.cc:3060] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.723941  8291 raft_consensus.cc:515] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:00.725967  8291 leader_election.cc:290] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [CANDIDATE]: Term 1 election: Requested vote from peers a07045b945684233bc871fbb82844387 (127.27.71.131:37367), 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129:41207)
I20260709 13:09:00.726882  8244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa6cb1236584b8b8b545409fe2fb7c6" candidate_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07045b945684233bc871fbb82844387"
I20260709 13:09:00.727052  8095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa6cb1236584b8b8b545409fe2fb7c6" candidate_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617b49f5079e45d3934f23cfd7130d93"
I20260709 13:09:00.727528  8244 raft_consensus.cc:3060] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.727746  8095 raft_consensus.cc:3060] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.734119  8095 raft_consensus.cc:2468] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bdf3c8792c41f4993f9f63ebe059b2 in term 1.
I20260709 13:09:00.734252  8244 raft_consensus.cc:2468] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bdf3c8792c41f4993f9f63ebe059b2 in term 1.
I20260709 13:09:00.735502  8135 leader_election.cc:304] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b49f5079e45d3934f23cfd7130d93, 67bdf3c8792c41f4993f9f63ebe059b2; no voters: 
I20260709 13:09:00.736508  8291 raft_consensus.cc:2804] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:00.737476  8291 raft_consensus.cc:697] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 LEADER]: Becoming Leader. State: Replica: 67bdf3c8792c41f4993f9f63ebe059b2, State: Running, Role: LEADER
I20260709 13:09:00.738224  8291 consensus_queue.cc:237] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:00.746256  7993 catalog_manager.cc:5697] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:00.831050  7993 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39360:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:00.833791  7993 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:00.856012  8290 consensus_queue.cc:1048] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:00.864771  8084 tablet_service.cc:1511] Processing CreateTablet for tablet 5092d2ec1f6447268cad1232a1e94f74 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3a0f88501eb446cd90a283193af8432a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:00.866106  8084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5092d2ec1f6447268cad1232a1e94f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.868424  8160 tablet_service.cc:1511] Processing CreateTablet for tablet 5092d2ec1f6447268cad1232a1e94f74 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3a0f88501eb446cd90a283193af8432a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:00.870110  8160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5092d2ec1f6447268cad1232a1e94f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.873694  8233 tablet_service.cc:1511] Processing CreateTablet for tablet 5092d2ec1f6447268cad1232a1e94f74 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3a0f88501eb446cd90a283193af8432a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:09:00.875276  8233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5092d2ec1f6447268cad1232a1e94f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.891808  8085 tablet_service.cc:1511] Processing CreateTablet for tablet 17af902b039f47c597f39ee88ebaa971 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3a0f88501eb446cd90a283193af8432a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:00.893415  8085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17af902b039f47c597f39ee88ebaa971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.900125  8297 consensus_queue.cc:1048] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:00.908506  8159 tablet_service.cc:1511] Processing CreateTablet for tablet 17af902b039f47c597f39ee88ebaa971 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3a0f88501eb446cd90a283193af8432a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:00.910019  8159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17af902b039f47c597f39ee88ebaa971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.914043  8234 tablet_service.cc:1511] Processing CreateTablet for tablet 17af902b039f47c597f39ee88ebaa971 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=3a0f88501eb446cd90a283193af8432a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:09:00.918862  8313 tablet_bootstrap.cc:492] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2: Bootstrap starting.
I20260709 13:09:00.920359  8234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17af902b039f47c597f39ee88ebaa971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:00.929901  8312 tablet_bootstrap.cc:492] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93: Bootstrap starting.
I20260709 13:09:00.949949  8312 tablet_bootstrap.cc:654] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.959589  8313 tablet_bootstrap.cc:654] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.967981  8316 tablet_bootstrap.cc:492] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387: Bootstrap starting.
I20260709 13:09:00.978359  8316 tablet_bootstrap.cc:654] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.007874  8312 tablet_bootstrap.cc:492] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93: No bootstrap required, opened a new log
I20260709 13:09:01.008524  8312 ts_tablet_manager.cc:1403] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93: Time spent bootstrapping tablet: real 0.079s	user 0.019s	sys 0.039s
I20260709 13:09:01.009743  8316 tablet_bootstrap.cc:492] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387: No bootstrap required, opened a new log
I20260709 13:09:01.009773  8313 tablet_bootstrap.cc:492] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2: No bootstrap required, opened a new log
I20260709 13:09:01.010617  8316 ts_tablet_manager.cc:1403] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.011s
I20260709 13:09:01.010617  8313 ts_tablet_manager.cc:1403] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2: Time spent bootstrapping tablet: real 0.092s	user 0.029s	sys 0.028s
I20260709 13:09:01.012104  8312 raft_consensus.cc:359] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.012964  8312 raft_consensus.cc:385] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.013341  8312 raft_consensus.cc:740] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 617b49f5079e45d3934f23cfd7130d93, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.014196  8313 raft_consensus.cc:359] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.014222  8312 consensus_queue.cc:260] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.014447  8316 raft_consensus.cc:359] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.015399  8313 raft_consensus.cc:385] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.015661  8316 raft_consensus.cc:385] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.015856  8313 raft_consensus.cc:740] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67bdf3c8792c41f4993f9f63ebe059b2, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.016146  8316 raft_consensus.cc:740] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07045b945684233bc871fbb82844387, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.016957  8313 consensus_queue.cc:260] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.017550  8312 ts_tablet_manager.cc:1434] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 13:09:01.017179  8316 consensus_queue.cc:260] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.019037  8312 tablet_bootstrap.cc:492] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93: Bootstrap starting.
I20260709 13:09:01.020299  8316 ts_tablet_manager.cc:1434] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.003s
I20260709 13:09:01.021566  8316 tablet_bootstrap.cc:492] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387: Bootstrap starting.
I20260709 13:09:01.021888  8313 ts_tablet_manager.cc:1434] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:09:01.023214  8313 tablet_bootstrap.cc:492] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2: Bootstrap starting.
I20260709 13:09:01.028472  8312 tablet_bootstrap.cc:654] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.030529  8316 tablet_bootstrap.cc:654] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.030650  8313 tablet_bootstrap.cc:654] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.043296  8312 tablet_bootstrap.cc:492] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93: No bootstrap required, opened a new log
I20260709 13:09:01.044314  8312 ts_tablet_manager.cc:1403] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.004s
I20260709 13:09:01.048979  8312 raft_consensus.cc:359] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.052836  8312 raft_consensus.cc:385] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.053459  8312 raft_consensus.cc:740] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 617b49f5079e45d3934f23cfd7130d93, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.054939  8312 consensus_queue.cc:260] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.056069  8316 tablet_bootstrap.cc:492] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387: No bootstrap required, opened a new log
I20260709 13:09:01.056870  8316 ts_tablet_manager.cc:1403] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.012s
I20260709 13:09:01.057721  8312 ts_tablet_manager.cc:1434] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:09:01.060276  8316 raft_consensus.cc:359] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.061311  8316 raft_consensus.cc:385] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.061720  8316 raft_consensus.cc:740] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07045b945684233bc871fbb82844387, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.062779  8316 consensus_queue.cc:260] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.073948  8316 ts_tablet_manager.cc:1434] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.010s
I20260709 13:09:01.075911  8313 tablet_bootstrap.cc:492] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2: No bootstrap required, opened a new log
I20260709 13:09:01.076802  8313 ts_tablet_manager.cc:1403] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.032s
I20260709 13:09:01.079813  8313 raft_consensus.cc:359] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.080569  8313 raft_consensus.cc:385] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.080861  8313 raft_consensus.cc:740] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67bdf3c8792c41f4993f9f63ebe059b2, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.081710  8313 consensus_queue.cc:260] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.084201  8313 ts_tablet_manager.cc:1434] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:09:01.131107  8291 raft_consensus.cc:493] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.131940  8291 raft_consensus.cc:515] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.134254  8291 leader_election.cc:290] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129:41207), a07045b945684233bc871fbb82844387 (127.27.71.131:37367)
I20260709 13:09:01.135404  8095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5092d2ec1f6447268cad1232a1e94f74" candidate_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617b49f5079e45d3934f23cfd7130d93" is_pre_election: true
I20260709 13:09:01.135551  8244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5092d2ec1f6447268cad1232a1e94f74" candidate_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07045b945684233bc871fbb82844387" is_pre_election: true
I20260709 13:09:01.136186  8095 raft_consensus.cc:2468] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67bdf3c8792c41f4993f9f63ebe059b2 in term 0.
I20260709 13:09:01.136260  8244 raft_consensus.cc:2468] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67bdf3c8792c41f4993f9f63ebe059b2 in term 0.
I20260709 13:09:01.137651  8135 leader_election.cc:304] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b49f5079e45d3934f23cfd7130d93, 67bdf3c8792c41f4993f9f63ebe059b2; no voters: 
I20260709 13:09:01.138857  8291 raft_consensus.cc:2804] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.139292  8291 raft_consensus.cc:493] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.139581  8291 raft_consensus.cc:3060] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.146760  8291 raft_consensus.cc:515] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.148603  8291 leader_election.cc:290] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [CANDIDATE]: Term 1 election: Requested vote from peers 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129:41207), a07045b945684233bc871fbb82844387 (127.27.71.131:37367)
I20260709 13:09:01.149719  8244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5092d2ec1f6447268cad1232a1e94f74" candidate_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07045b945684233bc871fbb82844387"
I20260709 13:09:01.149721  8095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5092d2ec1f6447268cad1232a1e94f74" candidate_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617b49f5079e45d3934f23cfd7130d93"
I20260709 13:09:01.150432  8244 raft_consensus.cc:3060] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.151535  8095 raft_consensus.cc:3060] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.157774  8244 raft_consensus.cc:2468] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bdf3c8792c41f4993f9f63ebe059b2 in term 1.
I20260709 13:09:01.157774  8095 raft_consensus.cc:2468] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bdf3c8792c41f4993f9f63ebe059b2 in term 1.
I20260709 13:09:01.159281  8134 leader_election.cc:304] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67bdf3c8792c41f4993f9f63ebe059b2, a07045b945684233bc871fbb82844387; no voters: 
I20260709 13:09:01.160108  8291 raft_consensus.cc:2804] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.160646  8291 raft_consensus.cc:697] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 LEADER]: Becoming Leader. State: Replica: 67bdf3c8792c41f4993f9f63ebe059b2, State: Running, Role: LEADER
I20260709 13:09:01.161527  8291 consensus_queue.cc:237] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.169770  7990 catalog_manager.cc:5697] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:01.175386  8291 consensus_queue.cc:1048] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:01.188724  8301 consensus_queue.cc:1048] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:01.292080  8290 raft_consensus.cc:493] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.292724  8290 raft_consensus.cc:515] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.294961  8290 leader_election.cc:290] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129:41207), 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130:38887)
I20260709 13:09:01.295994  8095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971" candidate_uuid: "a07045b945684233bc871fbb82844387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617b49f5079e45d3934f23cfd7130d93" is_pre_election: true
I20260709 13:09:01.296298  8170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971" candidate_uuid: "a07045b945684233bc871fbb82844387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" is_pre_election: true
I20260709 13:09:01.296741  8095 raft_consensus.cc:2468] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07045b945684233bc871fbb82844387 in term 0.
I20260709 13:09:01.297089  8170 raft_consensus.cc:2468] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07045b945684233bc871fbb82844387 in term 0.
I20260709 13:09:01.297892  8209 leader_election.cc:304] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b49f5079e45d3934f23cfd7130d93, a07045b945684233bc871fbb82844387; no voters: 
I20260709 13:09:01.298880  8290 raft_consensus.cc:2804] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.299350  8290 raft_consensus.cc:493] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.299624  8290 raft_consensus.cc:3060] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.305807  8290 raft_consensus.cc:515] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.307739  8290 leader_election.cc:290] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [CANDIDATE]: Term 1 election: Requested vote from peers 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129:41207), 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130:38887)
I20260709 13:09:01.308678  8095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971" candidate_uuid: "a07045b945684233bc871fbb82844387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617b49f5079e45d3934f23cfd7130d93"
I20260709 13:09:01.308897  8170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971" candidate_uuid: "a07045b945684233bc871fbb82844387" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
I20260709 13:09:01.309341  8095 raft_consensus.cc:3060] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.309561  8170 raft_consensus.cc:3060] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.315681  8095 raft_consensus.cc:2468] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07045b945684233bc871fbb82844387 in term 1.
I20260709 13:09:01.316136  8170 raft_consensus.cc:2468] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07045b945684233bc871fbb82844387 in term 1.
I20260709 13:09:01.316919  8209 leader_election.cc:304] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b49f5079e45d3934f23cfd7130d93, a07045b945684233bc871fbb82844387; no voters: 
I20260709 13:09:01.317909  8290 raft_consensus.cc:2804] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.318385  8290 raft_consensus.cc:697] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 1 LEADER]: Becoming Leader. State: Replica: a07045b945684233bc871fbb82844387, State: Running, Role: LEADER
I20260709 13:09:01.319343  8290 consensus_queue.cc:237] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.327718  7993 catalog_manager.cc:5697] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 reported cstate change: term changed from 0 to 1, leader changed from <none> to a07045b945684233bc871fbb82844387 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "a07045b945684233bc871fbb82844387" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:01.436609  8170 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fa6cb1236584b8b8b545409fe2fb7c6"
dest_uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
mode: GRACEFUL
new_leader_uuid: "617b49f5079e45d3934f23cfd7130d93"
 from {username='slave'} at 127.0.0.1:43888
I20260709 13:09:01.437269  8170 raft_consensus.cc:606] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 LEADER]: Received request to transfer leadership to TS 617b49f5079e45d3934f23cfd7130d93
I20260709 13:09:01.456633  8244 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6b4038d8cb9459e94488278937c9e30"
dest_uuid: "a07045b945684233bc871fbb82844387"
mode: GRACEFUL
new_leader_uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
 from {username='slave'} at 127.0.0.1:59894
I20260709 13:09:01.457333  8244 raft_consensus.cc:606] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 1 LEADER]: Received request to transfer leadership to TS 67bdf3c8792c41f4993f9f63ebe059b2
I20260709 13:09:01.465211  8170 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5092d2ec1f6447268cad1232a1e94f74"
dest_uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
mode: GRACEFUL
new_leader_uuid: "617b49f5079e45d3934f23cfd7130d93"
 from {username='slave'} at 127.0.0.1:43888
I20260709 13:09:01.465975  8170 raft_consensus.cc:606] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 LEADER]: Received request to transfer leadership to TS 617b49f5079e45d3934f23cfd7130d93
I20260709 13:09:01.474354  8244 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971"
dest_uuid: "a07045b945684233bc871fbb82844387"
mode: GRACEFUL
new_leader_uuid: "617b49f5079e45d3934f23cfd7130d93"
 from {username='slave'} at 127.0.0.1:59894
I20260709 13:09:01.475312  8244 raft_consensus.cc:606] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 1 LEADER]: Received request to transfer leadership to TS 617b49f5079e45d3934f23cfd7130d93
I20260709 13:09:01.597458  8301 consensus_queue.cc:1048] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:01.599992  8301 raft_consensus.cc:993] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2: : Instructing follower 617b49f5079e45d3934f23cfd7130d93 to start an election
I20260709 13:09:01.600540  8326 raft_consensus.cc:1081] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 LEADER]: Signalling peer 617b49f5079e45d3934f23cfd7130d93 to start an election
I20260709 13:09:01.604295  8095 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3fa6cb1236584b8b8b545409fe2fb7c6"
dest_uuid: "617b49f5079e45d3934f23cfd7130d93"
 from {username='slave'} at 127.0.0.1:53568
I20260709 13:09:01.605022  8095 raft_consensus.cc:493] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:01.605469  8095 raft_consensus.cc:3060] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.619159  8301 consensus_queue.cc:1048] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:01.628981  8291 raft_consensus.cc:993] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2: : Instructing follower 617b49f5079e45d3934f23cfd7130d93 to start an election
I20260709 13:09:01.629511  8291 raft_consensus.cc:1081] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 LEADER]: Signalling peer 617b49f5079e45d3934f23cfd7130d93 to start an election
I20260709 13:09:01.631644  8094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5092d2ec1f6447268cad1232a1e94f74"
dest_uuid: "617b49f5079e45d3934f23cfd7130d93"
 from {username='slave'} at 127.0.0.1:53568
I20260709 13:09:01.632282  8094 raft_consensus.cc:493] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:01.632679  8094 raft_consensus.cc:3060] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.639890  8094 raft_consensus.cc:515] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.640959  8095 raft_consensus.cc:515] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:01.650069  8094 leader_election.cc:290] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [CANDIDATE]: Term 2 election: Requested vote from peers 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130:38887), a07045b945684233bc871fbb82844387 (127.27.71.131:37367)
I20260709 13:09:01.654320  8095 leader_election.cc:290] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [CANDIDATE]: Term 2 election: Requested vote from peers a07045b945684233bc871fbb82844387 (127.27.71.131:37367), 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130:38887)
I20260709 13:09:01.661747  8243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa6cb1236584b8b8b545409fe2fb7c6" candidate_uuid: "617b49f5079e45d3934f23cfd7130d93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a07045b945684233bc871fbb82844387"
I20260709 13:09:01.661824  8244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5092d2ec1f6447268cad1232a1e94f74" candidate_uuid: "617b49f5079e45d3934f23cfd7130d93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a07045b945684233bc871fbb82844387"
I20260709 13:09:01.662493  8243 raft_consensus.cc:3060] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.662575  8244 raft_consensus.cc:3060] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.670661  8244 raft_consensus.cc:2468] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 617b49f5079e45d3934f23cfd7130d93 in term 2.
I20260709 13:09:01.670717  8243 raft_consensus.cc:2468] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 617b49f5079e45d3934f23cfd7130d93 in term 2.
I20260709 13:09:01.671095  8170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fa6cb1236584b8b8b545409fe2fb7c6" candidate_uuid: "617b49f5079e45d3934f23cfd7130d93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
I20260709 13:09:01.671747  8170 raft_consensus.cc:3055] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:01.671742  8169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5092d2ec1f6447268cad1232a1e94f74" candidate_uuid: "617b49f5079e45d3934f23cfd7130d93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
I20260709 13:09:01.672302  8170 raft_consensus.cc:740] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67bdf3c8792c41f4993f9f63ebe059b2, State: Running, Role: LEADER
I20260709 13:09:01.672287  8059 leader_election.cc:304] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b49f5079e45d3934f23cfd7130d93, a07045b945684233bc871fbb82844387; no voters: 
I20260709 13:09:01.672636  8169 raft_consensus.cc:3055] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:01.673518  8314 raft_consensus.cc:2804] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:01.674001  8169 raft_consensus.cc:740] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67bdf3c8792c41f4993f9f63ebe059b2, State: Running, Role: LEADER
I20260709 13:09:01.673527  8170 consensus_queue.cc:260] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:01.675161  8170 raft_consensus.cc:3060] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.673760  8059 leader_election.cc:304] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b49f5079e45d3934f23cfd7130d93, a07045b945684233bc871fbb82844387; no voters: 
I20260709 13:09:01.675175  8169 consensus_queue.cc:260] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.679257  8169 raft_consensus.cc:3060] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.684537  8170 raft_consensus.cc:2468] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 617b49f5079e45d3934f23cfd7130d93 in term 2.
I20260709 13:09:01.685214  8314 raft_consensus.cc:697] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 2 LEADER]: Becoming Leader. State: Replica: 617b49f5079e45d3934f23cfd7130d93, State: Running, Role: LEADER
I20260709 13:09:01.686852  8314 consensus_queue.cc:237] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.692487  8169 raft_consensus.cc:2468] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 617b49f5079e45d3934f23cfd7130d93 in term 2.
I20260709 13:09:01.694859  8337 raft_consensus.cc:2804] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:01.695479  8337 raft_consensus.cc:697] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 2 LEADER]: Becoming Leader. State: Replica: 617b49f5079e45d3934f23cfd7130d93, State: Running, Role: LEADER
I20260709 13:09:01.696426  8337 consensus_queue.cc:237] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } }
I20260709 13:09:01.699005  7990 catalog_manager.cc:5697] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 reported cstate change: term changed from 1 to 2, leader changed from 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130) to 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "617b49f5079e45d3934f23cfd7130d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:01.713189  7990 catalog_manager.cc:5697] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 reported cstate change: term changed from 1 to 2, leader changed from 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130) to 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "617b49f5079e45d3934f23cfd7130d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:01.811029  8297 consensus_queue.cc:1048] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:01.811417  8290 raft_consensus.cc:993] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387: : Instructing follower 67bdf3c8792c41f4993f9f63ebe059b2 to start an election
I20260709 13:09:01.812115  8329 raft_consensus.cc:1081] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 1 LEADER]: Signalling peer 67bdf3c8792c41f4993f9f63ebe059b2 to start an election
I20260709 13:09:01.813813  8169 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6b4038d8cb9459e94488278937c9e30"
dest_uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
 from {username='slave'} at 127.0.0.1:37664
I20260709 13:09:01.814371  8169 raft_consensus.cc:493] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:01.814731  8169 raft_consensus.cc:3060] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.818710  8297 raft_consensus.cc:993] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387: : Instructing follower 617b49f5079e45d3934f23cfd7130d93 to start an election
I20260709 13:09:01.819253  8290 raft_consensus.cc:1081] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 1 LEADER]: Signalling peer 617b49f5079e45d3934f23cfd7130d93 to start an election
I20260709 13:09:01.826077  8297 consensus_queue.cc:1048] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:01.826396  8169 raft_consensus.cc:515] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.830049  8243 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6b4038d8cb9459e94488278937c9e30" candidate_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a07045b945684233bc871fbb82844387"
I20260709 13:09:01.830797  8243 raft_consensus.cc:3055] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:01.831298  8243 raft_consensus.cc:740] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a07045b945684233bc871fbb82844387, State: Running, Role: LEADER
I20260709 13:09:01.832428  8243 consensus_queue.cc:260] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.832896  8094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971"
dest_uuid: "617b49f5079e45d3934f23cfd7130d93"
 from {username='slave'} at 127.0.0.1:53556
I20260709 13:09:01.833630  8094 raft_consensus.cc:493] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:01.833694  8243 raft_consensus.cc:3060] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.834131  8094 raft_consensus.cc:3060] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.832690  8095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6b4038d8cb9459e94488278937c9e30" candidate_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "617b49f5079e45d3934f23cfd7130d93"
I20260709 13:09:01.836059  8095 raft_consensus.cc:3060] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.843000  8243 raft_consensus.cc:2468] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bdf3c8792c41f4993f9f63ebe059b2 in term 2.
I20260709 13:09:01.844563  8134 leader_election.cc:304] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67bdf3c8792c41f4993f9f63ebe059b2, a07045b945684233bc871fbb82844387; no voters: 
I20260709 13:09:01.845475  8301 raft_consensus.cc:2804] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:01.845963  8301 raft_consensus.cc:697] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 LEADER]: Becoming Leader. State: Replica: 67bdf3c8792c41f4993f9f63ebe059b2, State: Running, Role: LEADER
I20260709 13:09:01.846423  8095 raft_consensus.cc:2468] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67bdf3c8792c41f4993f9f63ebe059b2 in term 2.
I20260709 13:09:01.847054  8301 consensus_queue.cc:237] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.843000  8094 raft_consensus.cc:515] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.850518  8169 leader_election.cc:290] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [CANDIDATE]: Term 2 election: Requested vote from peers 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129:41207), a07045b945684233bc871fbb82844387 (127.27.71.131:37367)
I20260709 13:09:01.852229  8244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971" candidate_uuid: "617b49f5079e45d3934f23cfd7130d93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a07045b945684233bc871fbb82844387"
I20260709 13:09:01.852411  8170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971" candidate_uuid: "617b49f5079e45d3934f23cfd7130d93" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
I20260709 13:09:01.852926  8094 leader_election.cc:290] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [CANDIDATE]: Term 2 election: Requested vote from peers 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130:38887), a07045b945684233bc871fbb82844387 (127.27.71.131:37367)
I20260709 13:09:01.852944  8244 raft_consensus.cc:3055] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:01.853041  8170 raft_consensus.cc:3060] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.853685  8244 raft_consensus.cc:740] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a07045b945684233bc871fbb82844387, State: Running, Role: LEADER
I20260709 13:09:01.854748  8244 consensus_queue.cc:260] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.856019  8244 raft_consensus.cc:3060] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:01.862340  8170 raft_consensus.cc:2468] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 617b49f5079e45d3934f23cfd7130d93 in term 2.
I20260709 13:09:01.863787  8058 leader_election.cc:304] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b49f5079e45d3934f23cfd7130d93, 67bdf3c8792c41f4993f9f63ebe059b2; no voters: 
I20260709 13:09:01.864970  8337 raft_consensus.cc:2804] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:01.865474  8337 raft_consensus.cc:697] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 2 LEADER]: Becoming Leader. State: Replica: 617b49f5079e45d3934f23cfd7130d93, State: Running, Role: LEADER
I20260709 13:09:01.865242  8244 raft_consensus.cc:2468] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 617b49f5079e45d3934f23cfd7130d93 in term 2.
I20260709 13:09:01.866411  8337 consensus_queue.cc:237] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:01.868916  7993 catalog_manager.cc:5697] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 reported cstate change: term changed from 1 to 2, leader changed from a07045b945684233bc871fbb82844387 (127.27.71.131) to 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:01.880762  7990 catalog_manager.cc:5697] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 reported cstate change: term changed from 1 to 2, leader changed from a07045b945684233bc871fbb82844387 (127.27.71.131) to 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "617b49f5079e45d3934f23cfd7130d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:02.019867 27934 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:09:02.021695 27934 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:09:02.022123 27934 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:09:02.038846  8094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971"
dest_uuid: "617b49f5079e45d3934f23cfd7130d93"
mode: GRACEFUL
new_leader_uuid: "a07045b945684233bc871fbb82844387"
 from {username='slave'} at 127.0.0.1:59812
I20260709 13:09:02.039502  8094 raft_consensus.cc:606] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 2 LEADER]: Received request to transfer leadership to TS a07045b945684233bc871fbb82844387
I20260709 13:09:02.040992 27934 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:09:02.177802  8244 raft_consensus.cc:1275] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Refusing update from remote peer 617b49f5079e45d3934f23cfd7130d93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.179220  8337 consensus_queue.cc:1048] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:02.188723  8170 raft_consensus.cc:1275] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Refusing update from remote peer 617b49f5079e45d3934f23cfd7130d93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.190680  8314 consensus_queue.cc:1048] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:02.224798  8170 raft_consensus.cc:1275] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Refusing update from remote peer 617b49f5079e45d3934f23cfd7130d93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.226184  8314 consensus_queue.cc:1048] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:02.236202  8244 raft_consensus.cc:1275] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Refusing update from remote peer 617b49f5079e45d3934f23cfd7130d93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.238013  8344 consensus_queue.cc:1048] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:02.304834  8244 raft_consensus.cc:1275] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Refusing update from remote peer 617b49f5079e45d3934f23cfd7130d93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.306565  8337 consensus_queue.cc:1048] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:02.312227  8344 raft_consensus.cc:993] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93: : Instructing follower a07045b945684233bc871fbb82844387 to start an election
I20260709 13:09:02.312186  8244 raft_consensus.cc:1275] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Refusing update from remote peer 67bdf3c8792c41f4993f9f63ebe059b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.313562  8337 raft_consensus.cc:1081] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 2 LEADER]: Signalling peer a07045b945684233bc871fbb82844387 to start an election
I20260709 13:09:02.314343  8326 consensus_queue.cc:1048] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:02.318455  8170 raft_consensus.cc:1275] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Refusing update from remote peer 617b49f5079e45d3934f23cfd7130d93: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.318815  8244 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971"
dest_uuid: "a07045b945684233bc871fbb82844387"
 from {username='slave'} at 127.0.0.1:59904
I20260709 13:09:02.319625  8244 raft_consensus.cc:493] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:02.320096  8244 raft_consensus.cc:3060] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:02.320492  8344 consensus_queue.cc:1048] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:02.322467  8094 raft_consensus.cc:1275] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Refusing update from remote peer 67bdf3c8792c41f4993f9f63ebe059b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:02.326180  8347 consensus_queue.cc:1048] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:02.343420  8244 raft_consensus.cc:515] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:02.346822  8094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971" candidate_uuid: "a07045b945684233bc871fbb82844387" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "617b49f5079e45d3934f23cfd7130d93"
I20260709 13:09:02.347514  8094 raft_consensus.cc:3055] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:02.347796  8094 raft_consensus.cc:740] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 617b49f5079e45d3934f23cfd7130d93, State: Running, Role: LEADER
I20260709 13:09:02.348758  8094 consensus_queue.cc:260] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:02.350406  8094 raft_consensus.cc:3060] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:02.352468  8244 leader_election.cc:290] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [CANDIDATE]: Term 3 election: Requested vote from peers 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129:41207), 67bdf3c8792c41f4993f9f63ebe059b2 (127.27.71.130:38887)
I20260709 13:09:02.354024  8170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17af902b039f47c597f39ee88ebaa971" candidate_uuid: "a07045b945684233bc871fbb82844387" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "67bdf3c8792c41f4993f9f63ebe059b2"
I20260709 13:09:02.355509  8170 raft_consensus.cc:3060] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:02.359692  8094 raft_consensus.cc:2468] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07045b945684233bc871fbb82844387 in term 3.
I20260709 13:09:02.361335  8209 leader_election.cc:304] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b49f5079e45d3934f23cfd7130d93, a07045b945684233bc871fbb82844387; no voters: 
I20260709 13:09:02.362107  8297 raft_consensus.cc:2804] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:02.362191  8170 raft_consensus.cc:2468] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07045b945684233bc871fbb82844387 in term 3.
I20260709 13:09:02.363121  8297 raft_consensus.cc:697] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 3 LEADER]: Becoming Leader. State: Replica: a07045b945684233bc871fbb82844387, State: Running, Role: LEADER
I20260709 13:09:02.364082  8297 consensus_queue.cc:237] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } }
I20260709 13:09:02.372521  7990 catalog_manager.cc:5697] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 reported cstate change: term changed from 2 to 3, leader changed from 617b49f5079e45d3934f23cfd7130d93 (127.27.71.129) to a07045b945684233bc871fbb82844387 (127.27.71.131). New cstate: current_term: 3 leader_uuid: "a07045b945684233bc871fbb82844387" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617b49f5079e45d3934f23cfd7130d93" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 41207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67bdf3c8792c41f4993f9f63ebe059b2" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07045b945684233bc871fbb82844387" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 37367 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:02.410800 27934 tablet_server.cc:179] TabletServer@127.27.71.129:0 shutting down...
I20260709 13:09:02.434187 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:02.434855 27934 tablet_replica.cc:333] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93: stopping tablet replica
I20260709 13:09:02.435418 27934 raft_consensus.cc:2243] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:02.435886 27934 raft_consensus.cc:2272] T 17af902b039f47c597f39ee88ebaa971 P 617b49f5079e45d3934f23cfd7130d93 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.438390 27934 tablet_replica.cc:333] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93: stopping tablet replica
I20260709 13:09:02.439020 27934 raft_consensus.cc:2243] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:02.439836 27934 raft_consensus.cc:2272] T 5092d2ec1f6447268cad1232a1e94f74 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.442584 27934 tablet_replica.cc:333] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93: stopping tablet replica
I20260709 13:09:02.443101 27934 raft_consensus.cc:2243] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:02.443976 27934 raft_consensus.cc:2272] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.446483 27934 tablet_replica.cc:333] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93: stopping tablet replica
I20260709 13:09:02.446977 27934 raft_consensus.cc:2243] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:02.447402 27934 raft_consensus.cc:2272] T d6b4038d8cb9459e94488278937c9e30 P 617b49f5079e45d3934f23cfd7130d93 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.500061 27934 tablet_server.cc:196] TabletServer@127.27.71.129:0 shutdown complete.
I20260709 13:09:02.532392 27934 tablet_server.cc:179] TabletServer@127.27.71.130:0 shutting down...
I20260709 13:09:02.559267 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:02.560286 27934 tablet_replica.cc:333] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2: stopping tablet replica
I20260709 13:09:02.561165 27934 raft_consensus.cc:2243] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:02.562007 27934 raft_consensus.cc:2272] T 17af902b039f47c597f39ee88ebaa971 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.566398 27934 tablet_replica.cc:333] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2: stopping tablet replica
I20260709 13:09:02.567005 27934 raft_consensus.cc:2243] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:02.567655 27934 raft_consensus.cc:2272] T 5092d2ec1f6447268cad1232a1e94f74 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.571669 27934 tablet_replica.cc:333] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2: stopping tablet replica
I20260709 13:09:02.572359 27934 raft_consensus.cc:2243] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:02.572989 27934 raft_consensus.cc:2272] T 3fa6cb1236584b8b8b545409fe2fb7c6 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.576654 27934 tablet_replica.cc:333] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2: stopping tablet replica
I20260709 13:09:02.577183 27934 raft_consensus.cc:2243] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:02.578692 27934 raft_consensus.cc:2272] T d6b4038d8cb9459e94488278937c9e30 P 67bdf3c8792c41f4993f9f63ebe059b2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.609184 27934 tablet_server.cc:196] TabletServer@127.27.71.130:0 shutdown complete.
I20260709 13:09:02.633999 27934 tablet_server.cc:179] TabletServer@127.27.71.131:0 shutting down...
I20260709 13:09:02.664840 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:02.665699 27934 tablet_replica.cc:333] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387: stopping tablet replica
I20260709 13:09:02.666492 27934 raft_consensus.cc:2243] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:02.667629 27934 pending_rounds.cc:70] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387: Trying to abort 1 pending ops.
I20260709 13:09:02.667858 27934 pending_rounds.cc:77] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305636013532909568 op_type: NO_OP noop_request { }
I20260709 13:09:02.668411 27934 raft_consensus.cc:2889] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:09:02.668802 27934 raft_consensus.cc:2272] T 17af902b039f47c597f39ee88ebaa971 P a07045b945684233bc871fbb82844387 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.672341 27934 tablet_replica.cc:333] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387: stopping tablet replica
I20260709 13:09:02.672897 27934 raft_consensus.cc:2243] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:02.673548 27934 raft_consensus.cc:2272] T 5092d2ec1f6447268cad1232a1e94f74 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.676972 27934 tablet_replica.cc:333] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387: stopping tablet replica
I20260709 13:09:02.677557 27934 raft_consensus.cc:2243] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:02.678210 27934 raft_consensus.cc:2272] T d6b4038d8cb9459e94488278937c9e30 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.681922 27934 tablet_replica.cc:333] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387: stopping tablet replica
I20260709 13:09:02.682538 27934 raft_consensus.cc:2243] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:02.683159 27934 raft_consensus.cc:2272] T 3fa6cb1236584b8b8b545409fe2fb7c6 P a07045b945684233bc871fbb82844387 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.711730 27934 tablet_server.cc:196] TabletServer@127.27.71.131:0 shutdown complete.
I20260709 13:09:02.741418 27934 master.cc:562] Master@127.27.71.190:45945 shutting down...
I20260709 13:09:02.769995 27934 raft_consensus.cc:2243] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:02.770776 27934 raft_consensus.cc:2272] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:02.771159 27934 tablet_replica.cc:333] T 00000000000000000000000000000000 P bdb6446b6b9d48ed9a5cb824df954423: stopping tablet replica
I20260709 13:09:02.795245 27934 master.cc:584] Master@127.27.71.190:45945 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3298 ms)
[----------] 6 tests from LeaderRebalancerTest (363182 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:09:02.845655 27934 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.71.190:37267
I20260709 13:09:02.846969 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:02.854241  8351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:02.855336  8352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:02.855446  8354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:02.857827 27934 server_base.cc:1061] running on GCE node
I20260709 13:09:02.858925 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:02.859141 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:02.859313 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602542859297 us; error 0 us; skew 500 ppm
I20260709 13:09:02.859886 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:02.862867 27934 webserver.cc:533] Webserver started at http://127.27.71.190:41183/ using document root <none> and password file <none>
I20260709 13:09:02.863454 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:02.863670 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:02.864009 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:02.865324 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/master-0-root/instance:
uuid: "62e53dabbcf34c36aba774737ee67a2e"
format_stamp: "Formatted at 2026-07-09 13:09:02 on dist-test-slave-mlt7"
I20260709 13:09:02.870568 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:09:02.874612  8359 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:02.875675 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:09:02.876044 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/master-0-root
uuid: "62e53dabbcf34c36aba774737ee67a2e"
format_stamp: "Formatted at 2026-07-09 13:09:02 on dist-test-slave-mlt7"
I20260709 13:09:02.876344 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:02.890477 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:02.891980 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:02.940032 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.190:37267
I20260709 13:09:02.940233  8410 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.190:37267 every 8 connection(s)
I20260709 13:09:02.944528  8411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:02.958463  8411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e: Bootstrap starting.
I20260709 13:09:02.964246  8411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.969393  8411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e: No bootstrap required, opened a new log
I20260709 13:09:02.972404  8411 raft_consensus.cc:359] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e53dabbcf34c36aba774737ee67a2e" member_type: VOTER }
I20260709 13:09:02.972923  8411 raft_consensus.cc:385] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.973230  8411 raft_consensus.cc:740] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e53dabbcf34c36aba774737ee67a2e, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.973979  8411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e53dabbcf34c36aba774737ee67a2e" member_type: VOTER }
I20260709 13:09:02.974615  8411 raft_consensus.cc:399] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:09:02.974862  8411 raft_consensus.cc:493] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:09:02.975206  8411 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.980890  8411 raft_consensus.cc:515] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e53dabbcf34c36aba774737ee67a2e" member_type: VOTER }
I20260709 13:09:02.981534  8411 leader_election.cc:304] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62e53dabbcf34c36aba774737ee67a2e; no voters: 
I20260709 13:09:02.982801  8411 leader_election.cc:290] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:09:02.983349  8414 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:02.984823  8414 raft_consensus.cc:697] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [term 1 LEADER]: Becoming Leader. State: Replica: 62e53dabbcf34c36aba774737ee67a2e, State: Running, Role: LEADER
I20260709 13:09:02.985623  8414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e53dabbcf34c36aba774737ee67a2e" member_type: VOTER }
I20260709 13:09:02.985903  8411 sys_catalog.cc:565] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:09:02.988897  8415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62e53dabbcf34c36aba774737ee67a2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e53dabbcf34c36aba774737ee67a2e" member_type: VOTER } }
I20260709 13:09:02.988947  8416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62e53dabbcf34c36aba774737ee67a2e. Latest consensus state: current_term: 1 leader_uuid: "62e53dabbcf34c36aba774737ee67a2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e53dabbcf34c36aba774737ee67a2e" member_type: VOTER } }
I20260709 13:09:02.989794  8415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:02.989864  8416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:02.997617  8419 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:09:03.002871  8419 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:09:03.003957 27934 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:09:03.012802  8419 catalog_manager.cc:1383] Generated new cluster ID: 37186a27ed18496aa405c6a97ceeb9cc
I20260709 13:09:03.013074  8419 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:09:03.050433  8419 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:09:03.054059  8419 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:09:03.072322  8419 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e: Generated new TSK 0
I20260709 13:09:03.072985  8419 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:09:03.135263 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:03.142220  8432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:03.143536  8433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:03.145574 27934 server_base.cc:1061] running on GCE node
W20260709 13:09:03.145533  8435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:03.146920 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:03.147153 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:03.147330 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602543147312 us; error 0 us; skew 500 ppm
I20260709 13:09:03.147971 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:03.150976 27934 webserver.cc:533] Webserver started at http://127.27.71.129:39789/ using document root <none> and password file <none>
I20260709 13:09:03.151556 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:03.151789 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:03.152114 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:03.153367 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-0-root/instance:
uuid: "3196546f9f674c63b347c608d8d7c81e"
format_stamp: "Formatted at 2026-07-09 13:09:03 on dist-test-slave-mlt7"
I20260709 13:09:03.158567 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:09:03.162341  8440 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:03.163354 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:09:03.163633 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-0-root
uuid: "3196546f9f674c63b347c608d8d7c81e"
format_stamp: "Formatted at 2026-07-09 13:09:03 on dist-test-slave-mlt7"
I20260709 13:09:03.163874 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:03.201153 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:03.203202 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:03.204943 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:03.208168 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:03.208379 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:03.208583 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:03.208725 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:03.253094 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.129:36551
I20260709 13:09:03.253209  8502 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.129:36551 every 8 connection(s)
I20260709 13:09:03.258668 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:03.267683  8507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:03.269443  8508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:03.275048  8510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:03.276062  8503 heartbeater.cc:344] Connected to a master server at 127.27.71.190:37267
I20260709 13:09:03.276568  8503 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:03.277518 27934 server_base.cc:1061] running on GCE node
I20260709 13:09:03.277650  8503 heartbeater.cc:507] Master 127.27.71.190:37267 requested a full tablet report, sending...
I20260709 13:09:03.278630 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:03.278910 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:03.279122 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602543279102 us; error 0 us; skew 500 ppm
I20260709 13:09:03.279763 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:03.280332  8376 ts_manager.cc:194] Registered new tserver with Master: 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551)
I20260709 13:09:03.282532 27934 webserver.cc:533] Webserver started at http://127.27.71.130:38359/ using document root <none> and password file <none>
I20260709 13:09:03.282783  8376 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37138
I20260709 13:09:03.283193 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:03.283496 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:03.283772 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:03.285131 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-1-root/instance:
uuid: "5ec5ac80f9fd413a957364f29ec50feb"
format_stamp: "Formatted at 2026-07-09 13:09:03 on dist-test-slave-mlt7"
I20260709 13:09:03.289822 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:09:03.294072  8515 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:03.295092 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:09:03.295372 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-1-root
uuid: "5ec5ac80f9fd413a957364f29ec50feb"
format_stamp: "Formatted at 2026-07-09 13:09:03 on dist-test-slave-mlt7"
I20260709 13:09:03.295625 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:03.320191 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:03.321415 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:03.322863 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:03.325201 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:03.325382 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:03.325680 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:03.325838 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:03.369237 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.130:45757
I20260709 13:09:03.369381  8577 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.130:45757 every 8 connection(s)
I20260709 13:09:03.374295 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:03.382210  8581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:03.384171  8582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:03.388392  8584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:03.388669  8578 heartbeater.cc:344] Connected to a master server at 127.27.71.190:37267
I20260709 13:09:03.389223  8578 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:03.389348 27934 server_base.cc:1061] running on GCE node
I20260709 13:09:03.390205  8578 heartbeater.cc:507] Master 127.27.71.190:37267 requested a full tablet report, sending...
I20260709 13:09:03.390528 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:03.390833 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:03.391026 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602543391009 us; error 0 us; skew 500 ppm
I20260709 13:09:03.391815 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:03.392588  8376 ts_manager.cc:194] Registered new tserver with Master: 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:03.394305  8376 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37142
I20260709 13:09:03.394459 27934 webserver.cc:533] Webserver started at http://127.27.71.131:37507/ using document root <none> and password file <none>
I20260709 13:09:03.395155 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:03.395401 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:03.395726 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:03.397156 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-2-root/instance:
uuid: "acb23023045448a0a9eb67cbc2a361e3"
format_stamp: "Formatted at 2026-07-09 13:09:03 on dist-test-slave-mlt7"
I20260709 13:09:03.402297 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:09:03.406147  8589 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:03.407053 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 13:09:03.407399 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-2-root
uuid: "acb23023045448a0a9eb67cbc2a361e3"
format_stamp: "Formatted at 2026-07-09 13:09:03 on dist-test-slave-mlt7"
I20260709 13:09:03.407788 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602179482400-27934-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:03.419170 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:03.420563 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:03.422117 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:03.424563 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:03.424806 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:03.425071 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:03.425251 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:03.470063 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.131:40041
I20260709 13:09:03.470176  8651 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.131:40041 every 8 connection(s)
I20260709 13:09:03.485244  8652 heartbeater.cc:344] Connected to a master server at 127.27.71.190:37267
I20260709 13:09:03.485780  8652 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:03.486577  8652 heartbeater.cc:507] Master 127.27.71.190:37267 requested a full tablet report, sending...
I20260709 13:09:03.488816  8376 ts_manager.cc:194] Registered new tserver with Master: acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:03.489364 27934 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016122639s
I20260709 13:09:03.490289  8376 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37146
I20260709 13:09:03.519083  8376 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37162:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:03.522092  8376 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:03.609622  8615 tablet_service.cc:1511] Processing CreateTablet for tablet 0befb22a314d4548b054ecae7a41f369 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:03.610002  8617 tablet_service.cc:1511] Processing CreateTablet for tablet 1704f9c82d24421cb4444772e150092d (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:03.611248  8615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0befb22a314d4548b054ecae7a41f369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.612433  8616 tablet_service.cc:1511] Processing CreateTablet for tablet 0906217e5d2b4ab08f5d25f61dd28cb5 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:03.613896  8616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0906217e5d2b4ab08f5d25f61dd28cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.617154  8613 tablet_service.cc:1511] Processing CreateTablet for tablet be2c3b332154426388d8e090ce588f2f (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:03.618696  8613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be2c3b332154426388d8e090ce588f2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.621223  8612 tablet_service.cc:1511] Processing CreateTablet for tablet c1c56c3b74ac4b26b51002ad7f31adc1 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:03.622771  8612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c56c3b74ac4b26b51002ad7f31adc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.624472  8614 tablet_service.cc:1511] Processing CreateTablet for tablet f8851f8110794e71b484de5b98a34b51 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:03.624806  8611 tablet_service.cc:1511] Processing CreateTablet for tablet 847a35bccaa640e1b0c3f4c26aac3793 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:03.626014  8614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8851f8110794e71b484de5b98a34b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.626768  8611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847a35bccaa640e1b0c3f4c26aac3793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.630046  8468 tablet_service.cc:1511] Processing CreateTablet for tablet 1704f9c82d24421cb4444772e150092d (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:03.631507  8468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1704f9c82d24421cb4444772e150092d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.636693  8610 tablet_service.cc:1511] Processing CreateTablet for tablet 715c5d36abd94180a1febc2638238061 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:03.636833  8466 tablet_service.cc:1511] Processing CreateTablet for tablet 0befb22a314d4548b054ecae7a41f369 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:03.638451  8466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0befb22a314d4548b054ecae7a41f369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.633149  8467 tablet_service.cc:1511] Processing CreateTablet for tablet 0906217e5d2b4ab08f5d25f61dd28cb5 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:03.640964  8467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0906217e5d2b4ab08f5d25f61dd28cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.644088  8617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1704f9c82d24421cb4444772e150092d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.644842  8610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 715c5d36abd94180a1febc2638238061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.649624  8465 tablet_service.cc:1511] Processing CreateTablet for tablet f8851f8110794e71b484de5b98a34b51 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:03.651170  8465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8851f8110794e71b484de5b98a34b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.652567  8609 tablet_service.cc:1511] Processing CreateTablet for tablet ecb95cc556024fbf86a589163f93093d (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:03.654070  8609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecb95cc556024fbf86a589163f93093d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.653714  8464 tablet_service.cc:1511] Processing CreateTablet for tablet be2c3b332154426388d8e090ce588f2f (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:03.655134  8464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be2c3b332154426388d8e090ce588f2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.657734  8463 tablet_service.cc:1511] Processing CreateTablet for tablet c1c56c3b74ac4b26b51002ad7f31adc1 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:03.659171  8463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c56c3b74ac4b26b51002ad7f31adc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.666177  8462 tablet_service.cc:1511] Processing CreateTablet for tablet 847a35bccaa640e1b0c3f4c26aac3793 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:03.666893  8461 tablet_service.cc:1511] Processing CreateTablet for tablet 715c5d36abd94180a1febc2638238061 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:03.669193  8460 tablet_service.cc:1511] Processing CreateTablet for tablet ecb95cc556024fbf86a589163f93093d (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:03.670601  8461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 715c5d36abd94180a1febc2638238061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.671360  8460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecb95cc556024fbf86a589163f93093d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.672561  8462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847a35bccaa640e1b0c3f4c26aac3793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.679603  8543 tablet_service.cc:1511] Processing CreateTablet for tablet 1704f9c82d24421cb4444772e150092d (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:03.681114  8543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1704f9c82d24421cb4444772e150092d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.681566  8667 tablet_bootstrap.cc:492] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:03.684710  8542 tablet_service.cc:1511] Processing CreateTablet for tablet 0906217e5d2b4ab08f5d25f61dd28cb5 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:03.686266  8542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0906217e5d2b4ab08f5d25f61dd28cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.689087  8667 tablet_bootstrap.cc:654] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.694819  8667 tablet_bootstrap.cc:492] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:03.695333  8667 ts_tablet_manager.cc:1403] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.014s	user 0.004s	sys 0.009s
I20260709 13:09:03.697780  8541 tablet_service.cc:1511] Processing CreateTablet for tablet 0befb22a314d4548b054ecae7a41f369 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:03.699404  8541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0befb22a314d4548b054ecae7a41f369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.699050  8667 raft_consensus.cc:359] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.700161  8667 raft_consensus.cc:385] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:03.700574  8667 raft_consensus.cc:740] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:03.701866  8667 consensus_queue.cc:260] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.704847  8669 tablet_bootstrap.cc:492] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:03.709249  8540 tablet_service.cc:1511] Processing CreateTablet for tablet f8851f8110794e71b484de5b98a34b51 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:03.710801  8540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8851f8110794e71b484de5b98a34b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.714561  8652 heartbeater.cc:499] Master 127.27.71.190:37267 was elected leader, sending a full tablet report...
I20260709 13:09:03.715384  8669 tablet_bootstrap.cc:654] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.715700  8667 ts_tablet_manager.cc:1434] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.007s
I20260709 13:09:03.716859  8667 tablet_bootstrap.cc:492] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:03.720376  8539 tablet_service.cc:1511] Processing CreateTablet for tablet be2c3b332154426388d8e090ce588f2f (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:03.721832  8539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be2c3b332154426388d8e090ce588f2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.734887  8538 tablet_service.cc:1511] Processing CreateTablet for tablet c1c56c3b74ac4b26b51002ad7f31adc1 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:03.736703  8538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c56c3b74ac4b26b51002ad7f31adc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.754864  8540 tablet_service.cc:1511] Processing CreateTablet for tablet 847a35bccaa640e1b0c3f4c26aac3793 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:03.756567  8540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847a35bccaa640e1b0c3f4c26aac3793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.757182  8537 tablet_service.cc:1511] Processing CreateTablet for tablet 715c5d36abd94180a1febc2638238061 (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:03.758715  8537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 715c5d36abd94180a1febc2638238061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.761355  8536 tablet_service.cc:1511] Processing CreateTablet for tablet ecb95cc556024fbf86a589163f93093d (DEFAULT_TABLE table=test-workload [id=f8e81c759d1e4de68e6aecb45d124793]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:03.762897  8536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecb95cc556024fbf86a589163f93093d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:03.761694  8667 tablet_bootstrap.cc:654] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.803669  8672 tablet_bootstrap.cc:492] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:03.811523  8672 tablet_bootstrap.cc:654] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.854633  8669 tablet_bootstrap.cc:492] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:03.855180  8669 ts_tablet_manager.cc:1403] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.151s	user 0.029s	sys 0.013s
I20260709 13:09:03.858460  8669 raft_consensus.cc:359] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:03.859205  8669 raft_consensus.cc:385] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:03.859525  8669 raft_consensus.cc:740] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:03.860220  8672 tablet_bootstrap.cc:492] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:03.861038  8672 ts_tablet_manager.cc:1403] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.010s
I20260709 13:09:03.861202  8669 consensus_queue.cc:260] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:03.864336  8667 tablet_bootstrap.cc:492] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:03.865106  8667 ts_tablet_manager.cc:1403] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.149s	user 0.011s	sys 0.021s
I20260709 13:09:03.864611  8672 raft_consensus.cc:359] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.865604  8672 raft_consensus.cc:385] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:03.866065  8672 raft_consensus.cc:740] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:03.867147  8672 consensus_queue.cc:260] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.868813  8667 raft_consensus.cc:359] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.870033  8667 raft_consensus.cc:385] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:03.870471  8667 raft_consensus.cc:740] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:03.870589  8503 heartbeater.cc:499] Master 127.27.71.190:37267 was elected leader, sending a full tablet report...
I20260709 13:09:03.871773  8669 ts_tablet_manager.cc:1434] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.005s
I20260709 13:09:03.871937  8667 consensus_queue.cc:260] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.872887  8669 tablet_bootstrap.cc:492] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:03.875263  8667 ts_tablet_manager.cc:1434] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.006s
I20260709 13:09:03.876520  8667 tablet_bootstrap.cc:492] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:03.876806  8578 heartbeater.cc:499] Master 127.27.71.190:37267 was elected leader, sending a full tablet report...
I20260709 13:09:03.877203  8672 ts_tablet_manager.cc:1434] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.005s
I20260709 13:09:03.878575  8672 tablet_bootstrap.cc:492] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:03.882728  8669 tablet_bootstrap.cc:654] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.884267  8667 tablet_bootstrap.cc:654] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.886523  8672 tablet_bootstrap.cc:654] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.896715  8669 tablet_bootstrap.cc:492] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:03.897343  8669 ts_tablet_manager.cc:1403] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.025s	user 0.004s	sys 0.015s
I20260709 13:09:03.900849  8669 raft_consensus.cc:359] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.901630  8669 raft_consensus.cc:385] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:03.901927  8669 raft_consensus.cc:740] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:03.902843  8669 consensus_queue.cc:260] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.905458  8669 ts_tablet_manager.cc:1434] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:09:03.906451  8669 tablet_bootstrap.cc:492] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:03.913139  8669 tablet_bootstrap.cc:654] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.924990  8669 tablet_bootstrap.cc:492] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:03.925571  8669 ts_tablet_manager.cc:1403] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 13:09:03.928896  8669 raft_consensus.cc:359] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.929710  8669 raft_consensus.cc:385] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:03.930001  8669 raft_consensus.cc:740] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:03.930948  8669 consensus_queue.cc:260] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.933460  8669 ts_tablet_manager.cc:1434] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:09:03.934535  8669 tablet_bootstrap.cc:492] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:03.941210  8669 tablet_bootstrap.cc:654] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.952994  8669 tablet_bootstrap.cc:492] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:03.953528  8669 ts_tablet_manager.cc:1403] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 13:09:03.956908  8669 raft_consensus.cc:359] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.957662  8669 raft_consensus.cc:385] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:03.957979  8669 raft_consensus.cc:740] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:03.958976  8669 consensus_queue.cc:260] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.960265  8670 raft_consensus.cc:493] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.960901  8670 raft_consensus.cc:515] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.961493  8672 tablet_bootstrap.cc:492] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:03.962110  8672 ts_tablet_manager.cc:1403] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.084s	user 0.026s	sys 0.055s
I20260709 13:09:03.963129  8669 ts_tablet_manager.cc:1434] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:09:03.964285  8669 tablet_bootstrap.cc:492] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:03.966426  8672 raft_consensus.cc:359] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.967334  8672 raft_consensus.cc:385] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:03.967660  8672 raft_consensus.cc:740] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:03.968713  8672 consensus_queue.cc:260] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.972754  8669 tablet_bootstrap.cc:654] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.974289  8672 ts_tablet_manager.cc:1434] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 13:09:03.974390  8670 leader_election.cc:290] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:03.975664  8672 tablet_bootstrap.cc:492] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:03.981125  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb95cc556024fbf86a589163f93093d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:03.982240  8478 raft_consensus.cc:2468] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:03.985211  8672 tablet_bootstrap.cc:654] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.991483  8669 tablet_bootstrap.cc:492] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:03.992081  8669 ts_tablet_manager.cc:1403] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.006s
I20260709 13:09:03.995222  8669 raft_consensus.cc:359] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:03.995872  8669 raft_consensus.cc:385] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:09:03.998526  8424 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:03.996234  8669 raft_consensus.cc:740] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:03.999372  8672 tablet_bootstrap.cc:492] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:04.000288  8672 ts_tablet_manager.cc:1403] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.009s
I20260709 13:09:03.999401  8667 tablet_bootstrap.cc:492] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:04.000113  8669 consensus_queue.cc:260] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.000985  8667 ts_tablet_manager.cc:1403] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.125s	user 0.025s	sys 0.059s
I20260709 13:09:04.002928  8669 ts_tablet_manager.cc:1434] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:09:04.003731  8672 raft_consensus.cc:359] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.004637  8669 tablet_bootstrap.cc:492] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:04.004676  8672 raft_consensus.cc:385] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.005400  8672 raft_consensus.cc:740] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.006467  8672 consensus_queue.cc:260] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.009153  8592 leader_election.cc:304] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:04.010269  8670 raft_consensus.cc:2804] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:04.010783  8670 raft_consensus.cc:493] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:04.011163  8670 raft_consensus.cc:3060] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.012722  8672 ts_tablet_manager.cc:1434] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:09:04.013352  8669 tablet_bootstrap.cc:654] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.014050  8672 tablet_bootstrap.cc:492] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:04.022434  8672 tablet_bootstrap.cc:654] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.024878  8669 tablet_bootstrap.cc:492] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:04.025465  8669 ts_tablet_manager.cc:1403] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 13:09:04.028342  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb95cc556024fbf86a589163f93093d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:04.028637  8669 raft_consensus.cc:359] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.029522  8669 raft_consensus.cc:385] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.029912  8669 raft_consensus.cc:740] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.030979  8669 consensus_queue.cc:260] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
W20260709 13:09:04.031123  8592 leader_election.cc:343] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:04.033025  8667 raft_consensus.cc:359] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.034803  8667 raft_consensus.cc:385] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.035400  8667 raft_consensus.cc:740] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.034190  8669 ts_tablet_manager.cc:1434] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:04.035986  8676 raft_consensus.cc:493] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.036770  8667 consensus_queue.cc:260] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.036890  8676 raft_consensus.cc:515] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.037757  8669 tablet_bootstrap.cc:492] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:04.023689  8670 raft_consensus.cc:515] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.043319  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb95cc556024fbf86a589163f93093d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:04.044217  8478 raft_consensus.cc:3060] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.048808  8669 tablet_bootstrap.cc:654] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.055855  8676 leader_election.cc:290] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:04.057292  8670 leader_election.cc:290] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:04.061591  8667 ts_tablet_manager.cc:1434] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.001s
I20260709 13:09:04.063426  8672 tablet_bootstrap.cc:492] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:04.061702  8669 tablet_bootstrap.cc:492] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:04.064116  8672 ts_tablet_manager.cc:1403] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.050s	user 0.008s	sys 0.016s
I20260709 13:09:04.064400  8669 ts_tablet_manager.cc:1403] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.027s	user 0.023s	sys 0.001s
I20260709 13:09:04.062867  8667 tablet_bootstrap.cc:492] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:04.070040  8675 raft_consensus.cc:493] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.070155  8672 raft_consensus.cc:359] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.076341  8672 raft_consensus.cc:385] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.071369  8675 raft_consensus.cc:515] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.072813  8478 raft_consensus.cc:2468] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:04.077972  8672 raft_consensus.cc:740] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.081962  8592 leader_election.cc:304] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:04.082579  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb95cc556024fbf86a589163f93093d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:04.085665  8672 consensus_queue.cc:260] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.087218  8675 leader_election.cc:290] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:04.090543  8672 ts_tablet_manager.cc:1434] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.003s
I20260709 13:09:04.091905  8672 tablet_bootstrap.cc:492] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:04.095534  8667 tablet_bootstrap.cc:654] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.083218  8669 raft_consensus.cc:359] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.097394  8669 raft_consensus.cc:385] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.097808  8669 raft_consensus.cc:740] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.098702  8669 consensus_queue.cc:260] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.100245  8672 tablet_bootstrap.cc:654] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.106873  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0befb22a314d4548b054ecae7a41f369" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:04.110167  8478 raft_consensus.cc:2468] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 0.
I20260709 13:09:04.112514  8518 leader_election.cc:304] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:04.113243  8669 ts_tablet_manager.cc:1434] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.002s
I20260709 13:09:04.113632  8676 raft_consensus.cc:2804] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:04.114070  8676 raft_consensus.cc:493] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:04.114454  8676 raft_consensus.cc:3060] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.114567  8669 tablet_bootstrap.cc:492] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:04.121701  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8851f8110794e71b484de5b98a34b51" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:04.122500  8553 raft_consensus.cc:2468] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 0.
I20260709 13:09:04.123907  8443 leader_election.cc:304] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:04.123996  8676 raft_consensus.cc:515] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.125008  8675 raft_consensus.cc:2804] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:04.125393  8675 raft_consensus.cc:493] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:04.125746  8675 raft_consensus.cc:3060] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.128052  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0befb22a314d4548b054ecae7a41f369" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:04.128877  8478 raft_consensus.cc:3060] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.129048  8676 leader_election.cc:290] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:04.130389  8687 raft_consensus.cc:2804] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:04.132221  8687 raft_consensus.cc:697] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:04.136924  8675 raft_consensus.cc:515] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.138410  8687 consensus_queue.cc:237] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.144632  8675 leader_election.cc:290] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Requested vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:04.145385  8675 raft_consensus.cc:493] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.145972  8675 raft_consensus.cc:515] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
W20260709 13:09:04.147099  8592 leader_election.cc:343] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:04.153486  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8851f8110794e71b484de5b98a34b51" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:04.154316  8553 raft_consensus.cc:3060] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.154148  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0befb22a314d4548b054ecae7a41f369" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:04.154721  8675 leader_election.cc:290] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:04.155365  8626 raft_consensus.cc:3060] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.156543  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0befb22a314d4548b054ecae7a41f369" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:04.157840  8624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8851f8110794e71b484de5b98a34b51" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:04.160425  8625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8851f8110794e71b484de5b98a34b51" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
W20260709 13:09:04.160595  8442 leader_election.cc:343] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:04.161156  8667 tablet_bootstrap.cc:492] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:04.161737  8667 ts_tablet_manager.cc:1403] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.099s	user 0.013s	sys 0.002s
I20260709 13:09:04.158182  8623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1704f9c82d24421cb4444772e150092d" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:04.163053  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1704f9c82d24421cb4444772e150092d" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
W20260709 13:09:04.164706  8443 leader_election.cc:343] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:04.165099  8442 leader_election.cc:343] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:04.166405  8442 leader_election.cc:343] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:04.167039  8442 leader_election.cc:304] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e; no voters: 5ec5ac80f9fd413a957364f29ec50feb, acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:04.168114  8667 raft_consensus.cc:359] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.168955  8667 raft_consensus.cc:385] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.169279  8667 raft_consensus.cc:740] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.170703  8667 consensus_queue.cc:260] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.171499  8626 raft_consensus.cc:2468] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 1.
I20260709 13:09:04.173431  8517 leader_election.cc:304] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:04.174273  8675 raft_consensus.cc:2749] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:04.174404  8676 raft_consensus.cc:2804] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:04.174995  8667 ts_tablet_manager.cc:1434] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:09:04.189697  8667 tablet_bootstrap.cc:492] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:04.195678  8553 raft_consensus.cc:2468] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 1.
I20260709 13:09:04.188089  8478 raft_consensus.cc:2468] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 1.
I20260709 13:09:04.197587  8443 leader_election.cc:304] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:04.198972  8675 raft_consensus.cc:2804] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:04.199632  8675 raft_consensus.cc:697] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Becoming Leader. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Running, Role: LEADER
I20260709 13:09:04.199131  8376 catalog_manager.cc:5697] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:04.200631  8675 consensus_queue.cc:237] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.196691  8669 tablet_bootstrap.cc:654] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.205353  8676 raft_consensus.cc:697] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Becoming Leader. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:04.209710  8676 consensus_queue.cc:237] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.212319  8667 tablet_bootstrap.cc:654] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.214625  8373 catalog_manager.cc:5697] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3196546f9f674c63b347c608d8d7c81e (127.27.71.129). New cstate: current_term: 1 leader_uuid: "3196546f9f674c63b347c608d8d7c81e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:04.215977  8672 tablet_bootstrap.cc:492] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:04.216562  8672 ts_tablet_manager.cc:1403] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.125s	user 0.029s	sys 0.035s
I20260709 13:09:04.218216  8698 raft_consensus.cc:493] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.219249  8698 raft_consensus.cc:515] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.221109  8672 raft_consensus.cc:359] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:04.222236  8672 raft_consensus.cc:385] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.222641  8672 raft_consensus.cc:740] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.224117  8672 consensus_queue.cc:260] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:04.234696  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847a35bccaa640e1b0c3f4c26aac3793" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:04.235567  8626 raft_consensus.cc:2468] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 0.
I20260709 13:09:04.236500  8374 catalog_manager.cc:5697] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130). New cstate: current_term: 1 leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:04.239977  8517 leader_election.cc:304] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:04.243144  8698 leader_election.cc:290] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:04.243511  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847a35bccaa640e1b0c3f4c26aac3793" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:04.244104  8698 raft_consensus.cc:2804] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:04.244686  8478 raft_consensus.cc:2468] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 0.
I20260709 13:09:04.244700  8698 raft_consensus.cc:493] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:04.248095  8698 raft_consensus.cc:3060] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.253551  8667 tablet_bootstrap.cc:492] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:04.254272  8667 ts_tablet_manager.cc:1403] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.066s	user 0.006s	sys 0.015s
I20260709 13:09:04.257848  8672 ts_tablet_manager.cc:1434] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.041s	user 0.014s	sys 0.004s
I20260709 13:09:04.257990  8667 raft_consensus.cc:359] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.259167  8667 raft_consensus.cc:385] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.259536  8667 raft_consensus.cc:740] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.260066  8672 tablet_bootstrap.cc:492] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:04.261583  8669 tablet_bootstrap.cc:492] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:04.262197  8669 ts_tablet_manager.cc:1403] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.148s	user 0.022s	sys 0.012s
I20260709 13:09:04.265643  8669 raft_consensus.cc:359] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.266405  8669 raft_consensus.cc:385] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.266723  8669 raft_consensus.cc:740] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.267906  8667 consensus_queue.cc:260] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.269863  8675 raft_consensus.cc:493] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.270444  8675 raft_consensus.cc:515] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.271400  8672 tablet_bootstrap.cc:654] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.272902  8675 leader_election.cc:290] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:04.273833  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0906217e5d2b4ab08f5d25f61dd28cb5" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:04.274636  8553 raft_consensus.cc:2468] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 0.
I20260709 13:09:04.275048  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0906217e5d2b4ab08f5d25f61dd28cb5" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:04.275782  8626 raft_consensus.cc:2468] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 0.
I20260709 13:09:04.276167  8443 leader_election.cc:304] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:04.277917  8675 raft_consensus.cc:2804] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:04.278466  8675 raft_consensus.cc:493] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:04.278847  8675 raft_consensus.cc:3060] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.281486  8698 raft_consensus.cc:515] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.284806  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847a35bccaa640e1b0c3f4c26aac3793" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:04.285470  8478 raft_consensus.cc:3060] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.269325  8669 consensus_queue.cc:260] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.286841  8696 raft_consensus.cc:493] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.287333  8696 raft_consensus.cc:515] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.294350  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c56c3b74ac4b26b51002ad7f31adc1" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:04.295688  8478 raft_consensus.cc:2468] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 1.
W20260709 13:09:04.295981  8443 leader_election.cc:343] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:04.297147  8518 leader_election.cc:304] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:04.298287  8705 raft_consensus.cc:2804] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:04.298599  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c56c3b74ac4b26b51002ad7f31adc1" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:04.299012  8705 raft_consensus.cc:697] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Becoming Leader. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:04.285526  8667 ts_tablet_manager.cc:1434] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.006s
W20260709 13:09:04.300280  8442 leader_election.cc:343] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:04.300231  8705 consensus_queue.cc:237] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.301003  8442 leader_election.cc:304] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e; no voters: 5ec5ac80f9fd413a957364f29ec50feb, acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:04.302665  8698 leader_election.cc:290] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:04.302788  8696 leader_election.cc:290] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:04.303752  8696 raft_consensus.cc:2749] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:04.303994  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847a35bccaa640e1b0c3f4c26aac3793" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:04.304700  8626 raft_consensus.cc:3060] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.308511  8669 ts_tablet_manager.cc:1434] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.001s
I20260709 13:09:04.301611  8667 tablet_bootstrap.cc:492] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:04.311249  8672 tablet_bootstrap.cc:492] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:04.317363  8672 ts_tablet_manager.cc:1403] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.058s	user 0.023s	sys 0.016s
I20260709 13:09:04.319792  8672 raft_consensus.cc:359] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.320482  8672 raft_consensus.cc:385] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.320748  8672 raft_consensus.cc:740] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.321403  8672 consensus_queue.cc:260] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.322532  8667 tablet_bootstrap.cc:654] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.323082  8672 ts_tablet_manager.cc:1434] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:09:04.314510  8626 raft_consensus.cc:2468] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 1.
I20260709 13:09:04.315512  8675 raft_consensus.cc:515] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.316350  8669 tablet_bootstrap.cc:492] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:04.326251  8675 leader_election.cc:290] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Requested vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:04.326445  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0906217e5d2b4ab08f5d25f61dd28cb5" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:04.327196  8553 raft_consensus.cc:3060] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.324035  8672 tablet_bootstrap.cc:492] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:04.335157  8669 tablet_bootstrap.cc:654] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.335237  8672 tablet_bootstrap.cc:654] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.336334  8553 raft_consensus.cc:2468] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 1.
I20260709 13:09:04.337759  8443 leader_election.cc:304] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:04.338725  8675 raft_consensus.cc:2804] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:04.339249  8675 raft_consensus.cc:697] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Becoming Leader. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Running, Role: LEADER
I20260709 13:09:04.339654  8705 raft_consensus.cc:493] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.339486  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0906217e5d2b4ab08f5d25f61dd28cb5" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:04.340222  8705 raft_consensus.cc:515] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:04.340718  8626 raft_consensus.cc:3060] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.340261  8675 consensus_queue.cc:237] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.343252  8623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1704f9c82d24421cb4444772e150092d" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:04.344549  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1704f9c82d24421cb4444772e150092d" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
W20260709 13:09:04.344947  8517 leader_election.cc:343] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:04.345263  8478 raft_consensus.cc:2468] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 0.
I20260709 13:09:04.346653  8518 leader_election.cc:304] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:04.347568  8698 raft_consensus.cc:2804] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:04.347987  8698 raft_consensus.cc:493] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:04.348336  8698 raft_consensus.cc:3060] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.350744  8626 raft_consensus.cc:2468] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 1.
I20260709 13:09:04.351601  8374 catalog_manager.cc:5697] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130). New cstate: current_term: 1 leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:04.356031  8705 leader_election.cc:290] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041), 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551)
I20260709 13:09:04.357187  8698 raft_consensus.cc:515] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:04.361577  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1704f9c82d24421cb4444772e150092d" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:04.362386  8478 raft_consensus.cc:3060] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.362669  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1704f9c82d24421cb4444772e150092d" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
W20260709 13:09:04.364233  8517 leader_election.cc:343] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:04.366772  8373 catalog_manager.cc:5697] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3196546f9f674c63b347c608d8d7c81e (127.27.71.129). New cstate: current_term: 1 leader_uuid: "3196546f9f674c63b347c608d8d7c81e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:04.370780  8667 tablet_bootstrap.cc:492] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:04.371372  8667 ts_tablet_manager.cc:1403] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.070s	user 0.014s	sys 0.006s
I20260709 13:09:04.372104  8698 leader_election.cc:290] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 election: Requested vote from peers acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041), 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551)
I20260709 13:09:04.373350  8478 raft_consensus.cc:2468] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 1.
I20260709 13:09:04.374191  8667 raft_consensus.cc:359] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:04.375092  8667 raft_consensus.cc:385] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.374876  8518 leader_election.cc:304] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:04.375443  8667 raft_consensus.cc:740] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.376822  8669 tablet_bootstrap.cc:492] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:04.376478  8667 consensus_queue.cc:260] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:04.377313  8669 ts_tablet_manager.cc:1403] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.001s
I20260709 13:09:04.379709  8667 ts_tablet_manager.cc:1434] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:09:04.379767  8672 tablet_bootstrap.cc:492] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:04.381110  8667 tablet_bootstrap.cc:492] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:04.381315  8672 ts_tablet_manager.cc:1403] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.009s
I20260709 13:09:04.382230  8698 raft_consensus.cc:2804] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:04.384610  8672 raft_consensus.cc:359] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.380326  8669 raft_consensus.cc:359] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.385314  8698 raft_consensus.cc:697] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Becoming Leader. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:04.385557  8669 raft_consensus.cc:385] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.386608  8669 raft_consensus.cc:740] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.387229  8698 consensus_queue.cc:237] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:04.385425  8672 raft_consensus.cc:385] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.388412  8672 raft_consensus.cc:740] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.387578  8669 consensus_queue.cc:260] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.390027  8667 tablet_bootstrap.cc:654] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.390972  8669 ts_tablet_manager.cc:1434] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 13:09:04.391548  8672 consensus_queue.cc:260] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.394063  8672 ts_tablet_manager.cc:1434] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:09:04.395133  8672 tablet_bootstrap.cc:492] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:04.399746  8667 tablet_bootstrap.cc:492] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:04.400440  8667 ts_tablet_manager.cc:1403] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.002s
I20260709 13:09:04.403827  8672 tablet_bootstrap.cc:654] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.404301  8667 raft_consensus.cc:359] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.405246  8667 raft_consensus.cc:385] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.405629  8667 raft_consensus.cc:740] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.406630  8667 consensus_queue.cc:260] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.403375  8373 catalog_manager.cc:5697] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130). New cstate: current_term: 1 leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:04.409798  8667 ts_tablet_manager.cc:1434] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 13:09:04.411008  8667 tablet_bootstrap.cc:492] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:04.417383  8672 tablet_bootstrap.cc:492] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:04.417953  8672 ts_tablet_manager.cc:1403] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.000s
I20260709 13:09:04.418812  8667 tablet_bootstrap.cc:654] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.420715  8672 raft_consensus.cc:359] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.421480  8672 raft_consensus.cc:385] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.421748  8672 raft_consensus.cc:740] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.422494  8672 consensus_queue.cc:260] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.424674  8667 tablet_bootstrap.cc:492] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:04.424715  8672 ts_tablet_manager.cc:1434] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:09:04.425403  8667 ts_tablet_manager.cc:1403] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 13:09:04.425957  8672 tablet_bootstrap.cc:492] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:04.428519  8667 raft_consensus.cc:359] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.429257  8667 raft_consensus.cc:385] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.429591  8667 raft_consensus.cc:740] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.430373  8667 consensus_queue.cc:260] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.433048  8667 ts_tablet_manager.cc:1434] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:09:04.433250  8672 tablet_bootstrap.cc:654] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.434120  8667 tablet_bootstrap.cc:492] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:04.438850  8672 tablet_bootstrap.cc:492] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:04.439433  8672 ts_tablet_manager.cc:1403] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:09:04.440871  8667 tablet_bootstrap.cc:654] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.441828  8672 raft_consensus.cc:359] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.442378  8672 raft_consensus.cc:385] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.442605  8672 raft_consensus.cc:740] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.443213  8672 consensus_queue.cc:260] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.445180  8672 ts_tablet_manager.cc:1434] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:09:04.450501  8667 tablet_bootstrap.cc:492] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:04.450708  8687 raft_consensus.cc:493] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.451063  8667 ts_tablet_manager.cc:1403] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.006s
I20260709 13:09:04.451290  8687 raft_consensus.cc:515] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.453658  8687 leader_election.cc:290] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:04.454902  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be2c3b332154426388d8e090ce588f2f" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:04.454902  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be2c3b332154426388d8e090ce588f2f" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:04.454267  8667 raft_consensus.cc:359] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.455973  8553 raft_consensus.cc:2468] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:04.456013  8478 raft_consensus.cc:2468] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:04.456415  8667 raft_consensus.cc:385] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.456983  8667 raft_consensus.cc:740] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.457422  8592 leader_election.cc:304] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:04.457785  8667 consensus_queue.cc:260] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.458295  8687 raft_consensus.cc:2804] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:04.458770  8687 raft_consensus.cc:493] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:04.459236  8687 raft_consensus.cc:3060] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.460034  8667 ts_tablet_manager.cc:1434] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:09:04.466316  8687 raft_consensus.cc:515] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.468745  8687 leader_election.cc:290] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:04.469681  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be2c3b332154426388d8e090ce588f2f" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:04.470119  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be2c3b332154426388d8e090ce588f2f" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:04.470453  8478 raft_consensus.cc:3060] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.470796  8553 raft_consensus.cc:3060] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.478965  8478 raft_consensus.cc:2468] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:04.478969  8553 raft_consensus.cc:2468] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:04.480566  8592 leader_election.cc:304] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:04.481336  8687 raft_consensus.cc:2804] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:04.481882  8687 raft_consensus.cc:697] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:04.482805  8687 consensus_queue.cc:237] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.490904  8373 catalog_manager.cc:5697] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:04.540819  8687 consensus_queue.cc:1048] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.552964  8553 raft_consensus.cc:3060] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.561909  8670 consensus_queue.cc:1048] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.597407  8670 raft_consensus.cc:493] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.598059  8670 raft_consensus.cc:515] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.600260  8670 leader_election.cc:290] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:04.601186  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "715c5d36abd94180a1febc2638238061" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:04.601500  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "715c5d36abd94180a1febc2638238061" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:04.602073  8478 raft_consensus.cc:2468] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:04.602221  8553 raft_consensus.cc:2468] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:04.603559  8592 leader_election.cc:304] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:04.604748  8670 raft_consensus.cc:2804] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:04.605206  8670 raft_consensus.cc:493] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:04.605531  8670 raft_consensus.cc:3060] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.607833  8724 raft_consensus.cc:493] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.608428  8724 raft_consensus.cc:515] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.610147  8724 leader_election.cc:290] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:04.611054  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c56c3b74ac4b26b51002ad7f31adc1" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:04.611302  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c56c3b74ac4b26b51002ad7f31adc1" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:04.611811  8478 raft_consensus.cc:2468] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:04.612061  8553 raft_consensus.cc:2468] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:04.612048  8670 raft_consensus.cc:515] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.613348  8592 leader_election.cc:304] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:04.614190  8724 raft_consensus.cc:2804] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:04.614500  8670 leader_election.cc:290] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:04.614619  8724 raft_consensus.cc:493] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:04.615091  8724 raft_consensus.cc:3060] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.615204  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "715c5d36abd94180a1febc2638238061" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:04.615839  8478 raft_consensus.cc:3060] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.616139  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "715c5d36abd94180a1febc2638238061" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:04.616786  8553 raft_consensus.cc:3060] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.621659  8724 raft_consensus.cc:515] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.623726  8724 leader_election.cc:290] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:04.623805  8478 raft_consensus.cc:2468] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:04.624629  8553 raft_consensus.cc:2468] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:04.625097  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c56c3b74ac4b26b51002ad7f31adc1" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:04.625252  8592 leader_election.cc:304] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:04.625692  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c56c3b74ac4b26b51002ad7f31adc1" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:04.625859  8477 raft_consensus.cc:3060] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.626221  8724 raft_consensus.cc:2804] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:04.626338  8552 raft_consensus.cc:3060] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.627024  8724 raft_consensus.cc:697] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:04.628080  8724 consensus_queue.cc:237] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.635739  8477 raft_consensus.cc:2468] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:04.635878  8552 raft_consensus.cc:2468] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:04.637368  8592 leader_election.cc:304] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:04.638116  8724 raft_consensus.cc:2804] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:04.638652  8724 raft_consensus.cc:697] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:04.639354  8724 consensus_queue.cc:237] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.639490  8374 catalog_manager.cc:5697] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:04.654181  8374 catalog_manager.cc:5697] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:04.654848  8724 raft_consensus.cc:493] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.655496  8724 raft_consensus.cc:515] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:04.657897  8724 leader_election.cc:290] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:04.659324  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1704f9c82d24421cb4444772e150092d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:04.659390  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1704f9c82d24421cb4444772e150092d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:04.660315  8477 raft_consensus.cc:2393] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acb23023045448a0a9eb67cbc2a361e3 in current term 1: Already voted for candidate 5ec5ac80f9fd413a957364f29ec50feb in this term.
I20260709 13:09:04.661329  8592 leader_election.cc:304] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acb23023045448a0a9eb67cbc2a361e3; no voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb
I20260709 13:09:04.662175  8724 raft_consensus.cc:3060] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.669269  8724 raft_consensus.cc:2749] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:04.690236  8698 consensus_queue.cc:1048] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:04.695358  8698 consensus_queue.cc:1048] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.705466  8698 consensus_queue.cc:1048] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.718680  8705 consensus_queue.cc:1048] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:04.746845  8626 raft_consensus.cc:3060] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.756776  8675 consensus_queue.cc:1048] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.768281  8552 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1704f9c82d24421cb4444772e150092d"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
mode: GRACEFUL
new_leader_uuid: "3196546f9f674c63b347c608d8d7c81e"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:09:04.769143  8552 raft_consensus.cc:606] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Received request to transfer leadership to TS 3196546f9f674c63b347c608d8d7c81e
I20260709 13:09:04.770668  8696 consensus_queue.cc:1048] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.781584  8552 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0befb22a314d4548b054ecae7a41f369"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
mode: GRACEFUL
new_leader_uuid: "3196546f9f674c63b347c608d8d7c81e"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:09:04.782232  8552 raft_consensus.cc:606] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Received request to transfer leadership to TS 3196546f9f674c63b347c608d8d7c81e
I20260709 13:09:04.802615  8626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be2c3b332154426388d8e090ce588f2f"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
mode: GRACEFUL
new_leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:43746
I20260709 13:09:04.803201  8626 raft_consensus.cc:606] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Received request to transfer leadership to TS 5ec5ac80f9fd413a957364f29ec50feb
I20260709 13:09:04.809588  8626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1c56c3b74ac4b26b51002ad7f31adc1"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
mode: GRACEFUL
new_leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:43746
I20260709 13:09:04.810212  8626 raft_consensus.cc:606] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Received request to transfer leadership to TS 5ec5ac80f9fd413a957364f29ec50feb
I20260709 13:09:04.820585  8626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "715c5d36abd94180a1febc2638238061"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
mode: GRACEFUL
new_leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:43746
I20260709 13:09:04.821225  8626 raft_consensus.cc:606] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Received request to transfer leadership to TS 5ec5ac80f9fd413a957364f29ec50feb
I20260709 13:09:04.821902  8696 consensus_queue.cc:1048] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.833693  8675 consensus_queue.cc:1048] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.855894  8698 consensus_queue.cc:1048] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.864221  8705 raft_consensus.cc:993] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb: : Instructing follower 3196546f9f674c63b347c608d8d7c81e to start an election
I20260709 13:09:04.864734  8705 raft_consensus.cc:1081] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Signalling peer 3196546f9f674c63b347c608d8d7c81e to start an election
I20260709 13:09:04.867151  8477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1704f9c82d24421cb4444772e150092d"
dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
 from {username='slave'} at 127.0.0.1:58206
I20260709 13:09:04.867833  8477 raft_consensus.cc:493] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:04.868263  8477 raft_consensus.cc:3060] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:04.870092  8676 consensus_queue.cc:1048] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:04.876400  8477 raft_consensus.cc:515] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:04.877696  8478 raft_consensus.cc:1240] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Rejecting Update request from peer 5ec5ac80f9fd413a957364f29ec50feb for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:04.879884  8623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1704f9c82d24421cb4444772e150092d" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:04.880321  8477 leader_election.cc:290] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 2 election: Requested vote from peers acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:04.880728  8623 raft_consensus.cc:3060] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:04.881731  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1704f9c82d24421cb4444772e150092d" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:04.882377  8553 raft_consensus.cc:3055] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:04.882926  8553 raft_consensus.cc:740] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:04.883791  8553 consensus_queue.cc:260] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:04.884639  8676 consensus_queue.cc:1059] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:04.886396  8553 raft_consensus.cc:3060] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:04.891881  8623 raft_consensus.cc:2468] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 2.
I20260709 13:09:04.893391  8442 leader_election.cc:304] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:04.894352  8675 raft_consensus.cc:2804] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:04.894896  8675 raft_consensus.cc:697] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 2 LEADER]: Becoming Leader. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Running, Role: LEADER
I20260709 13:09:04.894610  8553 raft_consensus.cc:2468] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 2.
I20260709 13:09:04.896014  8675 consensus_queue.cc:237] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } }
I20260709 13:09:04.904296  8376 catalog_manager.cc:5697] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e reported cstate change: term changed from 1 to 2, leader changed from 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130) to 3196546f9f674c63b347c608d8d7c81e (127.27.71.129). New cstate: current_term: 2 leader_uuid: "3196546f9f674c63b347c608d8d7c81e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:04.949925  8724 consensus_queue.cc:1048] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.956871  8670 raft_consensus.cc:993] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3: : Instructing follower 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:04.957418  8687 raft_consensus.cc:1081] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Signalling peer 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:04.959331  8552 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be2c3b332154426388d8e090ce588f2f"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:51992
I20260709 13:09:04.960040  8552 raft_consensus.cc:493] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:04.960497  8552 raft_consensus.cc:3060] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:04.962406  8687 consensus_queue.cc:1048] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.969081  8552 raft_consensus.cc:515] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.970428  8553 raft_consensus.cc:1240] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Rejecting Update request from peer acb23023045448a0a9eb67cbc2a361e3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:04.972332  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be2c3b332154426388d8e090ce588f2f" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:04.973006  8478 raft_consensus.cc:3060] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:04.972739  8687 consensus_queue.cc:1059] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:04.973718  8687 raft_consensus.cc:3055] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:04.974150  8687 raft_consensus.cc:740] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:04.974293  8623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be2c3b332154426388d8e090ce588f2f" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:04.975090  8552 leader_election.cc:290] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:04.975230  8687 consensus_queue.cc:260] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.976714  8687 raft_consensus.cc:3060] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:04.980739  8478 raft_consensus.cc:2468] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:04.982020  8518 leader_election.cc:304] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:04.982867  8676 raft_consensus.cc:2804] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:04.983374  8676 raft_consensus.cc:697] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Becoming Leader. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:04.984254  8676 consensus_queue.cc:237] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:04.990460  8623 raft_consensus.cc:2468] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:04.992869  8376 catalog_manager.cc:5697] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb reported cstate change: term changed from 1 to 2, leader changed from acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131) to 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130). New cstate: current_term: 2 leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:05.098241  8687 consensus_queue.cc:1048] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:05.110793  8670 consensus_queue.cc:1048] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:05.118827  8724 raft_consensus.cc:993] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3: : Instructing follower 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:05.119315  8670 raft_consensus.cc:1081] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Signalling peer 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:05.121191  8552 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "715c5d36abd94180a1febc2638238061"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:51992
I20260709 13:09:05.121980  8552 raft_consensus.cc:493] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:05.122431  8552 raft_consensus.cc:3060] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.128893  8552 raft_consensus.cc:515] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:05.131769  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "715c5d36abd94180a1febc2638238061" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:05.131052  8552 leader_election.cc:290] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:05.132503  8478 raft_consensus.cc:3060] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.133358  8623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "715c5d36abd94180a1febc2638238061" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:05.134320  8623 raft_consensus.cc:3055] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:05.134713  8623 raft_consensus.cc:740] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:05.135378  8670 consensus_queue.cc:1048] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:05.135746  8623 consensus_queue.cc:260] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:05.137367  8623 raft_consensus.cc:3060] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.141779  8478 raft_consensus.cc:2468] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:05.143347  8518 leader_election.cc:304] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:05.144347  8676 raft_consensus.cc:2804] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:05.144853  8676 raft_consensus.cc:697] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Becoming Leader. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:05.145694  8676 consensus_queue.cc:237] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:05.151643  8623 raft_consensus.cc:2468] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:05.154838  8687 raft_consensus.cc:993] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3: : Instructing follower 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:05.155560  8687 raft_consensus.cc:1081] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Signalling peer 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:05.157716  8552 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c1c56c3b74ac4b26b51002ad7f31adc1"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:51992
I20260709 13:09:05.158377  8552 raft_consensus.cc:493] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:05.158839  8552 raft_consensus.cc:3060] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.162691  8376 catalog_manager.cc:5697] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb reported cstate change: term changed from 1 to 2, leader changed from acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131) to 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130). New cstate: current_term: 2 leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:05.163488  8670 consensus_queue.cc:1048] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:05.169337  8552 raft_consensus.cc:515] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:05.172366  8552 leader_election.cc:290] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:05.173036  8623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c56c3b74ac4b26b51002ad7f31adc1" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:05.173744  8623 raft_consensus.cc:3055] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:05.174098  8623 raft_consensus.cc:740] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:05.174993  8623 consensus_queue.cc:260] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:05.176662  8623 raft_consensus.cc:3060] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.176652  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1c56c3b74ac4b26b51002ad7f31adc1" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:05.177609  8477 raft_consensus.cc:3060] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.186311  8477 raft_consensus.cc:2468] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:05.186313  8623 raft_consensus.cc:2468] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:05.187875  8517 leader_election.cc:304] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:05.188859  8705 raft_consensus.cc:2804] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:05.189482  8705 raft_consensus.cc:697] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Becoming Leader. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:05.190621  8705 consensus_queue.cc:237] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:05.197486  8676 raft_consensus.cc:993] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb: : Instructing follower 3196546f9f674c63b347c608d8d7c81e to start an election
I20260709 13:09:05.198042  8705 raft_consensus.cc:1081] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Signalling peer 3196546f9f674c63b347c608d8d7c81e to start an election
I20260709 13:09:05.200168  8477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0befb22a314d4548b054ecae7a41f369"
dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
 from {username='slave'} at 127.0.0.1:58206
I20260709 13:09:05.200744  8477 raft_consensus.cc:493] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:05.201138  8477 raft_consensus.cc:3060] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.200773  8373 catalog_manager.cc:5697] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb reported cstate change: term changed from 1 to 2, leader changed from acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131) to 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130). New cstate: current_term: 2 leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:05.208642  8477 raft_consensus.cc:515] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:05.211285  8477 leader_election.cc:290] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 2 election: Requested vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:05.212332  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0befb22a314d4548b054ecae7a41f369" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:05.212479  8623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0befb22a314d4548b054ecae7a41f369" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:05.213125  8552 raft_consensus.cc:3055] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:05.213344  8623 raft_consensus.cc:3060] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.213558  8552 raft_consensus.cc:740] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:05.214589  8552 consensus_queue.cc:260] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:05.215981  8552 raft_consensus.cc:3060] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:05.219852  8623 raft_consensus.cc:2468] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 2.
I20260709 13:09:05.221365  8442 leader_election.cc:304] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:05.222167  8675 raft_consensus.cc:2804] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:05.222522  8552 raft_consensus.cc:2468] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 2.
I20260709 13:09:05.222826  8675 raft_consensus.cc:697] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 2 LEADER]: Becoming Leader. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Running, Role: LEADER
I20260709 13:09:05.223778  8675 consensus_queue.cc:237] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:05.231700  8373 catalog_manager.cc:5697] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e reported cstate change: term changed from 1 to 2, leader changed from 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130) to 3196546f9f674c63b347c608d8d7c81e (127.27.71.129). New cstate: current_term: 2 leader_uuid: "3196546f9f674c63b347c608d8d7c81e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:05.358729  8623 raft_consensus.cc:1275] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Refusing update from remote peer 3196546f9f674c63b347c608d8d7c81e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:05.360219  8696 consensus_queue.cc:1048] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:05.363811  8477 raft_consensus.cc:1275] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:05.365561  8705 consensus_queue.cc:1048] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:05.376092  8626 raft_consensus.cc:1275] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:05.378444  8733 consensus_queue.cc:1048] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:05.382201  8552 raft_consensus.cc:1275] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Refusing update from remote peer 3196546f9f674c63b347c608d8d7c81e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:05.388383  8750 consensus_queue.cc:1048] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:05.630167  8626 raft_consensus.cc:1275] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:05.631803  8705 consensus_queue.cc:1048] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:05.641100  8477 raft_consensus.cc:1275] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:05.642887  8733 consensus_queue.cc:1048] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:05.719398  8477 raft_consensus.cc:1275] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:05.720989  8733 consensus_queue.cc:1048] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:05.732304  8626 raft_consensus.cc:1275] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:05.733914  8705 consensus_queue.cc:1048] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:05.748478  8552 raft_consensus.cc:1275] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Refusing update from remote peer 3196546f9f674c63b347c608d8d7c81e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:05.750232  8696 consensus_queue.cc:1048] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:05.762950  8623 raft_consensus.cc:1275] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Refusing update from remote peer 3196546f9f674c63b347c608d8d7c81e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:05.765618  8675 consensus_queue.cc:1048] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:07.865455  8374 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37178:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:07.868178  8374 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:07.947467  8461 tablet_service.cc:1511] Processing CreateTablet for tablet 63d2d2a3a2cc4a30bb446d2ea2548a3c (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:07.949709  8461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63d2d2a3a2cc4a30bb446d2ea2548a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.949245  8536 tablet_service.cc:1511] Processing CreateTablet for tablet 63d2d2a3a2cc4a30bb446d2ea2548a3c (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:07.952121  8536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63d2d2a3a2cc4a30bb446d2ea2548a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.952167  8613 tablet_service.cc:1511] Processing CreateTablet for tablet 63d2d2a3a2cc4a30bb446d2ea2548a3c (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:07.954300  8613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63d2d2a3a2cc4a30bb446d2ea2548a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.959602  8463 tablet_service.cc:1511] Processing CreateTablet for tablet fb1a6adb6ddc49b4be091bc5f654f2a7 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:07.961419  8463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb1a6adb6ddc49b4be091bc5f654f2a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.976071  8538 tablet_service.cc:1511] Processing CreateTablet for tablet fb1a6adb6ddc49b4be091bc5f654f2a7 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:07.978221  8538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb1a6adb6ddc49b4be091bc5f654f2a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.983129  8612 tablet_service.cc:1511] Processing CreateTablet for tablet fb1a6adb6ddc49b4be091bc5f654f2a7 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:07.985409  8612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb1a6adb6ddc49b4be091bc5f654f2a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.990070  8466 tablet_service.cc:1511] Processing CreateTablet for tablet b11a0be526a648f18570f48676d10618 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:07.991124  8537 tablet_service.cc:1511] Processing CreateTablet for tablet b11a0be526a648f18570f48676d10618 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:07.992799  8537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b11a0be526a648f18570f48676d10618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.993906  8466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b11a0be526a648f18570f48676d10618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:07.997447  8614 tablet_service.cc:1511] Processing CreateTablet for tablet b11a0be526a648f18570f48676d10618 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:07.999177  8614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b11a0be526a648f18570f48676d10618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.006783  8764 tablet_bootstrap.cc:492] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:08.018445  8765 tablet_bootstrap.cc:492] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:08.020170  8461 tablet_service.cc:1511] Processing CreateTablet for tablet cdcbea82a6134a298c338293a58e5490 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:08.021993  8461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcbea82a6134a298c338293a58e5490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.028180  8764 tablet_bootstrap.cc:654] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.040005  8617 tablet_service.cc:1511] Processing CreateTablet for tablet cdcbea82a6134a298c338293a58e5490 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:08.050118  8617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcbea82a6134a298c338293a58e5490. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:09:08.043613  8424 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:08.035411  8536 tablet_service.cc:1511] Processing CreateTablet for tablet cdcbea82a6134a298c338293a58e5490 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:08.053791  8536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdcbea82a6134a298c338293a58e5490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.054610  8765 tablet_bootstrap.cc:654] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.055331  8612 tablet_service.cc:1511] Processing CreateTablet for tablet e73babad456c45d3b43b5372ab11d91d (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:08.057005  8612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73babad456c45d3b43b5372ab11d91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.044653  8766 tablet_bootstrap.cc:492] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:08.044209  8539 tablet_service.cc:1511] Processing CreateTablet for tablet e73babad456c45d3b43b5372ab11d91d (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:08.060989  8539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73babad456c45d3b43b5372ab11d91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.045203  8464 tablet_service.cc:1511] Processing CreateTablet for tablet e73babad456c45d3b43b5372ab11d91d (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:08.071473  8464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73babad456c45d3b43b5372ab11d91d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.066293  8465 tablet_service.cc:1511] Processing CreateTablet for tablet 6a309382875f481da2f0cc0f428599ce (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:08.075030  8465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a309382875f481da2f0cc0f428599ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.118901  8538 tablet_service.cc:1511] Processing CreateTablet for tablet 6a309382875f481da2f0cc0f428599ce (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:08.120270  8614 tablet_service.cc:1511] Processing CreateTablet for tablet 6a309382875f481da2f0cc0f428599ce (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:08.120764  8538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a309382875f481da2f0cc0f428599ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.121845  8614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a309382875f481da2f0cc0f428599ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.130415  8766 tablet_bootstrap.cc:654] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.133968  8764 tablet_bootstrap.cc:492] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:08.134838  8764 ts_tablet_manager.cc:1403] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.129s	user 0.009s	sys 0.012s
I20260709 13:09:08.138850  8764 raft_consensus.cc:359] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.164517  8764 raft_consensus.cc:385] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.164903  8764 raft_consensus.cc:740] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.149415  8465 tablet_service.cc:1511] Processing CreateTablet for tablet 2f57d57d28bd40148450355438278216 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:08.154511  8536 tablet_service.cc:1511] Processing CreateTablet for tablet 2f57d57d28bd40148450355438278216 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:08.163940  8766 tablet_bootstrap.cc:492] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:08.147099  8765 tablet_bootstrap.cc:492] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:08.167901  8536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f57d57d28bd40148450355438278216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.168380  8766 ts_tablet_manager.cc:1403] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.124s	user 0.010s	sys 0.008s
I20260709 13:09:08.168445  8765 ts_tablet_manager.cc:1403] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.150s	user 0.019s	sys 0.045s
I20260709 13:09:08.167773  8614 tablet_service.cc:1511] Processing CreateTablet for tablet 2f57d57d28bd40148450355438278216 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:08.172333  8766 raft_consensus.cc:359] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.166335  8764 consensus_queue.cc:260] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.172474  8765 raft_consensus.cc:359] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.173153  8766 raft_consensus.cc:385] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.173528  8765 raft_consensus.cc:385] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.174319  8765 raft_consensus.cc:740] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.175436  8765 consensus_queue.cc:260] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.180311  8538 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3a55bd4e5d492cb37217100e25267a (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:08.180837  8464 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3a55bd4e5d492cb37217100e25267a (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:08.181931  8538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3a55bd4e5d492cb37217100e25267a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.181010  8764 ts_tablet_manager.cc:1434] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.002s
I20260709 13:09:08.183692  8764 tablet_bootstrap.cc:492] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:08.181877  8614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f57d57d28bd40148450355438278216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.183290  8612 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3a55bd4e5d492cb37217100e25267a (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:08.185226  8612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3a55bd4e5d492cb37217100e25267a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.173635  8766 raft_consensus.cc:740] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.178316  8765 ts_tablet_manager.cc:1434] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:09:08.193354  8765 tablet_bootstrap.cc:492] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:08.167897  8465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f57d57d28bd40148450355438278216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.199672  8464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3a55bd4e5d492cb37217100e25267a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.200373  8766 consensus_queue.cc:260] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.207552  8463 tablet_service.cc:1511] Processing CreateTablet for tablet 4197c71b8306458e9b7ccb0af7c6f253 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:08.211746  8617 tablet_service.cc:1511] Processing CreateTablet for tablet 4197c71b8306458e9b7ccb0af7c6f253 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:08.218181  8765 tablet_bootstrap.cc:654] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.219187  8617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4197c71b8306458e9b7ccb0af7c6f253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.222486  8539 tablet_service.cc:1511] Processing CreateTablet for tablet 4197c71b8306458e9b7ccb0af7c6f253 (DEFAULT_TABLE table=soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:08.219187  8463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4197c71b8306458e9b7ccb0af7c6f253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.224756  8539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4197c71b8306458e9b7ccb0af7c6f253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:08.275949  8765 tablet_bootstrap.cc:492] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:08.276182  8764 tablet_bootstrap.cc:654] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.276705  8765 ts_tablet_manager.cc:1403] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.084s	user 0.014s	sys 0.005s
I20260709 13:09:08.265903  8766 ts_tablet_manager.cc:1434] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.097s	user 0.011s	sys 0.003s
I20260709 13:09:08.281059  8765 raft_consensus.cc:359] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.281883  8765 raft_consensus.cc:385] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.282195  8765 raft_consensus.cc:740] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.283479  8765 consensus_queue.cc:260] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.284013  8766 tablet_bootstrap.cc:492] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:08.291260  8765 ts_tablet_manager.cc:1434] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.007s
I20260709 13:09:08.292847  8765 tablet_bootstrap.cc:492] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:08.302428  8765 tablet_bootstrap.cc:654] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.313987  8764 tablet_bootstrap.cc:492] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:08.316881  8765 tablet_bootstrap.cc:492] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:08.318478  8764 ts_tablet_manager.cc:1403] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.135s	user 0.023s	sys 0.008s
I20260709 13:09:08.318655  8765 ts_tablet_manager.cc:1403] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.007s
I20260709 13:09:08.320644  8766 tablet_bootstrap.cc:654] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.322793  8764 raft_consensus.cc:359] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.323714  8764 raft_consensus.cc:385] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.324122  8764 raft_consensus.cc:740] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.323803  8765 raft_consensus.cc:359] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.324893  8765 raft_consensus.cc:385] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.325264  8765 raft_consensus.cc:740] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.325254  8764 consensus_queue.cc:260] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.326519  8765 consensus_queue.cc:260] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.329491  8765 ts_tablet_manager.cc:1434] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:09:08.330840  8765 tablet_bootstrap.cc:492] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:08.338968  8765 tablet_bootstrap.cc:654] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.340423  8764 ts_tablet_manager.cc:1434] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.005s
I20260709 13:09:08.341823  8764 tablet_bootstrap.cc:492] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:08.350639  8709 raft_consensus.cc:493] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.351369  8709 raft_consensus.cc:515] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.359057  8709 leader_election.cc:290] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.362485  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63d2d2a3a2cc4a30bb446d2ea2548a3c" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:08.363685  8626 raft_consensus.cc:2468] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 0.
I20260709 13:09:08.365356  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63d2d2a3a2cc4a30bb446d2ea2548a3c" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:08.366688  8552 raft_consensus.cc:2468] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 0.
I20260709 13:09:08.369944  8443 leader_election.cc:304] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:08.373775  8709 raft_consensus.cc:2804] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:08.374200  8709 raft_consensus.cc:493] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.374554  8709 raft_consensus.cc:3060] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.375146  8766 tablet_bootstrap.cc:492] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:08.375167  8765 tablet_bootstrap.cc:492] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:08.376041  8766 ts_tablet_manager.cc:1403] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.092s	user 0.024s	sys 0.010s
I20260709 13:09:08.376082  8765 ts_tablet_manager.cc:1403] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.015s
I20260709 13:09:08.391036  8765 raft_consensus.cc:359] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.390940  8766 raft_consensus.cc:359] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.392002  8765 raft_consensus.cc:385] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.392195  8766 raft_consensus.cc:385] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.392390  8765 raft_consensus.cc:740] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.392710  8766 raft_consensus.cc:740] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.393337  8765 consensus_queue.cc:260] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.396256  8765 ts_tablet_manager.cc:1434] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.016s
I20260709 13:09:08.397637  8765 tablet_bootstrap.cc:492] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:08.397953  8709 raft_consensus.cc:515] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.401857  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63d2d2a3a2cc4a30bb446d2ea2548a3c" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:08.402508  8552 raft_consensus.cc:3060] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.394089  8766 consensus_queue.cc:260] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.405315  8765 tablet_bootstrap.cc:654] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.406692  8766 ts_tablet_manager.cc:1434] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.030s	user 0.015s	sys 0.005s
I20260709 13:09:08.392726  8764 tablet_bootstrap.cc:654] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.409006  8766 tablet_bootstrap.cc:492] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:08.419287  8764 tablet_bootstrap.cc:492] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:08.420039  8765 tablet_bootstrap.cc:492] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:08.420063  8764 ts_tablet_manager.cc:1403] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.079s	user 0.018s	sys 0.007s
I20260709 13:09:08.420807  8765 ts_tablet_manager.cc:1403] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.009s
I20260709 13:09:08.423673  8766 tablet_bootstrap.cc:654] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.424580  8764 raft_consensus.cc:359] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.429109  8772 raft_consensus.cc:493] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.425945  8765 raft_consensus.cc:359] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.432277  8765 raft_consensus.cc:385] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.430289  8772 raft_consensus.cc:515] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.433055  8765 raft_consensus.cc:740] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.435288  8765 consensus_queue.cc:260] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.437649  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1a6adb6ddc49b4be091bc5f654f2a7" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:08.438313  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1a6adb6ddc49b4be091bc5f654f2a7" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:08.438571  8626 raft_consensus.cc:2468] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 0.
I20260709 13:09:08.439077  8765 ts_tablet_manager.cc:1434] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.003s
I20260709 13:09:08.439117  8553 raft_consensus.cc:2468] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 0.
I20260709 13:09:08.441160  8765 tablet_bootstrap.cc:492] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:08.441102  8442 leader_election.cc:304] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:08.444562  8772 leader_election.cc:290] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.445225  8772 raft_consensus.cc:2804] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:08.446013  8772 raft_consensus.cc:493] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.446535  8772 raft_consensus.cc:3060] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.427666  8764 raft_consensus.cc:385] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.449839  8764 raft_consensus.cc:740] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.451850  8764 consensus_queue.cc:260] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.455360  8764 ts_tablet_manager.cc:1434] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.003s
I20260709 13:09:08.464840  8765 tablet_bootstrap.cc:654] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.466156  8709 leader_election.cc:290] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Requested vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.469424  8764 tablet_bootstrap.cc:492] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:08.483450  8765 tablet_bootstrap.cc:492] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:08.484439  8765 ts_tablet_manager.cc:1403] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.012s
I20260709 13:09:08.487047  8772 raft_consensus.cc:515] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.489776  8765 raft_consensus.cc:359] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.491362  8765 raft_consensus.cc:385] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.491083  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1a6adb6ddc49b4be091bc5f654f2a7" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:08.492017  8765 raft_consensus.cc:740] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.492381  8553 raft_consensus.cc:3060] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.493152  8765 consensus_queue.cc:260] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.494516  8764 tablet_bootstrap.cc:654] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.498612  8772 leader_election.cc:290] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Requested vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.498886  8765 ts_tablet_manager.cc:1434] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:09:08.500738  8765 tablet_bootstrap.cc:492] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:08.513104  8765 tablet_bootstrap.cc:654] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.515545  8705 raft_consensus.cc:493] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.516297  8705 raft_consensus.cc:515] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
W20260709 13:09:08.519690  8442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.131:40041, user_credentials={real_user=slave}} blocked reactor thread for 78972.5us
I20260709 13:09:08.521941  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63d2d2a3a2cc4a30bb446d2ea2548a3c" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:08.522979  8626 raft_consensus.cc:3060] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.527617  8765 tablet_bootstrap.cc:492] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:08.528543  8765 ts_tablet_manager.cc:1403] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.009s
I20260709 13:09:08.531316  8623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcbea82a6134a298c338293a58e5490" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:08.531423  8705 leader_election.cc:290] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.532502  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcbea82a6134a298c338293a58e5490" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:08.532552  8765 raft_consensus.cc:359] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.553709  8765 raft_consensus.cc:385] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:09:08.553857  8517 leader_election.cc:343] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.554212  8765 raft_consensus.cc:740] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.554741  8766 tablet_bootstrap.cc:492] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:08.530800  8553 raft_consensus.cc:2468] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 1.
I20260709 13:09:08.555464  8766 ts_tablet_manager.cc:1403] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.147s	user 0.045s	sys 0.069s
I20260709 13:09:08.555312  8765 consensus_queue.cc:260] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.541913  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb1a6adb6ddc49b4be091bc5f654f2a7" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:08.556962  8443 leader_election.cc:304] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:08.557338  8627 raft_consensus.cc:3060] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:08.555226  8518 leader_election.cc:343] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.558066  8772 raft_consensus.cc:2804] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.558756  8772 raft_consensus.cc:697] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Becoming Leader. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Running, Role: LEADER
I20260709 13:09:08.558590  8518 leader_election.cc:304] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb; no voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:08.550557  8764 tablet_bootstrap.cc:492] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:08.531136  8552 raft_consensus.cc:2468] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 1.
I20260709 13:09:08.560978  8764 ts_tablet_manager.cc:1403] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.092s	user 0.022s	sys 0.003s
I20260709 13:09:08.543213  8626 raft_consensus.cc:2468] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 1.
I20260709 13:09:08.562598  8443 leader_election.cc:304] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:08.559861  8772 consensus_queue.cc:237] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.564159  8765 ts_tablet_manager.cc:1434] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.035s	user 0.011s	sys 0.000s
I20260709 13:09:08.564183  8780 raft_consensus.cc:2804] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.569367  8780 raft_consensus.cc:697] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Becoming Leader. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Running, Role: LEADER
I20260709 13:09:08.569387  8770 raft_consensus.cc:2749] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.572340  8764 raft_consensus.cc:359] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.575970  8764 raft_consensus.cc:385] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.576537  8764 raft_consensus.cc:740] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.571580  8780 consensus_queue.cc:237] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.574821  8766 raft_consensus.cc:359] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.577858  8627 raft_consensus.cc:2468] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 1.
I20260709 13:09:08.577610  8764 consensus_queue.cc:260] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.577952  8766 raft_consensus.cc:385] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.579722  8766 raft_consensus.cc:740] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.581450  8709 raft_consensus.cc:493] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.581781  8764 ts_tablet_manager.cc:1434] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 13:09:08.582243  8765 tablet_bootstrap.cc:492] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:08.582108  8709 raft_consensus.cc:515] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.583475  8764 tablet_bootstrap.cc:492] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:08.584921  8709 leader_election.cc:290] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.588086  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b11a0be526a648f18570f48676d10618" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:08.589282  8552 raft_consensus.cc:2468] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 0.
I20260709 13:09:08.595010  8765 tablet_bootstrap.cc:654] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.596436  8764 tablet_bootstrap.cc:654] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.599737  8443 leader_election.cc:304] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:08.600952  8709 raft_consensus.cc:2804] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:08.608520  8709 raft_consensus.cc:493] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.609187  8709 raft_consensus.cc:3060] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.610751  8764 tablet_bootstrap.cc:492] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:08.611603  8764 ts_tablet_manager.cc:1403] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.007s
I20260709 13:09:08.612867  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b11a0be526a648f18570f48676d10618" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:08.610886  8765 tablet_bootstrap.cc:492] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:08.614393  8765 ts_tablet_manager.cc:1403] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.017s
I20260709 13:09:08.580715  8766 consensus_queue.cc:260] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.615797  8627 raft_consensus.cc:2468] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 0.
I20260709 13:09:08.617755  8766 ts_tablet_manager.cc:1434] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.062s	user 0.011s	sys 0.000s
I20260709 13:09:08.619083  8766 tablet_bootstrap.cc:492] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:08.621002  8764 raft_consensus.cc:359] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.622346  8764 raft_consensus.cc:385] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.622736  8764 raft_consensus.cc:740] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.623693  8764 consensus_queue.cc:260] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.627061  8764 ts_tablet_manager.cc:1434] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:09:08.629168  8764 tablet_bootstrap.cc:492] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:08.629376  8766 tablet_bootstrap.cc:654] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.620790  8765 raft_consensus.cc:359] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.636212  8765 raft_consensus.cc:385] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.636626  8765 raft_consensus.cc:740] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.637498  8765 consensus_queue.cc:260] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.641710  8765 ts_tablet_manager.cc:1434] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:09:08.639743  8374 catalog_manager.cc:5697] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3196546f9f674c63b347c608d8d7c81e (127.27.71.129). New cstate: current_term: 1 leader_uuid: "3196546f9f674c63b347c608d8d7c81e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.643034  8765 tablet_bootstrap.cc:492] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb: Bootstrap starting.
I20260709 13:09:08.661077  8765 tablet_bootstrap.cc:654] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.664083  8770 raft_consensus.cc:493] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.665077  8770 raft_consensus.cc:515] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.674984  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
W20260709 13:09:08.677567  8518 leader_election.cc:343] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.684882  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:08.686237  8709 raft_consensus.cc:515] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
W20260709 13:09:08.687059  8517 leader_election.cc:343] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.688951  8765 tablet_bootstrap.cc:492] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb: No bootstrap required, opened a new log
I20260709 13:09:08.689805  8765 ts_tablet_manager.cc:1403] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.009s
I20260709 13:09:08.691682  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b11a0be526a648f18570f48676d10618" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:08.689325  8517 leader_election.cc:304] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb; no voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:08.692782  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b11a0be526a648f18570f48676d10618" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:08.694167  8705 raft_consensus.cc:2749] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.694480  8552 raft_consensus.cc:3060] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.695410  8770 leader_election.cc:290] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.695590  8765 raft_consensus.cc:359] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.696506  8765 raft_consensus.cc:385] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.696823  8765 raft_consensus.cc:740] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.697505  8627 raft_consensus.cc:3060] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.697700  8765 consensus_queue.cc:260] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.699044  8709 leader_election.cc:290] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Requested vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.700531  8765 ts_tablet_manager.cc:1434] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 13:09:08.714076  8770 raft_consensus.cc:493] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.715111  8770 raft_consensus.cc:515] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.716007  8766 tablet_bootstrap.cc:492] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:08.717092  8627 raft_consensus.cc:2468] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 1.
I20260709 13:09:08.718648  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f57d57d28bd40148450355438278216" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:08.718956  8442 leader_election.cc:304] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
W20260709 13:09:08.720463  8518 leader_election.cc:343] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.720471  8709 raft_consensus.cc:2804] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.721714  8709 raft_consensus.cc:697] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Becoming Leader. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Running, Role: LEADER
I20260709 13:09:08.722664  8709 consensus_queue.cc:237] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.724284  8552 raft_consensus.cc:2468] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 1.
I20260709 13:09:08.717187  8766 ts_tablet_manager.cc:1403] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.098s	user 0.045s	sys 0.022s
I20260709 13:09:08.724565  8764 tablet_bootstrap.cc:654] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.731187  8766 raft_consensus.cc:359] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.732736  8766 raft_consensus.cc:385] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.733199  8766 raft_consensus.cc:740] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.734247  8766 consensus_queue.cc:260] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.735275  8764 tablet_bootstrap.cc:492] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:08.734658  8770 leader_election.cc:290] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.740507  8766 ts_tablet_manager.cc:1434] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.002s
I20260709 13:09:08.736238  8764 ts_tablet_manager.cc:1403] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.108s	user 0.014s	sys 0.009s
I20260709 13:09:08.741930  8766 tablet_bootstrap.cc:492] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:08.736169  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f57d57d28bd40148450355438278216" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
W20260709 13:09:08.744616  8517 leader_election.cc:343] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.745154  8517 leader_election.cc:304] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb; no voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:08.749734  8764 raft_consensus.cc:359] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.750583  8705 raft_consensus.cc:2749] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.750782  8764 raft_consensus.cc:385] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.751245  8764 raft_consensus.cc:740] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.752731  8766 tablet_bootstrap.cc:654] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.752389  8764 consensus_queue.cc:260] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.755208  8764 ts_tablet_manager.cc:1434] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:09:08.756649  8764 tablet_bootstrap.cc:492] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:08.763983  8705 raft_consensus.cc:493] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.764922  8705 raft_consensus.cc:515] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.769052  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a55bd4e5d492cb37217100e25267a" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:08.770515  8705 leader_election.cc:290] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.772604  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a55bd4e5d492cb37217100e25267a" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
W20260709 13:09:08.773295  8518 leader_election.cc:343] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:08.774434  8517 leader_election.cc:343] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:08.774957  8517 leader_election.cc:304] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb; no voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:08.775693  8764 tablet_bootstrap.cc:654] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.776398  8766 tablet_bootstrap.cc:492] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:08.776455  8705 raft_consensus.cc:2749] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:08.777365  8766 ts_tablet_manager.cc:1403] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.016s
I20260709 13:09:08.777909  8376 catalog_manager.cc:5697] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3196546f9f674c63b347c608d8d7c81e (127.27.71.129). New cstate: current_term: 1 leader_uuid: "3196546f9f674c63b347c608d8d7c81e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.780056  8376 catalog_manager.cc:5697] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3196546f9f674c63b347c608d8d7c81e (127.27.71.129). New cstate: current_term: 1 leader_uuid: "3196546f9f674c63b347c608d8d7c81e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.781937  8764 tablet_bootstrap.cc:492] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:08.782608  8764 ts_tablet_manager.cc:1403] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.005s
I20260709 13:09:08.781997  8766 raft_consensus.cc:359] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.783103  8766 raft_consensus.cc:385] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.783488  8766 raft_consensus.cc:740] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.784484  8766 consensus_queue.cc:260] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.787295  8766 ts_tablet_manager.cc:1434] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:09:08.788930  8766 tablet_bootstrap.cc:492] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:08.786284  8764 raft_consensus.cc:359] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.792299  8764 raft_consensus.cc:385] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.792719  8764 raft_consensus.cc:740] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.793519  8764 consensus_queue.cc:260] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.796078  8764 ts_tablet_manager.cc:1434] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.008s
I20260709 13:09:08.797214  8764 tablet_bootstrap.cc:492] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:08.797698  8766 tablet_bootstrap.cc:654] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.806717  8764 tablet_bootstrap.cc:654] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.808954  8766 tablet_bootstrap.cc:492] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:08.810072  8766 ts_tablet_manager.cc:1403] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 13:09:08.813180  8764 tablet_bootstrap.cc:492] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:08.813763  8764 ts_tablet_manager.cc:1403] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.017s	user 0.004s	sys 0.008s
I20260709 13:09:08.813555  8766 raft_consensus.cc:359] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.814354  8766 raft_consensus.cc:385] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.814723  8766 raft_consensus.cc:740] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.815887  8766 consensus_queue.cc:260] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.816867  8764 raft_consensus.cc:359] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.817646  8764 raft_consensus.cc:385] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.817919  8764 raft_consensus.cc:740] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.818367  8766 ts_tablet_manager.cc:1434] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:09:08.818732  8764 consensus_queue.cc:260] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.819957  8670 raft_consensus.cc:493] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.820214  8766 tablet_bootstrap.cc:492] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:08.820503  8670 raft_consensus.cc:515] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.820960  8764 ts_tablet_manager.cc:1434] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:09:08.822638  8764 tablet_bootstrap.cc:492] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e: Bootstrap starting.
I20260709 13:09:08.825225  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a309382875f481da2f0cc0f428599ce" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:08.825944  8477 raft_consensus.cc:2468] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:08.826943  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a309382875f481da2f0cc0f428599ce" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:08.827201  8592 leader_election.cc:304] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:08.831152  8766 tablet_bootstrap.cc:654] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.827740  8552 raft_consensus.cc:2468] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:08.837294  8764 tablet_bootstrap.cc:654] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.822767  8670 leader_election.cc:290] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:08.843983  8670 raft_consensus.cc:2804] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:08.844410  8670 raft_consensus.cc:493] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.844745  8670 raft_consensus.cc:3060] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.846186  8709 raft_consensus.cc:493] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.846781  8709 raft_consensus.cc:515] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.849056  8709 leader_election.cc:290] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.851837  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcbea82a6134a298c338293a58e5490" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:08.852741  8552 raft_consensus.cc:2468] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 0.
I20260709 13:09:08.854063  8443 leader_election.cc:304] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:08.855026  8709 raft_consensus.cc:2804] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:08.855454  8709 raft_consensus.cc:493] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.856015  8709 raft_consensus.cc:3060] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.860831  8670 raft_consensus.cc:515] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.863622  8670 leader_election.cc:290] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:08.868790  8766 tablet_bootstrap.cc:492] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:08.869640  8766 ts_tablet_manager.cc:1403] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.050s	user 0.022s	sys 0.024s
I20260709 13:09:08.873540  8766 raft_consensus.cc:359] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
W20260709 13:09:08.874830  8592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:36551, user_credentials={real_user=slave}} blocked reactor thread for 47742.6us
I20260709 13:09:08.876542  8766 raft_consensus.cc:385] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.877044  8766 raft_consensus.cc:740] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.877970  8766 consensus_queue.cc:260] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.878260  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a309382875f481da2f0cc0f428599ce" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:08.879055  8477 raft_consensus.cc:3060] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.881016  8766 ts_tablet_manager.cc:1434] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.001s
I20260709 13:09:08.882453  8766 tablet_bootstrap.cc:492] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:08.885277  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a309382875f481da2f0cc0f428599ce" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:08.886157  8552 raft_consensus.cc:3060] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.889643  8766 tablet_bootstrap.cc:654] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.900573  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcbea82a6134a298c338293a58e5490" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:08.901553  8627 raft_consensus.cc:2468] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 0.
I20260709 13:09:08.914603  8477 raft_consensus.cc:2468] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:08.923156  8766 tablet_bootstrap.cc:492] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:08.925153  8552 raft_consensus.cc:2468] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:08.925532  8709 raft_consensus.cc:515] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.927022  8766 ts_tablet_manager.cc:1403] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.045s	user 0.026s	sys 0.012s
I20260709 13:09:08.927809  8592 leader_election.cc:304] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:08.928791  8670 raft_consensus.cc:2804] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.929301  8670 raft_consensus.cc:697] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:08.930065  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcbea82a6134a298c338293a58e5490" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:08.930187  8670 consensus_queue.cc:237] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.930830  8553 raft_consensus.cc:3060] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.931252  8766 raft_consensus.cc:359] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.931317  8764 tablet_bootstrap.cc:492] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e: No bootstrap required, opened a new log
I20260709 13:09:08.932070  8766 raft_consensus.cc:385] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.932430  8766 raft_consensus.cc:740] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.933461  8709 leader_election.cc:290] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Requested vote from peers 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:08.933173  8766 consensus_queue.cc:260] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.932555  8764 ts_tablet_manager.cc:1403] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e: Time spent bootstrapping tablet: real 0.110s	user 0.050s	sys 0.047s
I20260709 13:09:08.934825  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcbea82a6134a298c338293a58e5490" candidate_uuid: "3196546f9f674c63b347c608d8d7c81e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:08.935503  8627 raft_consensus.cc:3060] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:08.937860  8766 ts_tablet_manager.cc:1434] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.006s
I20260709 13:09:08.940276  8766 tablet_bootstrap.cc:492] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3: Bootstrap starting.
I20260709 13:09:08.941983  8764 raft_consensus.cc:359] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.943231  8376 catalog_manager.cc:5697] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:08.943257  8764 raft_consensus.cc:385] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.946841  8553 raft_consensus.cc:2468] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 1.
I20260709 13:09:08.948624  8443 leader_election.cc:304] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:08.949262  8766 tablet_bootstrap.cc:654] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:08.949860  8709 raft_consensus.cc:2804] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:08.950125  8627 raft_consensus.cc:2468] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3196546f9f674c63b347c608d8d7c81e in term 1.
I20260709 13:09:08.951154  8709 raft_consensus.cc:697] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Becoming Leader. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Running, Role: LEADER
I20260709 13:09:08.947084  8764 raft_consensus.cc:740] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.952137  8764 consensus_queue.cc:260] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.952102  8709 consensus_queue.cc:237] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.954613  8764 ts_tablet_manager.cc:1434] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:09:08.961860  8766 tablet_bootstrap.cc:492] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3: No bootstrap required, opened a new log
I20260709 13:09:08.962502  8766 ts_tablet_manager.cc:1403] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.008s
I20260709 13:09:08.965345  8766 raft_consensus.cc:359] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.966166  8766 raft_consensus.cc:385] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:08.966449  8766 raft_consensus.cc:740] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Initialized, Role: FOLLOWER
I20260709 13:09:08.965862  8374 catalog_manager.cc:5697] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3196546f9f674c63b347c608d8d7c81e (127.27.71.129). New cstate: current_term: 1 leader_uuid: "3196546f9f674c63b347c608d8d7c81e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.967271  8766 consensus_queue.cc:260] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:08.969188  8766 ts_tablet_manager.cc:1434] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:09:08.998268  8709 consensus_queue.cc:1048] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:09.002749  8780 consensus_queue.cc:1048] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:09.026494  8670 raft_consensus.cc:493] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.027072  8670 raft_consensus.cc:515] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.028805  8780 consensus_queue.cc:1048] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:09.029448  8670 leader_election.cc:290] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:09.032162  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:09.032881  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:09.033375  8553 raft_consensus.cc:2468] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:09.033689  8477 raft_consensus.cc:2468] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:09.034853  8592 leader_election.cc:304] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:09.035946  8670 raft_consensus.cc:2804] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.036486  8670 raft_consensus.cc:493] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.037176  8670 raft_consensus.cc:3060] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.048269  8670 raft_consensus.cc:515] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.050241  8670 leader_election.cc:290] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:09.050850  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:09.051605  8477 raft_consensus.cc:3060] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.051828  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:09.052728  8553 raft_consensus.cc:3060] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:09.059849  8424 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:09.061057  8709 consensus_queue.cc:1048] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.061087  8553 raft_consensus.cc:2468] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:09.062834  8592 leader_election.cc:304] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:09.063915  8670 raft_consensus.cc:2804] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.064591  8670 raft_consensus.cc:697] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:09.065599  8670 consensus_queue.cc:237] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.070298  8477 raft_consensus.cc:2468] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:09.085021  8374 catalog_manager.cc:5697] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.086439  8705 raft_consensus.cc:493] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.087143  8705 raft_consensus.cc:515] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.089510  8705 leader_election.cc:290] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:09.090848  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4197c71b8306458e9b7ccb0af7c6f253" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3" is_pre_election: true
I20260709 13:09:09.091090  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4197c71b8306458e9b7ccb0af7c6f253" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:09.091751  8627 raft_consensus.cc:2468] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 0.
I20260709 13:09:09.091778  8477 raft_consensus.cc:2468] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 0.
I20260709 13:09:09.094079  8518 leader_election.cc:304] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:09.094913  8705 raft_consensus.cc:2804] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.095283  8705 raft_consensus.cc:493] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.095602  8705 raft_consensus.cc:3060] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.104460  8705 raft_consensus.cc:515] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.106904  8705 leader_election.cc:290] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:09.107782  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4197c71b8306458e9b7ccb0af7c6f253" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:09.108549  8477 raft_consensus.cc:3060] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.108032  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4197c71b8306458e9b7ccb0af7c6f253" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:09.109058  8780 consensus_queue.cc:1048] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.109794  8627 raft_consensus.cc:3060] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.117683  8477 raft_consensus.cc:2468] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 1.
I20260709 13:09:09.119557  8518 leader_election.cc:304] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:09.120057  8627 raft_consensus.cc:2468] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 1.
I20260709 13:09:09.120795  8705 raft_consensus.cc:2804] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.121435  8705 raft_consensus.cc:697] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Becoming Leader. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:09.122684  8705 consensus_queue.cc:237] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.138756  8781 consensus_queue.cc:1048] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.145859  8376 catalog_manager.cc:5697] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130). New cstate: current_term: 1 leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:09.196740  8670 raft_consensus.cc:493] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.197383  8670 raft_consensus.cc:515] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.199502  8670 leader_election.cc:290] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:09.201033  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f57d57d28bd40148450355438278216" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:09.201159  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f57d57d28bd40148450355438278216" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:09.201810  8553 raft_consensus.cc:2468] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:09.201926  8477 raft_consensus.cc:2468] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:09.202975  8592 leader_election.cc:304] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:09.204241  8670 raft_consensus.cc:2804] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.204874  8670 raft_consensus.cc:493] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.205487  8670 raft_consensus.cc:3060] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.213337  8670 raft_consensus.cc:515] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.215090  8670 leader_election.cc:290] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:09.216349  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f57d57d28bd40148450355438278216" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:09.216997  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f57d57d28bd40148450355438278216" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:09.217057  8477 raft_consensus.cc:3060] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.217697  8553 raft_consensus.cc:3060] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.224226  8553 raft_consensus.cc:2468] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:09.225755  8592 leader_election.cc:304] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:09.227126  8477 raft_consensus.cc:2468] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:09.227142  8670 raft_consensus.cc:2804] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.227387  8801 raft_consensus.cc:493] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.228607  8801 raft_consensus.cc:515] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.229060  8670 raft_consensus.cc:697] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:09.230129  8670 consensus_queue.cc:237] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.232707  8801 leader_election.cc:290] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:09.233703  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a55bd4e5d492cb37217100e25267a" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e" is_pre_election: true
I20260709 13:09:09.234557  8477 raft_consensus.cc:2468] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:09.234858  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a55bd4e5d492cb37217100e25267a" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb" is_pre_election: true
I20260709 13:09:09.236332  8553 raft_consensus.cc:2468] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 0.
I20260709 13:09:09.236745  8592 leader_election.cc:304] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:09.237953  8670 raft_consensus.cc:2804] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:09.238458  8670 raft_consensus.cc:493] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.238814  8670 raft_consensus.cc:3060] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.243022  8374 catalog_manager.cc:5697] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.248693  8670 raft_consensus.cc:515] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.251778  8670 leader_election.cc:290] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:09.252717  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a55bd4e5d492cb37217100e25267a" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:09.253096  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a55bd4e5d492cb37217100e25267a" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:09.253329  8477 raft_consensus.cc:3060] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.253743  8553 raft_consensus.cc:3060] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:09.260597  8553 raft_consensus.cc:2468] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:09.261485  8477 raft_consensus.cc:2468] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 1.
I20260709 13:09:09.261727  8592 leader_election.cc:304] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:09.262791  8670 raft_consensus.cc:2804] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:09.263478  8670 raft_consensus.cc:697] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:09.264304  8670 consensus_queue.cc:237] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.278007  8374 catalog_manager.cc:5697] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.326375  8670 consensus_queue.cc:1048] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.336014  8781 consensus_queue.cc:1048] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.340893  8801 consensus_queue.cc:1048] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.359030  8781 consensus_queue.cc:1048] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:09.389360  8627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
mode: GRACEFUL
new_leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:43746
I20260709 13:09:09.390018  8627 raft_consensus.cc:606] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Received request to transfer leadership to TS 5ec5ac80f9fd413a957364f29ec50feb
I20260709 13:09:09.397011  8627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a309382875f481da2f0cc0f428599ce"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
mode: GRACEFUL
new_leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:43746
I20260709 13:09:09.397634  8627 raft_consensus.cc:606] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Received request to transfer leadership to TS 5ec5ac80f9fd413a957364f29ec50feb
I20260709 13:09:09.403733  8627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f57d57d28bd40148450355438278216"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
mode: GRACEFUL
new_leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:43746
I20260709 13:09:09.404422  8627 raft_consensus.cc:606] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Received request to transfer leadership to TS 5ec5ac80f9fd413a957364f29ec50feb
I20260709 13:09:09.410199  8627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e3a55bd4e5d492cb37217100e25267a"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
mode: GRACEFUL
new_leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:43746
I20260709 13:09:09.410768  8627 raft_consensus.cc:606] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Received request to transfer leadership to TS 5ec5ac80f9fd413a957364f29ec50feb
I20260709 13:09:09.416627  8552 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4197c71b8306458e9b7ccb0af7c6f253"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
mode: GRACEFUL
new_leader_uuid: "acb23023045448a0a9eb67cbc2a361e3"
 from {username='slave'} at 127.0.0.1:52006
I20260709 13:09:09.417259  8552 raft_consensus.cc:606] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Received request to transfer leadership to TS acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:09.495067  8801 consensus_queue.cc:1048] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.508672  8670 consensus_queue.cc:1048] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.519266  8820 raft_consensus.cc:993] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3: : Instructing follower 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:09.519827  8820 raft_consensus.cc:1081] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Signalling peer 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:09.521732  8552 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:51992
I20260709 13:09:09.522271  8552 raft_consensus.cc:493] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:09.522637  8552 raft_consensus.cc:3060] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.528062  8552 raft_consensus.cc:515] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.530692  8552 leader_election.cc:290] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:09.531129  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:09.531809  8477 raft_consensus.cc:3060] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.533239  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:09.533913  8627 raft_consensus.cc:3055] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:09.534251  8627 raft_consensus.cc:740] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:09.535107  8627 consensus_queue.cc:260] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.536550  8627 raft_consensus.cc:3060] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.538400  8477 raft_consensus.cc:2468] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:09.539484  8518 leader_election.cc:304] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:09.540302  8770 raft_consensus.cc:2804] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:09.540845  8770 raft_consensus.cc:697] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Becoming Leader. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:09.541688  8770 consensus_queue.cc:237] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.546211  8627 raft_consensus.cc:2468] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:09.551131  8373 catalog_manager.cc:5697] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb reported cstate change: term changed from 1 to 2, leader changed from acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131) to 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130). New cstate: current_term: 2 leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:09.620716  8820 consensus_queue.cc:1048] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.633610  8801 consensus_queue.cc:1048] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.645092  8670 raft_consensus.cc:993] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3: : Instructing follower 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:09.645737  8670 raft_consensus.cc:1081] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Signalling peer 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:09.647557  8552 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f57d57d28bd40148450355438278216"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:51992
I20260709 13:09:09.648030  8552 raft_consensus.cc:493] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:09.648307  8552 raft_consensus.cc:3060] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.654100  8552 raft_consensus.cc:515] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.656030  8552 leader_election.cc:290] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:09.657344  8477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f57d57d28bd40148450355438278216" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:09.657569  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f57d57d28bd40148450355438278216" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:09.658360  8477 raft_consensus.cc:3060] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.658638  8626 raft_consensus.cc:3055] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:09.659054  8626 raft_consensus.cc:740] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:09.659891  8626 consensus_queue.cc:260] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.661185  8626 raft_consensus.cc:3060] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.666303  8477 raft_consensus.cc:2468] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:09.667557  8518 leader_election.cc:304] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:09.668337  8770 raft_consensus.cc:2804] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:09.668813  8770 raft_consensus.cc:697] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Becoming Leader. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:09.669330  8626 raft_consensus.cc:2468] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:09.669565  8770 consensus_queue.cc:237] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.674966  8670 consensus_queue.cc:1048] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:09.680245  8374 catalog_manager.cc:5697] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb reported cstate change: term changed from 1 to 2, leader changed from acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131) to 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130). New cstate: current_term: 2 leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:09.689055  8820 consensus_queue.cc:1048] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.699085  8770 consensus_queue.cc:1048] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:09.709177  8820 raft_consensus.cc:993] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3: : Instructing follower 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:09.709705  8820 raft_consensus.cc:1081] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Signalling peer 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:09.715579  8770 consensus_queue.cc:1048] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:09.718575  8552 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e3a55bd4e5d492cb37217100e25267a"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:51992
I20260709 13:09:09.720114  8552 raft_consensus.cc:493] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:09.724895  8552 raft_consensus.cc:3060] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.734192  8552 raft_consensus.cc:515] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.736653  8552 leader_election.cc:290] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:09.737224  8705 raft_consensus.cc:993] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb: : Instructing follower acb23023045448a0a9eb67cbc2a361e3 to start an election
I20260709 13:09:09.737576  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a55bd4e5d492cb37217100e25267a" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:09.737797  8705 raft_consensus.cc:1081] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Signalling peer acb23023045448a0a9eb67cbc2a361e3 to start an election
I20260709 13:09:09.738241  8478 raft_consensus.cc:3060] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.743860  8626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a55bd4e5d492cb37217100e25267a" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:09.744304  8627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4197c71b8306458e9b7ccb0af7c6f253"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
 from {username='slave'} at 127.0.0.1:43736
I20260709 13:09:09.744781  8626 raft_consensus.cc:3055] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:09.745170  8627 raft_consensus.cc:493] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:09.745312  8626 raft_consensus.cc:740] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:09.745676  8627 raft_consensus.cc:3060] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.746429  8626 consensus_queue.cc:260] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.747157  8478 raft_consensus.cc:2468] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:09.748188  8626 raft_consensus.cc:3060] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.748365  8518 leader_election.cc:304] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:09.749373  8770 raft_consensus.cc:2804] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:09.750118  8770 raft_consensus.cc:697] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Becoming Leader. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:09.751596  8770 consensus_queue.cc:237] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.754681  8626 raft_consensus.cc:2468] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:09.754938  8627 raft_consensus.cc:515] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.757047  8627 leader_election.cc:290] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 2 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:09.758006  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4197c71b8306458e9b7ccb0af7c6f253" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:09.758260  8552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4197c71b8306458e9b7ccb0af7c6f253" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:09.758705  8478 raft_consensus.cc:3060] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.758898  8552 raft_consensus.cc:3055] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:09.759819  8552 raft_consensus.cc:740] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:09.760803  8552 consensus_queue.cc:260] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.761986  8552 raft_consensus.cc:3060] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.764127  8374 catalog_manager.cc:5697] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb reported cstate change: term changed from 1 to 2, leader changed from acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131) to 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130). New cstate: current_term: 2 leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:09.768255  8478 raft_consensus.cc:2468] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 2.
I20260709 13:09:09.768839  8552 raft_consensus.cc:2468] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 2.
I20260709 13:09:09.769510  8592 leader_election.cc:304] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:09.770430  8670 raft_consensus.cc:2804] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:09.770895  8670 raft_consensus.cc:697] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 2 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:09.771869  8670 consensus_queue.cc:237] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.780133  8374 catalog_manager.cc:5697] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 1 to 2, leader changed from 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130) to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:09.902427  8670 raft_consensus.cc:993] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3: : Instructing follower 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:09.902940  8820 raft_consensus.cc:1081] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Signalling peer 5ec5ac80f9fd413a957364f29ec50feb to start an election
I20260709 13:09:09.904712  8552 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a309382875f481da2f0cc0f428599ce"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
 from {username='slave'} at 127.0.0.1:51992
I20260709 13:09:09.905314  8552 raft_consensus.cc:493] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:09.905684  8552 raft_consensus.cc:3060] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.913234  8552 raft_consensus.cc:515] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.914739  8552 leader_election.cc:290] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131:40041)
I20260709 13:09:09.915797  8478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a309382875f481da2f0cc0f428599ce" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:09.916025  8627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a309382875f481da2f0cc0f428599ce" candidate_uuid: "5ec5ac80f9fd413a957364f29ec50feb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
I20260709 13:09:09.916464  8478 raft_consensus.cc:3060] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.916671  8627 raft_consensus.cc:3055] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:09.917084  8627 raft_consensus.cc:740] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:09.917855  8627 consensus_queue.cc:260] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.919616  8627 raft_consensus.cc:3060] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.923014  8478 raft_consensus.cc:2468] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:09.924476  8518 leader_election.cc:304] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, 5ec5ac80f9fd413a957364f29ec50feb; no voters: 
I20260709 13:09:09.925357  8770 raft_consensus.cc:2804] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:09.925676  8627 raft_consensus.cc:2468] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec5ac80f9fd413a957364f29ec50feb in term 2.
I20260709 13:09:09.925871  8770 raft_consensus.cc:697] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Becoming Leader. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:09.926810  8770 consensus_queue.cc:237] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:09.933789  8374 catalog_manager.cc:5697] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb reported cstate change: term changed from 1 to 2, leader changed from acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131) to 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130). New cstate: current_term: 2 leader_uuid: "5ec5ac80f9fd413a957364f29ec50feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:10.050652  8626 raft_consensus.cc:1275] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:10.052630  8770 consensus_queue.cc:1048] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:10.066954  8478 raft_consensus.cc:1275] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:10.074201  8705 consensus_queue.cc:1048] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:10.079308  8626 raft_consensus.cc:1275] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:10.084393  8834 consensus_queue.cc:1048] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:10.091285  8478 raft_consensus.cc:1275] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:10.093261  8834 consensus_queue.cc:1048] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:10.260222  8478 raft_consensus.cc:1275] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:10.261708  8834 consensus_queue.cc:1048] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:10.270289  8626 raft_consensus.cc:1275] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:10.272517  8705 consensus_queue.cc:1048] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:10.309350  8478 raft_consensus.cc:1275] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Refusing update from remote peer acb23023045448a0a9eb67cbc2a361e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:10.310691  8670 consensus_queue.cc:1048] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:10.311988  8552 raft_consensus.cc:1275] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Refusing update from remote peer acb23023045448a0a9eb67cbc2a361e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:10.313546  8670 consensus_queue.cc:1048] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:10.419341  8478 raft_consensus.cc:1275] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:10.420876  8834 consensus_queue.cc:1048] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:10.423179  8626 raft_consensus.cc:1275] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Refusing update from remote peer 5ec5ac80f9fd413a957364f29ec50feb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:10.425006  8834 consensus_queue.cc:1048] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:12.421864  8375 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37178:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:09:12.445070  8375 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37178:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:09:12.512252 27934 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:09:12.612641  8861 log.cc:927] Time spent T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:09:12.777817  8867 log.cc:927] Time spent T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb: Append to log took a long time: real 0.163s	user 0.002s	sys 0.000s
I20260709 13:09:12.789116  8882 tablet.cc:1724] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:12.875488  8898 tablet.cc:1724] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:12.880143  8899 tablet.cc:1724] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:12.911332  8892 tablet.cc:1724] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:12.957472  8886 log.cc:927] Time spent T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:09:12.952824  8897 tablet.cc:1724] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:12.996340  8902 tablet.cc:1724] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:12.971810  8903 tablet.cc:1724] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:13.004410  8442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.131:40041, user_credentials={real_user=slave}} blocked reactor thread for 91964.8us
I20260709 13:09:13.009733  8906 tablet.cc:1724] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.008942  8901 tablet.cc:1724] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:13.020426  8579 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:4197c71b8306458e9b7ccb0af7c6f253)
I20260709 13:09:13.013458  8900 tablet.cc:1724] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:13.053663  8579 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:4197c71b8306458e9b7ccb0af7c6f253)
W20260709 13:09:13.055560  8579 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:4197c71b8306458e9b7ccb0af7c6f253)
I20260709 13:09:13.085716  8892 tablet.cc:1724] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.095975  8909 tablet.cc:1724] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.103106  8475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdcbea82a6134a298c338293a58e5490"
dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
mode: GRACEFUL
new_leader_uuid: "acb23023045448a0a9eb67cbc2a361e3"
 from {username='slave'} at 127.0.0.1:47970
I20260709 13:09:13.104326  8475 raft_consensus.cc:606] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Received request to transfer leadership to TS acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:13.072388  8906 tablet.cc:1724] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.129616  8898 tablet.cc:1724] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.142156  8868 raft_consensus.cc:993] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e: : Instructing follower acb23023045448a0a9eb67cbc2a361e3 to start an election
I20260709 13:09:13.142798  8868 raft_consensus.cc:1081] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Signalling peer acb23023045448a0a9eb67cbc2a361e3 to start an election
I20260709 13:09:13.152357  8897 tablet.cc:1724] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.159495  8899 tablet.cc:1724] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.162400  8909 tablet.cc:1724] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.133024  8892 tablet.cc:1724] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.183754  8899 tablet.cc:1724] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.187635  8902 tablet.cc:1724] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.188936  8909 tablet.cc:1724] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.206305  8903 tablet.cc:1724] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.210659  8901 tablet.cc:1724] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.152472  8912 tablet.cc:1724] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.218470  8900 tablet.cc:1724] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.176980  8882 tablet.cc:1724] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.252091  8624 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdcbea82a6134a298c338293a58e5490"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
 from {username='slave'} at 127.0.0.1:43744
I20260709 13:09:13.252882  8624 raft_consensus.cc:493] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:13.253350  8624 raft_consensus.cc:3060] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:13.263206  8553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
mode: GRACEFUL
new_leader_uuid: "acb23023045448a0a9eb67cbc2a361e3"
 from {username='slave'} at 127.0.0.1:46206
I20260709 13:09:13.264183  8553 raft_consensus.cc:606] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Received request to transfer leadership to TS acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:13.270392 27934 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:09:13.274420 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:13.286625  8474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0906217e5d2b4ab08f5d25f61dd28cb5"
dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
mode: GRACEFUL
new_leader_uuid: "acb23023045448a0a9eb67cbc2a361e3"
 from {username='slave'} at 127.0.0.1:47970
I20260709 13:09:13.287518  8474 raft_consensus.cc:606] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Received request to transfer leadership to TS acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:13.297811  8553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "847a35bccaa640e1b0c3f4c26aac3793"
dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
mode: GRACEFUL
new_leader_uuid: "acb23023045448a0a9eb67cbc2a361e3"
 from {username='slave'} at 127.0.0.1:46206
I20260709 13:09:13.298380  8553 raft_consensus.cc:606] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Received request to transfer leadership to TS acb23023045448a0a9eb67cbc2a361e3
I20260709 13:09:13.299988 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:13.300701 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:13.307281  8903 tablet.cc:1724] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:13.323521  8624 raft_consensus.cc:515] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.336643  8474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcbea82a6134a298c338293a58e5490" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:13.337463  8474 raft_consensus.cc:3055] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:13.337821  8474 raft_consensus.cc:740] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Running, Role: LEADER
I20260709 13:09:13.338899  8474 consensus_queue.cc:260] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.341876  8474 raft_consensus.cc:3060] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:13.349704  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdcbea82a6134a298c338293a58e5490" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:13.350476  8553 raft_consensus.cc:3060] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:13.353353  8474 raft_consensus.cc:2468] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 2.
I20260709 13:09:13.355095  8592 leader_election.cc:304] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:13.356107  8624 leader_election.cc:290] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 2 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:13.356467  8801 raft_consensus.cc:2804] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:13.360448  8801 raft_consensus.cc:697] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 2 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:13.362784  8553 raft_consensus.cc:2468] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 2.
I20260709 13:09:13.364831  8801 consensus_queue.cc:237] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.370096  8373 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e: soft_deleted_table [id=953a1f762b784757bde2a6c248d97d6f] alter complete (version 1)
I20260709 13:09:13.380007  8376 catalog_manager.cc:5697] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 1 to 2, leader changed from 3196546f9f674c63b347c608d8d7c81e (127.27.71.129) to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:13.592382  8891 raft_consensus.cc:993] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb: : Instructing follower acb23023045448a0a9eb67cbc2a361e3 to start an election
I20260709 13:09:13.592965  8884 raft_consensus.cc:1081] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Signalling peer acb23023045448a0a9eb67cbc2a361e3 to start an election
I20260709 13:09:13.594726  8624 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
 from {username='slave'} at 127.0.0.1:43736
I20260709 13:09:13.595198  8624 raft_consensus.cc:493] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:13.595460  8624 raft_consensus.cc:3060] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:13.603236  8624 raft_consensus.cc:515] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.605379  8624 leader_election.cc:290] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 3 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:13.606917  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:13.606933  8474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73babad456c45d3b43b5372ab11d91d" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:13.607539  8553 raft_consensus.cc:3055] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:13.608095  8553 raft_consensus.cc:740] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:13.608119  8474 raft_consensus.cc:3060] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:13.608899  8553 consensus_queue.cc:260] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.609939  8553 raft_consensus.cc:3060] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:13.615010  8474 raft_consensus.cc:2468] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 3.
I20260709 13:09:13.615137  8553 raft_consensus.cc:2468] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 3.
I20260709 13:09:13.616698  8592 leader_election.cc:304] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:13.617493  8801 raft_consensus.cc:2804] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:13.618012  8801 raft_consensus.cc:697] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 3 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:13.619006  8801 consensus_queue.cc:237] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.631477  8376 catalog_manager.cc:5697] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 2 to 3, leader changed from 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130) to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 3 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:13.701825  8781 raft_consensus.cc:993] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e: : Instructing follower acb23023045448a0a9eb67cbc2a361e3 to start an election
I20260709 13:09:13.702464  8905 raft_consensus.cc:1081] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Signalling peer acb23023045448a0a9eb67cbc2a361e3 to start an election
I20260709 13:09:13.704394  8624 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0906217e5d2b4ab08f5d25f61dd28cb5"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
 from {username='slave'} at 127.0.0.1:43744
I20260709 13:09:13.704948  8624 raft_consensus.cc:493] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:13.705283  8624 raft_consensus.cc:3060] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:13.705170  8891 raft_consensus.cc:993] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb: : Instructing follower acb23023045448a0a9eb67cbc2a361e3 to start an election
I20260709 13:09:13.706056  8884 raft_consensus.cc:1081] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Signalling peer acb23023045448a0a9eb67cbc2a361e3 to start an election
I20260709 13:09:13.707980  8623 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "847a35bccaa640e1b0c3f4c26aac3793"
dest_uuid: "acb23023045448a0a9eb67cbc2a361e3"
 from {username='slave'} at 127.0.0.1:43736
I20260709 13:09:13.708554  8623 raft_consensus.cc:493] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:13.708964  8623 raft_consensus.cc:3060] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:13.713325  8624 raft_consensus.cc:515] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.715818  8624 leader_election.cc:290] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 2 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:13.716683  8474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0906217e5d2b4ab08f5d25f61dd28cb5" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:13.716845  8553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0906217e5d2b4ab08f5d25f61dd28cb5" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:13.717389  8474 raft_consensus.cc:3055] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:13.717523  8553 raft_consensus.cc:3060] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:13.717814  8474 raft_consensus.cc:740] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3196546f9f674c63b347c608d8d7c81e, State: Running, Role: LEADER
I20260709 13:09:13.717931  8623 raft_consensus.cc:515] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.718921  8474 consensus_queue.cc:260] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.720916  8476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847a35bccaa640e1b0c3f4c26aac3793" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3196546f9f674c63b347c608d8d7c81e"
I20260709 13:09:13.721333  8474 raft_consensus.cc:3060] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:13.721593  8476 raft_consensus.cc:3060] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:13.721827  8550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847a35bccaa640e1b0c3f4c26aac3793" candidate_uuid: "acb23023045448a0a9eb67cbc2a361e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5ec5ac80f9fd413a957364f29ec50feb"
I20260709 13:09:13.722456  8550 raft_consensus.cc:3055] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:13.722752  8550 raft_consensus.cc:740] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ec5ac80f9fd413a957364f29ec50feb, State: Running, Role: LEADER
I20260709 13:09:13.723613  8550 consensus_queue.cc:260] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.724800  8550 raft_consensus.cc:3060] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:13.725054  8553 raft_consensus.cc:2468] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 2.
I20260709 13:09:13.726750  8592 leader_election.cc:304] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec5ac80f9fd413a957364f29ec50feb, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:13.727555  8801 raft_consensus.cc:2804] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:13.728065  8801 raft_consensus.cc:697] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 2 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:13.720296  8623 leader_election.cc:290] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 2 election: Requested vote from peers 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551), 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757)
I20260709 13:09:13.729226  8474 raft_consensus.cc:2468] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 2.
I20260709 13:09:13.729127  8801 consensus_queue.cc:237] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.729301  8476 raft_consensus.cc:2468] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 2.
I20260709 13:09:13.733282  8550 raft_consensus.cc:2468] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb23023045448a0a9eb67cbc2a361e3 in term 2.
I20260709 13:09:13.733186  8592 leader_election.cc:304] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3196546f9f674c63b347c608d8d7c81e, acb23023045448a0a9eb67cbc2a361e3; no voters: 
I20260709 13:09:13.734591  8890 raft_consensus.cc:2804] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:13.735201  8890 raft_consensus.cc:697] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 2 LEADER]: Becoming Leader. State: Replica: acb23023045448a0a9eb67cbc2a361e3, State: Running, Role: LEADER
I20260709 13:09:13.736713  8890 consensus_queue.cc:237] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } }
I20260709 13:09:13.741432  8376 catalog_manager.cc:5697] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 1 to 2, leader changed from 3196546f9f674c63b347c608d8d7c81e (127.27.71.129) to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:13.746411  8476 raft_consensus.cc:1275] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Refusing update from remote peer acb23023045448a0a9eb67cbc2a361e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:13.748170  8890 consensus_queue.cc:1048] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:13.760133  8373 catalog_manager.cc:5697] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 reported cstate change: term changed from 1 to 2, leader changed from 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130) to acb23023045448a0a9eb67cbc2a361e3 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "acb23023045448a0a9eb67cbc2a361e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb23023045448a0a9eb67cbc2a361e3" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 40041 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:13.764423  8550 raft_consensus.cc:1275] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Refusing update from remote peer acb23023045448a0a9eb67cbc2a361e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:13.766081  8923 consensus_queue.cc:1048] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:14.177428  8476 raft_consensus.cc:1275] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 3 FOLLOWER]: Refusing update from remote peer acb23023045448a0a9eb67cbc2a361e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:09:14.178836  8923 consensus_queue.cc:1048] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:14.189874  8550 raft_consensus.cc:1275] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 3 FOLLOWER]: Refusing update from remote peer acb23023045448a0a9eb67cbc2a361e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:09:14.194371  8890 consensus_queue.cc:1048] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:14.234503  8553 raft_consensus.cc:1275] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Refusing update from remote peer acb23023045448a0a9eb67cbc2a361e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:14.236200  8923 consensus_queue.cc:1048] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:14.248941  8476 raft_consensus.cc:1275] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Refusing update from remote peer acb23023045448a0a9eb67cbc2a361e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:14.250814  8923 consensus_queue.cc:1048] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:14.259050  8474 raft_consensus.cc:1275] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Refusing update from remote peer acb23023045448a0a9eb67cbc2a361e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:14.260990  8923 consensus_queue.cc:1048] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3196546f9f674c63b347c608d8d7c81e" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:14.277120  8553 raft_consensus.cc:1275] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Refusing update from remote peer acb23023045448a0a9eb67cbc2a361e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:14.278939  8923 consensus_queue.cc:1048] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec5ac80f9fd413a957364f29ec50feb" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:14.382273 27934 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:09:14.387691 27934 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:09:14.388202 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:14.393684 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:09:14.394052 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:15.437515 27934 tablet_server.cc:179] TabletServer@127.27.71.129:0 shutting down...
W20260709 13:09:15.458107  8518 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 13:09:15.464187  8518 consensus_peers.cc:597] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:15.467551  8518 consensus_peers.cc:597] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:15.484108  8518 consensus_peers.cc:597] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:15.530064 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:15.533033 27934 tablet_replica.cc:333] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.537113 27934 raft_consensus.cc:2243] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.538213 27934 raft_consensus.cc:2272] T 4197c71b8306458e9b7ccb0af7c6f253 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:15.539718  8592 consensus_peers.cc:597] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:15.542016 27934 tablet_replica.cc:333] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.543226 27934 raft_consensus.cc:2243] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.543905 27934 raft_consensus.cc:2272] T 8e3a55bd4e5d492cb37217100e25267a P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.547380 27934 tablet_replica.cc:333] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.548516 27934 raft_consensus.cc:2243] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.549216 27934 raft_consensus.cc:2272] T e73babad456c45d3b43b5372ab11d91d P 3196546f9f674c63b347c608d8d7c81e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.553841 27934 tablet_replica.cc:333] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.555734 27934 raft_consensus.cc:2243] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.556842 27934 raft_consensus.cc:2272] T cdcbea82a6134a298c338293a58e5490 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.561303 27934 tablet_replica.cc:333] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.563109 27934 raft_consensus.cc:2243] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.564693 27934 raft_consensus.cc:2272] T b11a0be526a648f18570f48676d10618 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.571895 27934 tablet_replica.cc:333] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.573077 27934 raft_consensus.cc:2243] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.573670 27934 raft_consensus.cc:2272] T 2f57d57d28bd40148450355438278216 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.576448 27934 tablet_replica.cc:333] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.577149 27934 raft_consensus.cc:2243] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.578261 27934 raft_consensus.cc:2272] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.581972 27934 tablet_replica.cc:333] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.583004 27934 raft_consensus.cc:2243] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.584252 27934 raft_consensus.cc:2272] T 1704f9c82d24421cb4444772e150092d P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.588433 27934 tablet_replica.cc:333] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.589277 27934 raft_consensus.cc:2243] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.589869 27934 raft_consensus.cc:2272] T be2c3b332154426388d8e090ce588f2f P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.595692 27934 tablet_replica.cc:333] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.596971 27934 raft_consensus.cc:2243] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.598461 27934 raft_consensus.cc:2272] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.602278 27934 tablet_replica.cc:333] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.603216 27934 raft_consensus.cc:2243] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.603900 27934 raft_consensus.cc:2272] T 6a309382875f481da2f0cc0f428599ce P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.607959 27934 tablet_replica.cc:333] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.608968 27934 raft_consensus.cc:2243] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.609615 27934 raft_consensus.cc:2272] T c1c56c3b74ac4b26b51002ad7f31adc1 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.612437 27934 tablet_replica.cc:333] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.613240 27934 raft_consensus.cc:2243] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.614013 27934 raft_consensus.cc:2272] T 715c5d36abd94180a1febc2638238061 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.617586 27934 tablet_replica.cc:333] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.618302 27934 raft_consensus.cc:2243] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.618904 27934 raft_consensus.cc:2272] T ecb95cc556024fbf86a589163f93093d P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.622146 27934 tablet_replica.cc:333] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.622896 27934 raft_consensus.cc:2243] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.624145 27934 raft_consensus.cc:2272] T 0befb22a314d4548b054ecae7a41f369 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.627106 27934 tablet_replica.cc:333] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.627805 27934 raft_consensus.cc:2243] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.628998 27934 raft_consensus.cc:2272] T f8851f8110794e71b484de5b98a34b51 P 3196546f9f674c63b347c608d8d7c81e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.632045 27934 tablet_replica.cc:333] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.632670 27934 raft_consensus.cc:2243] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.633164 27934 raft_consensus.cc:2272] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.635895 27934 tablet_replica.cc:333] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e: stopping tablet replica
I20260709 13:09:15.636554 27934 raft_consensus.cc:2243] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.637216 27934 raft_consensus.cc:2272] T 847a35bccaa640e1b0c3f4c26aac3793 P 3196546f9f674c63b347c608d8d7c81e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:15.660626  8518 consensus_peers.cc:597] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:15.694005 27934 tablet_server.cc:196] TabletServer@127.27.71.129:0 shutdown complete.
I20260709 13:09:15.761441 27934 tablet_server.cc:179] TabletServer@127.27.71.130:0 shutting down...
W20260709 13:09:15.786715  8592 consensus_peers.cc:597] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:36551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:15.818188  8592 consensus_peers.cc:597] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:36551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:15.825284  8592 consensus_peers.cc:597] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 -> Peer 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757): Couldn't send request to peer 5ec5ac80f9fd413a957364f29ec50feb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:15.838734 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:09:15.839877  8518 consensus_peers.cc:597] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:36551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:15.840135 27934 tablet_replica.cc:333] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.841240 27934 raft_consensus.cc:2243] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.841917 27934 raft_consensus.cc:2272] T 4197c71b8306458e9b7ccb0af7c6f253 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.844755 27934 tablet_replica.cc:333] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.856252 27934 raft_consensus.cc:2243] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.857748 27934 raft_consensus.cc:2272] T 8e3a55bd4e5d492cb37217100e25267a P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:15.862094  8518 consensus_peers.cc:597] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:36551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:15.862481 27934 tablet_replica.cc:333] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.863404 27934 raft_consensus.cc:2243] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.864032 27934 raft_consensus.cc:2272] T e73babad456c45d3b43b5372ab11d91d P 5ec5ac80f9fd413a957364f29ec50feb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.875785 27934 tablet_replica.cc:333] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.877091 27934 raft_consensus.cc:2243] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.878510 27934 raft_consensus.cc:2272] T cdcbea82a6134a298c338293a58e5490 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:15.878638  8592 consensus_peers.cc:597] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:36551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:15.882444  8592 consensus_peers.cc:597] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:36551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:15.883466 27934 tablet_replica.cc:333] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.885186 27934 raft_consensus.cc:2243] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.886538 27934 raft_consensus.cc:2272] T 2f57d57d28bd40148450355438278216 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.889393 27934 tablet_replica.cc:333] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.890043 27934 raft_consensus.cc:2243] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.890667 27934 raft_consensus.cc:2272] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.895864 27934 tablet_replica.cc:333] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.896703 27934 raft_consensus.cc:2243] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.898398 27934 raft_consensus.cc:2272] T 6a309382875f481da2f0cc0f428599ce P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.902719 27934 tablet_replica.cc:333] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.903824 27934 raft_consensus.cc:2243] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.905162 27934 raft_consensus.cc:2272] T c1c56c3b74ac4b26b51002ad7f31adc1 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:15.909426  8592 consensus_peers.cc:597] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 -> Peer 3196546f9f674c63b347c608d8d7c81e (127.27.71.129:36551): Couldn't send request to peer 3196546f9f674c63b347c608d8d7c81e. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:36551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:15.914547 27934 tablet_replica.cc:333] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.916363 27934 raft_consensus.cc:2243] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.917640 27934 raft_consensus.cc:2272] T 715c5d36abd94180a1febc2638238061 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.920616 27934 tablet_replica.cc:333] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.921541 27934 raft_consensus.cc:2243] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.922281 27934 raft_consensus.cc:2272] T fb1a6adb6ddc49b4be091bc5f654f2a7 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.925856 27934 tablet_replica.cc:333] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.926637 27934 raft_consensus.cc:2243] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.927523 27934 raft_consensus.cc:2272] T 1704f9c82d24421cb4444772e150092d P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.933657 27934 tablet_replica.cc:333] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.934540 27934 raft_consensus.cc:2243] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:15.936096 27934 raft_consensus.cc:2272] T be2c3b332154426388d8e090ce588f2f P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.942152 27934 tablet_replica.cc:333] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.943156 27934 raft_consensus.cc:2243] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.945196 27934 raft_consensus.cc:2272] T 0befb22a314d4548b054ecae7a41f369 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.948958 27934 tablet_replica.cc:333] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.949846 27934 raft_consensus.cc:2243] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.950500 27934 raft_consensus.cc:2272] T ecb95cc556024fbf86a589163f93093d P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.954267 27934 tablet_replica.cc:333] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.955082 27934 raft_consensus.cc:2243] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.955600 27934 raft_consensus.cc:2272] T 847a35bccaa640e1b0c3f4c26aac3793 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.958220 27934 tablet_replica.cc:333] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.958815 27934 raft_consensus.cc:2243] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.959286 27934 raft_consensus.cc:2272] T 0906217e5d2b4ab08f5d25f61dd28cb5 P 5ec5ac80f9fd413a957364f29ec50feb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.961530 27934 tablet_replica.cc:333] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.962100 27934 raft_consensus.cc:2243] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.962646 27934 raft_consensus.cc:2272] T b11a0be526a648f18570f48676d10618 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.965080 27934 tablet_replica.cc:333] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb: stopping tablet replica
I20260709 13:09:15.965791 27934 raft_consensus.cc:2243] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.966303 27934 raft_consensus.cc:2272] T f8851f8110794e71b484de5b98a34b51 P 5ec5ac80f9fd413a957364f29ec50feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.015368 27934 tablet_server.cc:196] TabletServer@127.27.71.130:0 shutdown complete.
W20260709 13:09:16.022138  8592 consensus_peers.cc:597] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 -> Peer 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757): Couldn't send request to peer 5ec5ac80f9fd413a957364f29ec50feb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:45757: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.063841 27934 tablet_server.cc:179] TabletServer@127.27.71.131:0 shutting down...
I20260709 13:09:16.107703 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:16.108989 27934 tablet_replica.cc:333] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.109845 27934 raft_consensus.cc:2243] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.110473 27934 raft_consensus.cc:2272] T 8e3a55bd4e5d492cb37217100e25267a P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.114209 27934 tablet_replica.cc:333] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.115664 27934 raft_consensus.cc:2243] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.116885 27934 raft_consensus.cc:2272] T 4197c71b8306458e9b7ccb0af7c6f253 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.120455 27934 tablet_replica.cc:333] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.121634 27934 raft_consensus.cc:2243] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.122673 27934 raft_consensus.cc:2272] T e73babad456c45d3b43b5372ab11d91d P acb23023045448a0a9eb67cbc2a361e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.125658 27934 tablet_replica.cc:333] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.126559 27934 raft_consensus.cc:2243] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:09:16.127271  8592 consensus_peers.cc:597] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 -> Peer 5ec5ac80f9fd413a957364f29ec50feb (127.27.71.130:45757): Couldn't send request to peer 5ec5ac80f9fd413a957364f29ec50feb. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:45757: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.127543 27934 raft_consensus.cc:2272] T cdcbea82a6134a298c338293a58e5490 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.129851 27934 tablet_replica.cc:333] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.130535 27934 raft_consensus.cc:2243] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.131026 27934 raft_consensus.cc:2272] T b11a0be526a648f18570f48676d10618 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.133179 27934 tablet_replica.cc:333] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.133796 27934 raft_consensus.cc:2243] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.134270 27934 raft_consensus.cc:2272] T 2f57d57d28bd40148450355438278216 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.136515 27934 tablet_replica.cc:333] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.137101 27934 raft_consensus.cc:2243] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.137598 27934 raft_consensus.cc:2272] T 63d2d2a3a2cc4a30bb446d2ea2548a3c P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.139758 27934 tablet_replica.cc:333] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.140445 27934 raft_consensus.cc:2243] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.140930 27934 raft_consensus.cc:2272] T 1704f9c82d24421cb4444772e150092d P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.143153 27934 tablet_replica.cc:333] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.143795 27934 raft_consensus.cc:2243] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.144353 27934 raft_consensus.cc:2272] T be2c3b332154426388d8e090ce588f2f P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.147537 27934 tablet_replica.cc:333] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.148447 27934 raft_consensus.cc:2243] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.148963 27934 raft_consensus.cc:2272] T fb1a6adb6ddc49b4be091bc5f654f2a7 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.151409 27934 tablet_replica.cc:333] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.152102 27934 raft_consensus.cc:2243] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.152604 27934 raft_consensus.cc:2272] T 6a309382875f481da2f0cc0f428599ce P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.154856 27934 tablet_replica.cc:333] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.155486 27934 raft_consensus.cc:2243] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.156088 27934 raft_consensus.cc:2272] T c1c56c3b74ac4b26b51002ad7f31adc1 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.158824 27934 tablet_replica.cc:333] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.159507 27934 raft_consensus.cc:2243] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.160005 27934 raft_consensus.cc:2272] T f8851f8110794e71b484de5b98a34b51 P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.162317 27934 tablet_replica.cc:333] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.162930 27934 raft_consensus.cc:2243] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.163805 27934 raft_consensus.cc:2272] T 0906217e5d2b4ab08f5d25f61dd28cb5 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.166124 27934 tablet_replica.cc:333] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.166728 27934 raft_consensus.cc:2243] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.167577 27934 raft_consensus.cc:2272] T 847a35bccaa640e1b0c3f4c26aac3793 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.169898 27934 tablet_replica.cc:333] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.170445 27934 raft_consensus.cc:2243] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.170962 27934 raft_consensus.cc:2272] T 715c5d36abd94180a1febc2638238061 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.173305 27934 tablet_replica.cc:333] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.173918 27934 raft_consensus.cc:2243] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.174880 27934 raft_consensus.cc:2272] T ecb95cc556024fbf86a589163f93093d P acb23023045448a0a9eb67cbc2a361e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.177639 27934 tablet_replica.cc:333] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3: stopping tablet replica
I20260709 13:09:16.178439 27934 raft_consensus.cc:2243] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:16.179052 27934 raft_consensus.cc:2272] T 0befb22a314d4548b054ecae7a41f369 P acb23023045448a0a9eb67cbc2a361e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.233328 27934 tablet_server.cc:196] TabletServer@127.27.71.131:0 shutdown complete.
I20260709 13:09:16.281466 27934 master.cc:562] Master@127.27.71.190:37267 shutting down...
I20260709 13:09:16.316195 27934 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:16.316860 27934 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:16.317245 27934 tablet_replica.cc:333] T 00000000000000000000000000000000 P 62e53dabbcf34c36aba774737ee67a2e: stopping tablet replica
I20260709 13:09:16.342011 27934 master.cc:584] Master@127.27.71.190:37267 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13563 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:09:16.409011 27934 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.71.190:44109
I20260709 13:09:16.410260 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:16.417578  8940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:16.418538  8939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:16.419585  8942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:16.421172 27934 server_base.cc:1061] running on GCE node
I20260709 13:09:16.422240 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:16.422425 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:16.422531 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602556422520 us; error 0 us; skew 500 ppm
I20260709 13:09:16.423010 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:16.427176 27934 webserver.cc:533] Webserver started at http://127.27.71.190:45815/ using document root <none> and password file <none>
I20260709 13:09:16.427857 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:16.428216 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:16.428469 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:16.429577 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/master-0-root/instance:
uuid: "23cd56ebed5841a8ab5d236a3267abd3"
format_stamp: "Formatted at 2026-07-09 13:09:16 on dist-test-slave-mlt7"
I20260709 13:09:16.435261 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:09:16.439561  8947 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:16.440467 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:09:16.440727 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/master-0-root
uuid: "23cd56ebed5841a8ab5d236a3267abd3"
format_stamp: "Formatted at 2026-07-09 13:09:16 on dist-test-slave-mlt7"
I20260709 13:09:16.440968 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:16.486096 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:16.487437 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:16.530469 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.190:44109
I20260709 13:09:16.530670  8998 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.190:44109 every 8 connection(s)
I20260709 13:09:16.535656  8999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:16.549844  8999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3: Bootstrap starting.
I20260709 13:09:16.555465  8999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:16.560465  8999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3: No bootstrap required, opened a new log
I20260709 13:09:16.563139  8999 raft_consensus.cc:359] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cd56ebed5841a8ab5d236a3267abd3" member_type: VOTER }
I20260709 13:09:16.563576  8999 raft_consensus.cc:385] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:16.563758  8999 raft_consensus.cc:740] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cd56ebed5841a8ab5d236a3267abd3, State: Initialized, Role: FOLLOWER
I20260709 13:09:16.564431  8999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cd56ebed5841a8ab5d236a3267abd3" member_type: VOTER }
I20260709 13:09:16.564858  8999 raft_consensus.cc:399] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:09:16.565052  8999 raft_consensus.cc:493] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:09:16.565263  8999 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:16.571182  8999 raft_consensus.cc:515] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cd56ebed5841a8ab5d236a3267abd3" member_type: VOTER }
I20260709 13:09:16.571728  8999 leader_election.cc:304] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23cd56ebed5841a8ab5d236a3267abd3; no voters: 
I20260709 13:09:16.573169  8999 leader_election.cc:290] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:09:16.573858  9002 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:16.575208  9002 raft_consensus.cc:697] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [term 1 LEADER]: Becoming Leader. State: Replica: 23cd56ebed5841a8ab5d236a3267abd3, State: Running, Role: LEADER
I20260709 13:09:16.575891  9002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cd56ebed5841a8ab5d236a3267abd3" member_type: VOTER }
I20260709 13:09:16.576588  8999 sys_catalog.cc:565] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:09:16.581797  9003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23cd56ebed5841a8ab5d236a3267abd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cd56ebed5841a8ab5d236a3267abd3" member_type: VOTER } }
I20260709 13:09:16.582762  9003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:16.583050  9004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23cd56ebed5841a8ab5d236a3267abd3. Latest consensus state: current_term: 1 leader_uuid: "23cd56ebed5841a8ab5d236a3267abd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cd56ebed5841a8ab5d236a3267abd3" member_type: VOTER } }
I20260709 13:09:16.584025  9004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [sys.catalog]: This master's current role is: LEADER
I20260709 13:09:16.585812  9009 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:09:16.591754  9009 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:09:16.597601 27934 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:09:16.603472  9009 catalog_manager.cc:1383] Generated new cluster ID: 394d0bbe15664128a373c6d0a638b402
I20260709 13:09:16.603849  9009 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:09:16.627604  9009 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:09:16.629269  9009 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:09:16.645262  9009 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3: Generated new TSK 0
I20260709 13:09:16.646258  9009 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:09:16.665596 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:16.672914  9020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:16.673925  9021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:16.675225  9023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:16.676563 27934 server_base.cc:1061] running on GCE node
I20260709 13:09:16.677662 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:16.677850 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:16.677999 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602556677982 us; error 0 us; skew 500 ppm
I20260709 13:09:16.678654 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:16.681291 27934 webserver.cc:533] Webserver started at http://127.27.71.129:45333/ using document root <none> and password file <none>
I20260709 13:09:16.681790 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:16.681957 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:16.682267 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:16.683395 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-0-root/instance:
uuid: "fcb5f9589eb7482cab677f08b73602a1"
format_stamp: "Formatted at 2026-07-09 13:09:16 on dist-test-slave-mlt7"
I20260709 13:09:16.689141 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:09:16.693171  9028 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:16.694114 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:09:16.694532 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-0-root
uuid: "fcb5f9589eb7482cab677f08b73602a1"
format_stamp: "Formatted at 2026-07-09 13:09:16 on dist-test-slave-mlt7"
I20260709 13:09:16.694989 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:16.712129 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:16.713629 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:16.715358 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:16.717973 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:16.718235 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:16.718511 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:16.718710 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:16.778400 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.129:37425
I20260709 13:09:16.778748  9090 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.129:37425 every 8 connection(s)
I20260709 13:09:16.784590 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:16.793069  9095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:16.793804  9096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:16.797035 27934 server_base.cc:1061] running on GCE node
W20260709 13:09:16.797320  9099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:16.799321 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:16.799595 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:16.799809 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602556799788 us; error 0 us; skew 500 ppm
I20260709 13:09:16.800469 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:16.802433  9091 heartbeater.cc:344] Connected to a master server at 127.27.71.190:44109
I20260709 13:09:16.802839  9091 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:16.803635 27934 webserver.cc:533] Webserver started at http://127.27.71.130:38799/ using document root <none> and password file <none>
I20260709 13:09:16.803881  9091 heartbeater.cc:507] Master 127.27.71.190:44109 requested a full tablet report, sending...
I20260709 13:09:16.804416 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:16.804698 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:16.805051 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:16.806493 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-1-root/instance:
uuid: "75f16c06c14045a5b104c416d3fd6242"
format_stamp: "Formatted at 2026-07-09 13:09:16 on dist-test-slave-mlt7"
I20260709 13:09:16.806833  8964 ts_manager.cc:194] Registered new tserver with Master: fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425)
I20260709 13:09:16.809015  8964 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50308
I20260709 13:09:16.811725 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:09:16.816083  9103 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:16.817013 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:09:16.817346 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-1-root
uuid: "75f16c06c14045a5b104c416d3fd6242"
format_stamp: "Formatted at 2026-07-09 13:09:16 on dist-test-slave-mlt7"
I20260709 13:09:16.817704 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:16.829423 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:16.830765 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:16.832420 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:16.834739 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:16.834990 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:16.835244 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:16.835425 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:16.885545 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.130:34911
I20260709 13:09:16.885691  9165 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.130:34911 every 8 connection(s)
I20260709 13:09:16.890739 27934 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:16.899129  9170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:16.899132  9169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:16.902804  9166 heartbeater.cc:344] Connected to a master server at 127.27.71.190:44109
I20260709 13:09:16.903317  9166 heartbeater.cc:461] Registering TS with master...
W20260709 13:09:16.903671  9172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:16.904332  9166 heartbeater.cc:507] Master 127.27.71.190:44109 requested a full tablet report, sending...
I20260709 13:09:16.906447  8964 ts_manager.cc:194] Registered new tserver with Master: 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:16.908200  8964 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50318
I20260709 13:09:16.909016 27934 server_base.cc:1061] running on GCE node
I20260709 13:09:16.910092 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:16.910336 27934 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:16.910523 27934 hybrid_clock.cc:648] HybridClock initialized: now 1783602556910509 us; error 0 us; skew 500 ppm
I20260709 13:09:16.911074 27934 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:16.914304 27934 webserver.cc:533] Webserver started at http://127.27.71.131:41011/ using document root <none> and password file <none>
I20260709 13:09:16.914831 27934 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:16.915057 27934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:16.915335 27934 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:16.916630 27934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-2-root/instance:
uuid: "dad482c12e774a2a8df70dc54dde1822"
format_stamp: "Formatted at 2026-07-09 13:09:16 on dist-test-slave-mlt7"
I20260709 13:09:16.921558 27934 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:09:16.925979  9177 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:16.926972 27934 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:09:16.927309 27934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-2-root
uuid: "dad482c12e774a2a8df70dc54dde1822"
format_stamp: "Formatted at 2026-07-09 13:09:16 on dist-test-slave-mlt7"
I20260709 13:09:16.927677 27934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskCzDK0N/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602179482400-27934-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:16.947419 27934 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:16.948773 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:16.950099 27934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:16.952463 27934 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:16.952705 27934 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:16.952952 27934 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:16.953125 27934 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:16.999814 27934 rpc_server.cc:307] RPC server started. Bound to: 127.27.71.131:36547
I20260709 13:09:17.000006  9239 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.71.131:36547 every 8 connection(s)
I20260709 13:09:17.015352  9240 heartbeater.cc:344] Connected to a master server at 127.27.71.190:44109
I20260709 13:09:17.015810  9240 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:17.016673  9240 heartbeater.cc:507] Master 127.27.71.190:44109 requested a full tablet report, sending...
I20260709 13:09:17.018850  8964 ts_manager.cc:194] Registered new tserver with Master: dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.019126 27934 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015709754s
I20260709 13:09:17.020527  8964 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50330
I20260709 13:09:17.051376  8964 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50334:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:17.054913  8964 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:17.133445  9056 tablet_service.cc:1511] Processing CreateTablet for tablet 3e01e0329e284d0b9f423618165717b5 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:17.135025  9056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e01e0329e284d0b9f423618165717b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.137920  9055 tablet_service.cc:1511] Processing CreateTablet for tablet 916527a7aa6340e092512d7175e79017 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:17.138782  9205 tablet_service.cc:1511] Processing CreateTablet for tablet 3e01e0329e284d0b9f423618165717b5 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:17.139509  9055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 916527a7aa6340e092512d7175e79017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.140187  9205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e01e0329e284d0b9f423618165717b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.169353  9204 tablet_service.cc:1511] Processing CreateTablet for tablet 916527a7aa6340e092512d7175e79017 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:17.171209  9204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 916527a7aa6340e092512d7175e79017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.169468  9255 tablet_bootstrap.cc:492] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:17.170749  9056 tablet_service.cc:1511] Processing CreateTablet for tablet b1b7213d7986410cb9d070e4e36e1b62 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:17.172847  9056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b7213d7986410cb9d070e4e36e1b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.181777  9255 tablet_bootstrap.cc:654] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.185544  9205 tablet_service.cc:1511] Processing CreateTablet for tablet b1b7213d7986410cb9d070e4e36e1b62 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:17.187263  9205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b7213d7986410cb9d070e4e36e1b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.188339  9055 tablet_service.cc:1511] Processing CreateTablet for tablet ac243bc7f6234c60951698df8322d1c0 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:17.189782  9055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac243bc7f6234c60951698df8322d1c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.191201  9203 tablet_service.cc:1511] Processing CreateTablet for tablet ac243bc7f6234c60951698df8322d1c0 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:17.192799  9203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac243bc7f6234c60951698df8322d1c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.196451  9054 tablet_service.cc:1511] Processing CreateTablet for tablet 7eb3a7f451bb463d87fb2f927470dd7f (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:17.197949  9054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eb3a7f451bb463d87fb2f927470dd7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.198984  9202 tablet_service.cc:1511] Processing CreateTablet for tablet 7eb3a7f451bb463d87fb2f927470dd7f (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:17.200675  9202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eb3a7f451bb463d87fb2f927470dd7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.202975  9053 tablet_service.cc:1511] Processing CreateTablet for tablet 0a1f95384455460a840a18f5746aaae8 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:17.183368  9256 tablet_bootstrap.cc:492] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:17.204707  9053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1f95384455460a840a18f5746aaae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.204855  9201 tablet_service.cc:1511] Processing CreateTablet for tablet 0a1f95384455460a840a18f5746aaae8 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:17.206499  9201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1f95384455460a840a18f5746aaae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.207091  9052 tablet_service.cc:1511] Processing CreateTablet for tablet 7c1edfd2067f45a789de9f93b37d683b (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:17.208606  9052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c1edfd2067f45a789de9f93b37d683b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.210189  9200 tablet_service.cc:1511] Processing CreateTablet for tablet 7c1edfd2067f45a789de9f93b37d683b (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:17.214175  9051 tablet_service.cc:1511] Processing CreateTablet for tablet ff8fc36481524d908afb6d9debb715ef (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:17.216138  9051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8fc36481524d908afb6d9debb715ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.216362  9200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c1edfd2067f45a789de9f93b37d683b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.218820  9199 tablet_service.cc:1511] Processing CreateTablet for tablet ff8fc36481524d908afb6d9debb715ef (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:17.221064  9199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8fc36481524d908afb6d9debb715ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.221184  9050 tablet_service.cc:1511] Processing CreateTablet for tablet d78d91e5ea574eeb99069dda72c76bac (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:17.222673  9050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d78d91e5ea574eeb99069dda72c76bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.223846  9198 tablet_service.cc:1511] Processing CreateTablet for tablet d78d91e5ea574eeb99069dda72c76bac (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:17.225490  9198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d78d91e5ea574eeb99069dda72c76bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.253252  9131 tablet_service.cc:1511] Processing CreateTablet for tablet 3e01e0329e284d0b9f423618165717b5 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:17.254813  9131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e01e0329e284d0b9f423618165717b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.257130  9130 tablet_service.cc:1511] Processing CreateTablet for tablet 916527a7aa6340e092512d7175e79017 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:17.258617  9130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 916527a7aa6340e092512d7175e79017. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.261168  9129 tablet_service.cc:1511] Processing CreateTablet for tablet b1b7213d7986410cb9d070e4e36e1b62 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:17.262758  9129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b7213d7986410cb9d070e4e36e1b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.263427  9256 tablet_bootstrap.cc:654] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.265302  9127 tablet_service.cc:1511] Processing CreateTablet for tablet 7eb3a7f451bb463d87fb2f927470dd7f (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:17.266942  9127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7eb3a7f451bb463d87fb2f927470dd7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.268991  9128 tablet_service.cc:1511] Processing CreateTablet for tablet ac243bc7f6234c60951698df8322d1c0 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:17.280122  9255 tablet_bootstrap.cc:492] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:17.280725  9255 ts_tablet_manager.cc:1403] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.112s	user 0.013s	sys 0.008s
I20260709 13:09:17.283161  9126 tablet_service.cc:1511] Processing CreateTablet for tablet 0a1f95384455460a840a18f5746aaae8 (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:17.284076  9255 raft_consensus.cc:359] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.284950  9255 raft_consensus.cc:385] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.285312  9255 raft_consensus.cc:740] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.285617  9125 tablet_service.cc:1511] Processing CreateTablet for tablet 7c1edfd2067f45a789de9f93b37d683b (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:17.286151  9255 consensus_queue.cc:260] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.288745  9128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac243bc7f6234c60951698df8322d1c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.288540  9124 tablet_service.cc:1511] Processing CreateTablet for tablet ff8fc36481524d908afb6d9debb715ef (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:17.289682  9123 tablet_service.cc:1511] Processing CreateTablet for tablet d78d91e5ea574eeb99069dda72c76bac (DEFAULT_TABLE table=test-workload [id=b333d0b842dc4f78a29dd05dfa9be36e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:17.292373  9124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8fc36481524d908afb6d9debb715ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.293183  9126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1f95384455460a840a18f5746aaae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.296378  9125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c1edfd2067f45a789de9f93b37d683b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.299124  9123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d78d91e5ea574eeb99069dda72c76bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.323855  9258 tablet_bootstrap.cc:492] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:17.331049  9258 tablet_bootstrap.cc:654] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.347523  9256 tablet_bootstrap.cc:492] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:17.348397  9256 ts_tablet_manager.cc:1403] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.165s	user 0.021s	sys 0.000s
I20260709 13:09:17.351606  9256 raft_consensus.cc:359] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } }
I20260709 13:09:17.352452  9256 raft_consensus.cc:385] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.352794  9256 raft_consensus.cc:740] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.353807  9256 consensus_queue.cc:260] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } }
I20260709 13:09:17.367033  9258 tablet_bootstrap.cc:492] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:17.367655  9258 ts_tablet_manager.cc:1403] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.044s	user 0.019s	sys 0.020s
I20260709 13:09:17.370970  9258 raft_consensus.cc:359] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.371712  9258 raft_consensus.cc:385] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.372025  9258 raft_consensus.cc:740] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.372891  9258 consensus_queue.cc:260] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.376498  9091 heartbeater.cc:499] Master 127.27.71.190:44109 was elected leader, sending a full tablet report...
I20260709 13:09:17.377377  9240 heartbeater.cc:499] Master 127.27.71.190:44109 was elected leader, sending a full tablet report...
I20260709 13:09:17.388487  9256 ts_tablet_manager.cc:1434] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.009s
I20260709 13:09:17.390511  9256 tablet_bootstrap.cc:492] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:17.394006  9258 ts_tablet_manager.cc:1434] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.019s
I20260709 13:09:17.395045  9258 tablet_bootstrap.cc:492] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:17.395316  9166 heartbeater.cc:499] Master 127.27.71.190:44109 was elected leader, sending a full tablet report...
I20260709 13:09:17.397569  9256 tablet_bootstrap.cc:654] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.389308  9255 ts_tablet_manager.cc:1434] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.108s	user 0.009s	sys 0.004s
I20260709 13:09:17.400076  9258 tablet_bootstrap.cc:654] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.400966  9255 tablet_bootstrap.cc:492] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:17.404222  9260 raft_consensus.cc:493] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.404806  9260 raft_consensus.cc:515] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.408883  9255 tablet_bootstrap.cc:654] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.411682  9256 tablet_bootstrap.cc:492] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:17.411973  9258 tablet_bootstrap.cc:492] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:17.412460  9256 ts_tablet_manager.cc:1403] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.013s
I20260709 13:09:17.412484  9258 ts_tablet_manager.cc:1403] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.018s	user 0.007s	sys 0.009s
I20260709 13:09:17.416312  9258 raft_consensus.cc:359] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } }
I20260709 13:09:17.416965  9260 leader_election.cc:290] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.418689  9256 raft_consensus.cc:359] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.419657  9256 raft_consensus.cc:385] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.420192  9256 raft_consensus.cc:740] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.417184  9258 raft_consensus.cc:385] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.421809  9258 raft_consensus.cc:740] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.423091  9258 consensus_queue.cc:260] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } }
I20260709 13:09:17.426362  9258 ts_tablet_manager.cc:1434] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:09:17.427672  9258 tablet_bootstrap.cc:492] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:17.421126  9256 consensus_queue.cc:260] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.435076  9256 ts_tablet_manager.cc:1434] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.002s
I20260709 13:09:17.436352  9256 tablet_bootstrap.cc:492] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:17.437456  9258 tablet_bootstrap.cc:654] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.441675  9255 tablet_bootstrap.cc:492] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:17.442546  9255 ts_tablet_manager.cc:1403] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.012s
I20260709 13:09:17.447965  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916527a7aa6340e092512d7175e79017" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:17.448904  9141 raft_consensus.cc:2468] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 0.
I20260709 13:09:17.449877  9255 raft_consensus.cc:359] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } }
I20260709 13:09:17.450950  9255 raft_consensus.cc:385] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.450840  9031 leader_election.cc:304] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:17.451532  9256 tablet_bootstrap.cc:654] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.451750  9255 raft_consensus.cc:740] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.452442  9260 raft_consensus.cc:2804] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:17.452978  9260 raft_consensus.cc:493] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.453374  9260 raft_consensus.cc:3060] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.453042  9255 consensus_queue.cc:260] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } }
I20260709 13:09:17.456776  9258 tablet_bootstrap.cc:492] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:17.457634  9258 ts_tablet_manager.cc:1403] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.006s
I20260709 13:09:17.462494  9258 raft_consensus.cc:359] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.463557  9258 raft_consensus.cc:385] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.463938  9258 raft_consensus.cc:740] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.464819  9258 consensus_queue.cc:260] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.468591  9258 ts_tablet_manager.cc:1434] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.001s
I20260709 13:09:17.469784  9256 tablet_bootstrap.cc:492] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:17.469991  9258 tablet_bootstrap.cc:492] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:17.470436  9256 ts_tablet_manager.cc:1403] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.007s
I20260709 13:09:17.473817  9256 raft_consensus.cc:359] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.474686  9256 raft_consensus.cc:385] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.474989  9256 raft_consensus.cc:740] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.475973  9256 consensus_queue.cc:260] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.478071  9258 tablet_bootstrap.cc:654] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.478725  9256 ts_tablet_manager.cc:1434] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:09:17.479987  9256 tablet_bootstrap.cc:492] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:17.489552  9256 tablet_bootstrap.cc:654] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.516453  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916527a7aa6340e092512d7175e79017" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
W20260709 13:09:17.519119  9032 leader_election.cc:343] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.521420  9260 raft_consensus.cc:515] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.533437  9263 raft_consensus.cc:493] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.534255  9263 raft_consensus.cc:515] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.536748  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916527a7aa6340e092512d7175e79017" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:17.537809  9141 raft_consensus.cc:3060] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.537962  9263 leader_election.cc:290] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.543982  9262 raft_consensus.cc:493] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.544997  9262 raft_consensus.cc:515] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.559957  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916527a7aa6340e092512d7175e79017" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:17.561363  9262 leader_election.cc:290] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
W20260709 13:09:17.562426  9032 leader_election.cc:343] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.564795  9260 leader_election.cc:290] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Requested vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.568640  9141 raft_consensus.cc:2468] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 1.
I20260709 13:09:17.573011  9255 ts_tablet_manager.cc:1434] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.126s	user 0.043s	sys 0.042s
I20260709 13:09:17.575520  9256 tablet_bootstrap.cc:492] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:17.576510  9256 ts_tablet_manager.cc:1403] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.097s	user 0.035s	sys 0.009s
I20260709 13:09:17.576731  9258 tablet_bootstrap.cc:492] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:17.577339  9258 ts_tablet_manager.cc:1403] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.108s	user 0.038s	sys 0.066s
I20260709 13:09:17.580513  9258 raft_consensus.cc:359] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.581475  9258 raft_consensus.cc:385] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.581655  9256 raft_consensus.cc:359] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.582633  9256 raft_consensus.cc:385] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.582518  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:17.583196  9256 raft_consensus.cc:740] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
W20260709 13:09:17.584275  9104 leader_election.cc:343] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.584606  9256 consensus_queue.cc:260] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.584720  9255 tablet_bootstrap.cc:492] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:17.587383  9256 ts_tablet_manager.cc:1434] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
W20260709 13:09:17.590515  9012 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:17.581784  9258 raft_consensus.cc:740] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.592437  9258 consensus_queue.cc:260] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.593888  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1f95384455460a840a18f5746aaae8" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:17.591578  9256 tablet_bootstrap.cc:492] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
W20260709 13:09:17.595481  9178 leader_election.cc:343] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.598098  9258 ts_tablet_manager.cc:1434] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:09:17.598685  9031 leader_election.cc:304] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: dad482c12e774a2a8df70dc54dde1822
I20260709 13:09:17.603224  9256 tablet_bootstrap.cc:654] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.605372  9258 tablet_bootstrap.cc:492] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:17.604614  9260 raft_consensus.cc:2804] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:17.608752  9260 raft_consensus.cc:697] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Becoming Leader. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:17.611625  9260 consensus_queue.cc:237] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.613927  9258 tablet_bootstrap.cc:654] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.618337  9255 tablet_bootstrap.cc:654] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.620850  9256 tablet_bootstrap.cc:492] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:17.621485  9256 ts_tablet_manager.cc:1403] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.008s
I20260709 13:09:17.625156  9258 tablet_bootstrap.cc:492] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:17.625790  9258 ts_tablet_manager.cc:1403] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.007s
I20260709 13:09:17.625284  9256 raft_consensus.cc:359] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.626216  9256 raft_consensus.cc:385] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.626549  9256 raft_consensus.cc:740] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.627565  9256 consensus_queue.cc:260] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.628795  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:17.628926  9258 raft_consensus.cc:359] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.629739  9258 raft_consensus.cc:385] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.630053  9258 raft_consensus.cc:740] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
W20260709 13:09:17.630400  9107 leader_election.cc:343] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.630927  9107 leader_election.cc:304] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242; no voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:17.630836  9258 consensus_queue.cc:260] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.632347  9263 raft_consensus.cc:2749] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:17.634323  9258 ts_tablet_manager.cc:1434] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260709 13:09:17.635370  9258 tablet_bootstrap.cc:492] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:17.635519  9256 ts_tablet_manager.cc:1434] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:09:17.636631  9256 tablet_bootstrap.cc:492] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:17.642110  9258 tablet_bootstrap.cc:654] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.643229  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1f95384455460a840a18f5746aaae8" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:17.643750  8964 catalog_manager.cc:5697] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "fcb5f9589eb7482cab677f08b73602a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:17.644795  9180 leader_election.cc:343] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.645315  9180 leader_election.cc:304] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822; no voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:17.646256  9262 raft_consensus.cc:2749] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:17.648341  9255 tablet_bootstrap.cc:492] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:17.648962  9255 ts_tablet_manager.cc:1403] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.064s	user 0.015s	sys 0.000s
I20260709 13:09:17.650122  9256 tablet_bootstrap.cc:654] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.652498  9255 raft_consensus.cc:359] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.653590  9255 raft_consensus.cc:385] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.654080  9255 raft_consensus.cc:740] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.655344  9255 consensus_queue.cc:260] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.659055  9255 ts_tablet_manager.cc:1434] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 13:09:17.660492  9255 tablet_bootstrap.cc:492] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:17.664719  9258 tablet_bootstrap.cc:492] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:17.665383  9258 ts_tablet_manager.cc:1403] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.003s
I20260709 13:09:17.667964  9260 raft_consensus.cc:493] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.668740  9260 raft_consensus.cc:515] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } }
I20260709 13:09:17.668766  9258 raft_consensus.cc:359] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.670120  9258 raft_consensus.cc:385] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.670508  9258 raft_consensus.cc:740] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.671669  9255 tablet_bootstrap.cc:654] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.671383  9258 consensus_queue.cc:260] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.672513  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e01e0329e284d0b9f423618165717b5" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:17.672712  9256 tablet_bootstrap.cc:492] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:17.675055  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e01e0329e284d0b9f423618165717b5" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:17.675357  9256 ts_tablet_manager.cc:1403] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.007s
I20260709 13:09:17.675827  9141 raft_consensus.cc:2468] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 0.
I20260709 13:09:17.677852  9031 leader_election.cc:304] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:17.679131  9271 raft_consensus.cc:2804] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:17.674024  9258 ts_tablet_manager.cc:1434] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:09:17.680073  9271 raft_consensus.cc:493] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.680830  9271 raft_consensus.cc:3060] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.681532  9258 tablet_bootstrap.cc:492] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:17.682180  9256 raft_consensus.cc:359] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.682938  9256 raft_consensus.cc:385] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.683398  9256 raft_consensus.cc:740] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.673244  9215 raft_consensus.cc:2468] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 0.
I20260709 13:09:17.684260  9256 consensus_queue.cc:260] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.674377  9260 leader_election.cc:290] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:17.691325  9256 ts_tablet_manager.cc:1434] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:09:17.691609  9258 tablet_bootstrap.cc:654] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.692077  9262 raft_consensus.cc:493] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.692544  9256 tablet_bootstrap.cc:492] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:17.692629  9262 raft_consensus.cc:515] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.695389  9271 raft_consensus.cc:515] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } }
I20260709 13:09:17.695971  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:17.698035  9291 raft_consensus.cc:493] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.698381  9255 tablet_bootstrap.cc:492] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:17.700263  9066 raft_consensus.cc:2468] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:17.700462  9255 ts_tablet_manager.cc:1403] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.017s
I20260709 13:09:17.701303  9256 tablet_bootstrap.cc:654] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.701155  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e01e0329e284d0b9f423618165717b5" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:17.701808  9215 raft_consensus.cc:3060] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.701661  9178 leader_election.cc:304] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:17.704535  9282 raft_consensus.cc:493] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.704576  9291 raft_consensus.cc:515] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.705757  9282 raft_consensus.cc:515] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.707283  9291 leader_election.cc:290] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:17.711992  9215 raft_consensus.cc:2468] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 1.
I20260709 13:09:17.712769  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:17.713438  9291 raft_consensus.cc:2804] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:17.713936  9291 raft_consensus.cc:493] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.714299  9291 raft_consensus.cc:3060] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.715241  9065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eb3a7f451bb463d87fb2f927470dd7f" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:17.715433  9032 leader_election.cc:304] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:17.716622  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eb3a7f451bb463d87fb2f927470dd7f" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:17.713804  9066 raft_consensus.cc:2468] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:17.717700  9263 raft_consensus.cc:493] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.718220  9104 leader_election.cc:304] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:17.725798  9271 leader_election.cc:290] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Requested vote from peers dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:17.718195  9263 raft_consensus.cc:515] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.730580  9263 leader_election.cc:290] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.731375  9263 raft_consensus.cc:2804] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:09:17.732118  9180 leader_election.cc:343] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.741947  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:17.716595  9260 raft_consensus.cc:2804] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:17.743703  9260 raft_consensus.cc:697] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Becoming Leader. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:17.744828  9260 consensus_queue.cc:237] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } }
I20260709 13:09:17.749822  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac243bc7f6234c60951698df8322d1c0" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:17.719309  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:17.750356  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e01e0329e284d0b9f423618165717b5" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:17.751076  9140 raft_consensus.cc:3060] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:17.752002  9107 leader_election.cc:343] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:17.754835  9178 leader_election.cc:343] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:17.756808  9241 tablet.cc:2406] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:09:17.760316  9140 raft_consensus.cc:2468] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 1.
I20260709 13:09:17.761590  9291 raft_consensus.cc:515] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.763329  9214 raft_consensus.cc:2393] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75f16c06c14045a5b104c416d3fd6242 in current term 1: Already voted for candidate dad482c12e774a2a8df70dc54dde1822 in this term.
I20260709 13:09:17.719295  9282 leader_election.cc:290] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.727648  9262 leader_election.cc:290] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:17.731838  9263 raft_consensus.cc:493] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.703804  9255 raft_consensus.cc:359] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.768465  9255 raft_consensus.cc:385] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.768811  9255 raft_consensus.cc:740] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.769757  9255 consensus_queue.cc:260] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.755703  9178 leader_election.cc:304] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822; no voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:17.773530  9262 raft_consensus.cc:2749] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:17.772087  9255 ts_tablet_manager.cc:1434] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.003s
I20260709 13:09:17.768340  9263 raft_consensus.cc:3060] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:17.775904  9104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:37425, user_credentials={real_user=slave}} blocked reactor thread for 57782.4us
I20260709 13:09:17.776930  9255 tablet_bootstrap.cc:492] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:17.778442  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac243bc7f6234c60951698df8322d1c0" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:17.778435  9256 tablet_bootstrap.cc:492] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:17.779321  9256 ts_tablet_manager.cc:1403] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.087s	user 0.023s	sys 0.032s
W20260709 13:09:17.780467  9104 leader_election.cc:343] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.781526  9104 leader_election.cc:304] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242; no voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:17.781888  9291 leader_election.cc:290] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:17.784026  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:17.784652  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:17.785310  9066 raft_consensus.cc:3060] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.785231  9256 raft_consensus.cc:359] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.786145  9256 raft_consensus.cc:385] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.787384  9256 raft_consensus.cc:740] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.788475  9256 consensus_queue.cc:260] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.791947  9255 tablet_bootstrap.cc:654] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.794232  9256 ts_tablet_manager.cc:1434] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 13:09:17.796916  9256 tablet_bootstrap.cc:492] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:17.801055  9066 raft_consensus.cc:2468] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:17.802372  9263 raft_consensus.cc:515] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.802697  9178 leader_election.cc:304] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 75f16c06c14045a5b104c416d3fd6242
I20260709 13:09:17.803769  9291 raft_consensus.cc:2804] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:17.804425  9291 raft_consensus.cc:697] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Leader. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:17.804857  9256 tablet_bootstrap.cc:654] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.805476  9291 consensus_queue.cc:237] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.808260  9293 raft_consensus.cc:2749] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:17.803802  8962 catalog_manager.cc:5697] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "fcb5f9589eb7482cab677f08b73602a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:17.811655  9258 tablet_bootstrap.cc:492] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:17.812273  9258 ts_tablet_manager.cc:1403] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.131s	user 0.034s	sys 0.039s
I20260709 13:09:17.823029  9141 raft_consensus.cc:2393] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dad482c12e774a2a8df70dc54dde1822 in current term 1: Already voted for candidate 75f16c06c14045a5b104c416d3fd6242 in this term.
I20260709 13:09:17.816236  9258 raft_consensus.cc:359] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.825049  9258 raft_consensus.cc:385] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.825450  9258 raft_consensus.cc:740] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.826536  9258 consensus_queue.cc:260] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.824687  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:17.829493  9291 raft_consensus.cc:493] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.829689  9066 raft_consensus.cc:2393] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 75f16c06c14045a5b104c416d3fd6242 in current term 1: Already voted for candidate dad482c12e774a2a8df70dc54dde1822 in this term.
I20260709 13:09:17.828575  9263 leader_election.cc:290] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.830181  9291 raft_consensus.cc:515] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.837452  8964 catalog_manager.cc:5697] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 reported cstate change: term changed from 0 to 1, leader changed from <none> to dad482c12e774a2a8df70dc54dde1822 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "dad482c12e774a2a8df70dc54dde1822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:17.840060  9291 leader_election.cc:290] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:17.842085  9258 ts_tablet_manager.cc:1434] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.003s
I20260709 13:09:17.842952  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78d91e5ea574eeb99069dda72c76bac" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:17.843555  9258 tablet_bootstrap.cc:492] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:17.845059  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:17.846673  9107 leader_election.cc:304] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242; no voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:17.847976  9263 raft_consensus.cc:2749] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:17.848075  9255 tablet_bootstrap.cc:492] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:17.849051  9255 ts_tablet_manager.cc:1403] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.072s	user 0.022s	sys 0.008s
I20260709 13:09:17.850782  9258 tablet_bootstrap.cc:654] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:17.852577  9180 leader_election.cc:343] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.854861  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78d91e5ea574eeb99069dda72c76bac" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:17.857046  9256 tablet_bootstrap.cc:492] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:17.857807  9256 ts_tablet_manager.cc:1403] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.061s	user 0.042s	sys 0.016s
I20260709 13:09:17.858186  9258 tablet_bootstrap.cc:492] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:17.858700  9258 ts_tablet_manager.cc:1403] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.015s	user 0.000s	sys 0.010s
I20260709 13:09:17.853621  9255 raft_consensus.cc:359] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.859071  9255 raft_consensus.cc:385] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.859370  9255 raft_consensus.cc:740] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.860255  9255 consensus_queue.cc:260] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.861325  9258 raft_consensus.cc:359] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.862035  9258 raft_consensus.cc:385] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.862427  9258 raft_consensus.cc:740] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.862604  9255 ts_tablet_manager.cc:1434] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:09:17.863276  9258 consensus_queue.cc:260] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.863752  9255 tablet_bootstrap.cc:492] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
W20260709 13:09:17.864338  9178 leader_election.cc:343] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:17.864898  9178 leader_election.cc:304] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822; no voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:17.865774  9258 ts_tablet_manager.cc:1434] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:17.865696  9291 raft_consensus.cc:2749] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:17.866796  9258 tablet_bootstrap.cc:492] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:17.872473  9255 tablet_bootstrap.cc:654] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.873193  9258 tablet_bootstrap.cc:654] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.860720  9256 raft_consensus.cc:359] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.876585  9256 raft_consensus.cc:385] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.876914  9256 raft_consensus.cc:740] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.877774  9256 consensus_queue.cc:260] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.878351  9255 tablet_bootstrap.cc:492] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:17.879055  9255 ts_tablet_manager.cc:1403] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.001s
I20260709 13:09:17.880014  9263 raft_consensus.cc:493] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.880716  9263 raft_consensus.cc:515] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.882697  9255 raft_consensus.cc:359] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.883551  9255 raft_consensus.cc:385] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.883882  9255 raft_consensus.cc:740] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.884370  9256 ts_tablet_manager.cc:1434] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:09:17.884996  9255 consensus_queue.cc:260] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.885596  9256 tablet_bootstrap.cc:492] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:17.885316  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1f95384455460a840a18f5746aaae8" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:17.886264  9214 raft_consensus.cc:2468] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:17.888115  9107 leader_election.cc:304] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:17.889324  9293 raft_consensus.cc:2804] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:17.889704  9293 raft_consensus.cc:493] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.890038  9293 raft_consensus.cc:3060] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.891450  9255 ts_tablet_manager.cc:1434] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:09:17.892354  9256 tablet_bootstrap.cc:654] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.893445  9263 leader_election.cc:290] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.894091  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1f95384455460a840a18f5746aaae8" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:17.894809  9066 raft_consensus.cc:2468] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:17.898994  9256 tablet_bootstrap.cc:492] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:17.899531  9256 ts_tablet_manager.cc:1403] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 13:09:17.901628  9255 tablet_bootstrap.cc:492] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:17.901835  9256 raft_consensus.cc:359] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.902454  9256 raft_consensus.cc:385] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.902689  9256 raft_consensus.cc:740] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.903335  9256 consensus_queue.cc:260] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.898732  9293 raft_consensus.cc:515] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.905433  9256 ts_tablet_manager.cc:1434] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:09:17.906456  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1f95384455460a840a18f5746aaae8" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:17.907186  9066 raft_consensus.cc:3060] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.908321  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1f95384455460a840a18f5746aaae8" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:17.909502  9214 raft_consensus.cc:3060] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.911307  9258 tablet_bootstrap.cc:492] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:17.911860  9258 ts_tablet_manager.cc:1403] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.003s
I20260709 13:09:17.914824  9258 raft_consensus.cc:359] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.915588  9258 raft_consensus.cc:385] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.915845  9258 raft_consensus.cc:740] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.916693  9258 consensus_queue.cc:260] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.917012  9293 leader_election.cc:290] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.919078  9258 ts_tablet_manager.cc:1434] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:09:17.909526  9255 tablet_bootstrap.cc:654] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.920007  9214 raft_consensus.cc:2468] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:17.921589  9107 leader_election.cc:304] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:17.922433  9263 raft_consensus.cc:2804] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:17.922912  9263 raft_consensus.cc:697] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Becoming Leader. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:17.923790  9263 consensus_queue.cc:237] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.925767  9255 tablet_bootstrap.cc:492] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:17.926391  9255 ts_tablet_manager.cc:1403] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.000s
I20260709 13:09:17.928002  9066 raft_consensus.cc:2468] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:17.929723  9255 raft_consensus.cc:359] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.930501  9255 raft_consensus.cc:385] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.930806  9255 raft_consensus.cc:740] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.931540  9255 consensus_queue.cc:260] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.933974  9255 ts_tablet_manager.cc:1434] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:17.935038  9255 tablet_bootstrap.cc:492] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:17.940006  8963 catalog_manager.cc:5697] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "75f16c06c14045a5b104c416d3fd6242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:17.941210  9255 tablet_bootstrap.cc:654] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.945760  9291 raft_consensus.cc:493] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.946352  9291 raft_consensus.cc:515] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.947781  9255 tablet_bootstrap.cc:492] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:17.948348  9255 ts_tablet_manager.cc:1403] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:09:17.948691  9291 leader_election.cc:290] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:17.950749  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916527a7aa6340e092512d7175e79017" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:17.951206  9255 raft_consensus.cc:359] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.951774  9141 raft_consensus.cc:2393] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dad482c12e774a2a8df70dc54dde1822 in current term 1: Already voted for candidate fcb5f9589eb7482cab677f08b73602a1 in this term.
I20260709 13:09:17.951997  9255 raft_consensus.cc:385] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.951866  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "916527a7aa6340e092512d7175e79017" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:17.952466  9255 raft_consensus.cc:740] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.953176  9255 consensus_queue.cc:260] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.953455  9178 leader_election.cc:304] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822; no voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:17.954298  9291 raft_consensus.cc:3060] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.955099  9255 ts_tablet_manager.cc:1434] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:09:17.956301  9255 tablet_bootstrap.cc:492] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:17.960592  9255 tablet_bootstrap.cc:654] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:17.961522  9291 raft_consensus.cc:2749] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:17.965154  9255 tablet_bootstrap.cc:492] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:17.965627  9255 ts_tablet_manager.cc:1403] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:09:17.967691  9255 raft_consensus.cc:359] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.968273  9255 raft_consensus.cc:385] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:17.968526  9293 raft_consensus.cc:493] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.968578  9255 raft_consensus.cc:740] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:17.969202  9293 raft_consensus.cc:515] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.969499  9255 consensus_queue.cc:260] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.970278  9263 raft_consensus.cc:493] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.970826  9263 raft_consensus.cc:515] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.971617  9255 ts_tablet_manager.cc:1434] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:09:17.973029  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eb3a7f451bb463d87fb2f927470dd7f" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:17.973189  9263 leader_election.cc:290] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.973834  9214 raft_consensus.cc:2468] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:17.974660  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8fc36481524d908afb6d9debb715ef" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:17.974620  9065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8fc36481524d908afb6d9debb715ef" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:17.975380  9107 leader_election.cc:304] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:17.975500  9215 raft_consensus.cc:2468] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:17.976275  9293 leader_election.cc:290] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.975558  9065 raft_consensus.cc:2468] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:17.976737  9263 raft_consensus.cc:2804] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:17.977180  9263 raft_consensus.cc:493] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.977157  9107 leader_election.cc:304] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:17.977731  9263 raft_consensus.cc:3060] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.978644  9293 raft_consensus.cc:2804] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:17.978977  9293 raft_consensus.cc:493] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.979266  9293 raft_consensus.cc:3060] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.984352  9263 raft_consensus.cc:515] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.985852  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eb3a7f451bb463d87fb2f927470dd7f" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:17.986598  9066 raft_consensus.cc:2468] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:17.987357  9065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eb3a7f451bb463d87fb2f927470dd7f" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:17.987140  9293 raft_consensus.cc:515] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:17.988005  9065 raft_consensus.cc:3060] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.988709  9263 leader_election.cc:290] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.989446  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7eb3a7f451bb463d87fb2f927470dd7f" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:17.990000  9215 raft_consensus.cc:3060] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.990640  9293 leader_election.cc:290] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:17.991303  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8fc36481524d908afb6d9debb715ef" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:17.991593  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8fc36481524d908afb6d9debb715ef" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:17.992049  9214 raft_consensus.cc:3060] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.992250  9066 raft_consensus.cc:3060] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:17.997360  9065 raft_consensus.cc:2468] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:17.998482  9215 raft_consensus.cc:2468] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:17.998896  9104 leader_election.cc:304] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:18.000002  9293 raft_consensus.cc:2804] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:18.000478  9293 raft_consensus.cc:697] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Becoming Leader. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:18.001065  9214 raft_consensus.cc:2468] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:18.001247  9293 consensus_queue.cc:237] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.002537  9107 leader_election.cc:304] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:18.002751  9066 raft_consensus.cc:2468] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:18.003522  9263 raft_consensus.cc:2804] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:18.004105  9263 raft_consensus.cc:697] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Becoming Leader. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:18.005262  9263 consensus_queue.cc:237] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.006176  9291 raft_consensus.cc:493] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.006762  9291 raft_consensus.cc:515] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.009922  9291 leader_election.cc:290] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:18.010092  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac243bc7f6234c60951698df8322d1c0" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:18.010522  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac243bc7f6234c60951698df8322d1c0" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:18.010874  9066 raft_consensus.cc:2468] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:18.011180  9141 raft_consensus.cc:2468] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:18.012336  9178 leader_election.cc:304] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:18.013674  9291 raft_consensus.cc:2804] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:18.014333  9291 raft_consensus.cc:493] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.014725  9291 raft_consensus.cc:3060] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:18.013706  8963 catalog_manager.cc:5697] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "75f16c06c14045a5b104c416d3fd6242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:18.022267  9291 raft_consensus.cc:515] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.024669  9291 leader_election.cc:290] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:18.025597  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac243bc7f6234c60951698df8322d1c0" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:18.026206  9066 raft_consensus.cc:3060] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:18.025976  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac243bc7f6234c60951698df8322d1c0" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:18.026742  9141 raft_consensus.cc:3060] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:18.030133  8963 catalog_manager.cc:5697] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "75f16c06c14045a5b104c416d3fd6242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:18.034195  9066 raft_consensus.cc:2468] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:18.034193  9141 raft_consensus.cc:2468] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:18.035938  9178 leader_election.cc:304] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:18.037776  9291 raft_consensus.cc:2804] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:18.038465  9291 raft_consensus.cc:697] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Leader. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:18.039556  9291 consensus_queue.cc:237] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.050709  8963 catalog_manager.cc:5697] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 reported cstate change: term changed from 0 to 1, leader changed from <none> to dad482c12e774a2a8df70dc54dde1822 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "dad482c12e774a2a8df70dc54dde1822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:18.087905  9260 consensus_queue.cc:1048] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.101186  9263 raft_consensus.cc:493] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.101794  9263 raft_consensus.cc:515] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.104013  9263 leader_election.cc:290] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:18.104671  9311 consensus_queue.cc:1048] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:18.104868  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78d91e5ea574eeb99069dda72c76bac" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:18.105764  9066 raft_consensus.cc:2468] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:18.106084  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78d91e5ea574eeb99069dda72c76bac" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:18.106783  9215 raft_consensus.cc:2468] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:18.107291  9104 leader_election.cc:304] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:18.108223  9263 raft_consensus.cc:2804] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:18.108654  9263 raft_consensus.cc:493] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.109074  9263 raft_consensus.cc:3060] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:18.117802  9263 raft_consensus.cc:515] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.120680  9263 leader_election.cc:290] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:18.120911  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78d91e5ea574eeb99069dda72c76bac" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:18.121340  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78d91e5ea574eeb99069dda72c76bac" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:18.121634  9066 raft_consensus.cc:3060] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:18.121950  9214 raft_consensus.cc:3060] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:18.129832  9066 raft_consensus.cc:2468] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:18.131038  9104 leader_election.cc:304] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:18.131057  9214 raft_consensus.cc:2468] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:18.131846  9263 raft_consensus.cc:2804] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:18.132395  9263 raft_consensus.cc:697] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Becoming Leader. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:18.133144  9263 consensus_queue.cc:237] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.141580  8963 catalog_manager.cc:5697] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "75f16c06c14045a5b104c416d3fd6242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:18.199452  9311 raft_consensus.cc:493] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.200145  9311 raft_consensus.cc:515] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.201828  9311 leader_election.cc:290] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:18.203011  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:18.202890  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:18.203966  9141 raft_consensus.cc:2468] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 0.
I20260709 13:09:18.203970  9214 raft_consensus.cc:2468] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 0.
I20260709 13:09:18.205493  9031 leader_election.cc:304] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:18.206221  9311 raft_consensus.cc:2804] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:18.206575  9311 raft_consensus.cc:493] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.206842  9311 raft_consensus.cc:3060] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:18.213181  9311 raft_consensus.cc:515] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.214888  9311 leader_election.cc:290] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Requested vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:18.215880  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:18.216015  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:18.216619  9141 raft_consensus.cc:3060] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:18.216804  9214 raft_consensus.cc:3060] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:18.223307  9141 raft_consensus.cc:2468] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 1.
I20260709 13:09:18.223975  9214 raft_consensus.cc:2468] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 1.
I20260709 13:09:18.224689  9031 leader_election.cc:304] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:18.225430  9311 raft_consensus.cc:2804] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:18.225934  9311 raft_consensus.cc:697] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Becoming Leader. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:18.226706  9311 consensus_queue.cc:237] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.230604  9311 consensus_queue.cc:1048] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.235185  9291 consensus_queue.cc:1048] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.235693  8963 catalog_manager.cc:5697] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "fcb5f9589eb7482cab677f08b73602a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:18.246502  9311 consensus_queue.cc:1048] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.254846  9319 consensus_queue.cc:1048] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:18.324628  9214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62"
dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
mode: GRACEFUL
new_leader_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:42752
I20260709 13:09:18.325205  9214 raft_consensus.cc:606] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Received request to transfer leadership to TS fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:18.330614  9214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac243bc7f6234c60951698df8322d1c0"
dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
mode: GRACEFUL
new_leader_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:42752
I20260709 13:09:18.331184  9214 raft_consensus.cc:606] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Received request to transfer leadership to TS fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:18.335776  9263 consensus_queue.cc:1048] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.348768  9282 consensus_queue.cc:1048] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.368837  9066 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b"
dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
mode: GRACEFUL
new_leader_uuid: "75f16c06c14045a5b104c416d3fd6242"
 from {username='slave'} at 127.0.0.1:37766
I20260709 13:09:18.369442  9066 raft_consensus.cc:606] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Received request to transfer leadership to TS 75f16c06c14045a5b104c416d3fd6242
I20260709 13:09:18.387398  9141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d78d91e5ea574eeb99069dda72c76bac"
dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
mode: GRACEFUL
new_leader_uuid: "dad482c12e774a2a8df70dc54dde1822"
 from {username='slave'} at 127.0.0.1:40254
I20260709 13:09:18.388087  9141 raft_consensus.cc:606] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Received request to transfer leadership to TS dad482c12e774a2a8df70dc54dde1822
I20260709 13:09:18.438037  9326 consensus_queue.cc:1048] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.439986  9326 consensus_queue.cc:1048] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.452374  9293 consensus_queue.cc:1048] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.456118  9321 consensus_queue.cc:1048] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.458839  9263 consensus_queue.cc:1048] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:18.461047  9319 consensus_queue.cc:1048] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.471020  9262 raft_consensus.cc:993] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822: : Instructing follower fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:18.471472  9262 raft_consensus.cc:1081] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Signalling peer fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:18.479048  9065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac243bc7f6234c60951698df8322d1c0"
dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:37752
I20260709 13:09:18.479733  9065 raft_consensus.cc:493] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:18.480216  9065 raft_consensus.cc:3060] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.487411  9065 raft_consensus.cc:515] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.490206  9065 leader_election.cc:290] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Requested vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:18.491433  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac243bc7f6234c60951698df8322d1c0" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:18.491552  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac243bc7f6234c60951698df8322d1c0" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:18.492304  9141 raft_consensus.cc:3060] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.492399  9215 raft_consensus.cc:3055] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:18.492904  9215 raft_consensus.cc:740] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:18.493796  9215 consensus_queue.cc:260] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.495314  9215 raft_consensus.cc:3060] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.500486  9141 raft_consensus.cc:2468] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:18.501787  9031 leader_election.cc:304] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:18.502580  9215 raft_consensus.cc:2468] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:18.502699  9311 raft_consensus.cc:2804] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.503330  9311 raft_consensus.cc:697] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Becoming Leader. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:18.504240  9311 consensus_queue.cc:237] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.510591  8963 catalog_manager.cc:5697] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 reported cstate change: term changed from 1 to 2, leader changed from dad482c12e774a2a8df70dc54dde1822 (127.27.71.131) to fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "fcb5f9589eb7482cab677f08b73602a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:18.747188  9326 consensus_queue.cc:1048] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.750698  9311 consensus_queue.cc:1048] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.753031  9319 raft_consensus.cc:993] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822: : Instructing follower fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:18.753569  9319 raft_consensus.cc:1081] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Signalling peer fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:18.755893  9065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62"
dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:37752
I20260709 13:09:18.756583  9065 raft_consensus.cc:493] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:18.756656  9293 raft_consensus.cc:993] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242: : Instructing follower dad482c12e774a2a8df70dc54dde1822 to start an election
I20260709 13:09:18.757022  9065 raft_consensus.cc:3060] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.757398  9293 raft_consensus.cc:1081] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Signalling peer dad482c12e774a2a8df70dc54dde1822 to start an election
I20260709 13:09:18.761659  9293 consensus_queue.cc:1048] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:18.762778  9215 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d78d91e5ea574eeb99069dda72c76bac"
dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
 from {username='slave'} at 127.0.0.1:42744
I20260709 13:09:18.763499  9215 raft_consensus.cc:493] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:18.763909  9215 raft_consensus.cc:3060] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.773020  9215 raft_consensus.cc:515] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.773406  9271 consensus_queue.cc:1048] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:18.777064  9064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78d91e5ea574eeb99069dda72c76bac" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:18.780068  9215 leader_election.cc:290] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 2 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:18.780697  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78d91e5ea574eeb99069dda72c76bac" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:18.781337  9140 raft_consensus.cc:3055] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:18.781653  9140 raft_consensus.cc:740] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:18.782419  9140 consensus_queue.cc:260] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.783635  9140 raft_consensus.cc:3060] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.793210  9271 raft_consensus.cc:993] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1: : Instructing follower 75f16c06c14045a5b104c416d3fd6242 to start an election
I20260709 13:09:18.793829  9271 raft_consensus.cc:1081] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Signalling peer 75f16c06c14045a5b104c416d3fd6242 to start an election
I20260709 13:09:18.798399  9141 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b"
dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
 from {username='slave'} at 127.0.0.1:40224
I20260709 13:09:18.799162  9141 raft_consensus.cc:493] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:18.800467  9140 raft_consensus.cc:2468] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 2.
I20260709 13:09:18.780211  9065 raft_consensus.cc:515] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.803162  9065 leader_election.cc:290] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Requested vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:18.805058  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:18.805761  9215 raft_consensus.cc:3055] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:18.806110  9215 raft_consensus.cc:740] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:18.807014  9215 consensus_queue.cc:260] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.808173  9141 raft_consensus.cc:3060] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.808696  9215 raft_consensus.cc:3060] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.808566  9180 leader_election.cc:304] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:18.810118  9291 raft_consensus.cc:2804] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.810153  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:18.810855  9140 raft_consensus.cc:3060] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.810757  9291 raft_consensus.cc:697] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 2 LEADER]: Becoming Leader. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:18.812031  9291 consensus_queue.cc:237] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.817787  9141 raft_consensus.cc:515] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.820829  9141 leader_election.cc:290] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 2 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:18.821000  9065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:18.821700  9065 raft_consensus.cc:3055] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:18.822041  9065 raft_consensus.cc:740] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:18.822642  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:18.822763  9140 raft_consensus.cc:2468] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:18.823386  9214 raft_consensus.cc:3060] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.822940  9065 consensus_queue.cc:260] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.825259  9031 leader_election.cc:304] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:18.825104  8963 catalog_manager.cc:5697] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 reported cstate change: term changed from 1 to 2, leader changed from 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130) to dad482c12e774a2a8df70dc54dde1822 (127.27.71.131). New cstate: current_term: 2 leader_uuid: "dad482c12e774a2a8df70dc54dde1822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:18.826131  9065 raft_consensus.cc:3060] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:18.829411  9311 raft_consensus.cc:2804] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.833139  9214 raft_consensus.cc:2468] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 2.
I20260709 13:09:18.833608  9311 raft_consensus.cc:697] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Becoming Leader. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:18.834856  9107 leader_election.cc:304] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:18.834973  9215 raft_consensus.cc:2468] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:18.835803  9293 raft_consensus.cc:2804] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:18.836414  9293 raft_consensus.cc:697] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Becoming Leader. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:18.837349  9293 consensus_queue.cc:237] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.834690  9311 consensus_queue.cc:237] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:18.844002  9065 raft_consensus.cc:2468] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 2.
I20260709 13:09:18.846547  8963 catalog_manager.cc:5697] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 reported cstate change: term changed from 1 to 2, leader changed from fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129) to 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "75f16c06c14045a5b104c416d3fd6242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:18.846858  8964 catalog_manager.cc:5697] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 reported cstate change: term changed from 1 to 2, leader changed from dad482c12e774a2a8df70dc54dde1822 (127.27.71.131) to fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "fcb5f9589eb7482cab677f08b73602a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:18.916460  9140 raft_consensus.cc:1275] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.918057  9311 consensus_queue.cc:1048] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:18.926931  9214 raft_consensus.cc:1275] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:18.928838  9271 consensus_queue.cc:1048] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:19.233865  9215 raft_consensus.cc:1275] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:19.235419  9311 consensus_queue.cc:1048] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:19.244153  9140 raft_consensus.cc:1275] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:19.246729  9260 consensus_queue.cc:1048] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:19.264801  9140 raft_consensus.cc:1275] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Refusing update from remote peer dad482c12e774a2a8df70dc54dde1822: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:19.266532  9291 consensus_queue.cc:1048] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:19.275671  9065 raft_consensus.cc:3060] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:19.284865  9065 raft_consensus.cc:1275] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Refusing update from remote peer dad482c12e774a2a8df70dc54dde1822: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:19.286237  9321 consensus_queue.cc:1048] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:19.402833  9215 raft_consensus.cc:1275] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Refusing update from remote peer 75f16c06c14045a5b104c416d3fd6242: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:19.404222  9293 consensus_queue.cc:1048] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:19.408813  9065 raft_consensus.cc:1275] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Refusing update from remote peer 75f16c06c14045a5b104c416d3fd6242: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:19.410787  9263 consensus_queue.cc:1048] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:21.418833  8964 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45784:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:09:21.421695  8964 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:09:21.479784  9055 tablet_service.cc:1511] Processing CreateTablet for tablet 7389efc195ac42f6a444c254e095eab8 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:21.481383  9055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7389efc195ac42f6a444c254e095eab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.481232  9126 tablet_service.cc:1511] Processing CreateTablet for tablet 7389efc195ac42f6a444c254e095eab8 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:21.482290  9199 tablet_service.cc:1511] Processing CreateTablet for tablet 7389efc195ac42f6a444c254e095eab8 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:09:21.483034  9126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7389efc195ac42f6a444c254e095eab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.483664  9199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7389efc195ac42f6a444c254e095eab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.495050  9050 tablet_service.cc:1511] Processing CreateTablet for tablet bcefbbc6c9ca4d0dbe09c57dec1b0ed2 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:21.496718  9050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcefbbc6c9ca4d0dbe09c57dec1b0ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.500175  9127 tablet_service.cc:1511] Processing CreateTablet for tablet bcefbbc6c9ca4d0dbe09c57dec1b0ed2 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:21.501792  9127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcefbbc6c9ca4d0dbe09c57dec1b0ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.502503  9203 tablet_service.cc:1511] Processing CreateTablet for tablet bcefbbc6c9ca4d0dbe09c57dec1b0ed2 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:09:21.505035  9203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcefbbc6c9ca4d0dbe09c57dec1b0ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.506346  9052 tablet_service.cc:1511] Processing CreateTablet for tablet f5a90f2828074148a8dd9ab98ade3083 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:21.508325  9052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a90f2828074148a8dd9ab98ade3083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.512754  9123 tablet_service.cc:1511] Processing CreateTablet for tablet f5a90f2828074148a8dd9ab98ade3083 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:21.515014  9123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a90f2828074148a8dd9ab98ade3083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.539050  9200 tablet_service.cc:1511] Processing CreateTablet for tablet f5a90f2828074148a8dd9ab98ade3083 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:09:21.538862  9053 tablet_service.cc:1511] Processing CreateTablet for tablet d4451c98b52b407cb8ef9af4abcf8426 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:21.541827  9053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4451c98b52b407cb8ef9af4abcf8426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.543021  9124 tablet_service.cc:1511] Processing CreateTablet for tablet d4451c98b52b407cb8ef9af4abcf8426 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:21.544485  9124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4451c98b52b407cb8ef9af4abcf8426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.550280  9200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a90f2828074148a8dd9ab98ade3083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.555498  9054 tablet_service.cc:1511] Processing CreateTablet for tablet 4fe5b7bf832245ecb1471c18d72caa7a (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:21.557546  9353 tablet_bootstrap.cc:492] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:21.557614  9054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe5b7bf832245ecb1471c18d72caa7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.567567  9128 tablet_service.cc:1511] Processing CreateTablet for tablet 4fe5b7bf832245ecb1471c18d72caa7a (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:21.573604  9128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe5b7bf832245ecb1471c18d72caa7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.568150  9356 tablet_bootstrap.cc:492] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:21.568908  9203 tablet_service.cc:1511] Processing CreateTablet for tablet d4451c98b52b407cb8ef9af4abcf8426 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:09:21.586618  9203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4451c98b52b407cb8ef9af4abcf8426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.589962  9353 tablet_bootstrap.cc:654] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:21.593674  9199 tablet_service.cc:1511] Processing CreateTablet for tablet 4fe5b7bf832245ecb1471c18d72caa7a (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:09:21.595312  9199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe5b7bf832245ecb1471c18d72caa7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.595625  9055 tablet_service.cc:1511] Processing CreateTablet for tablet 8ec08d78bd0d4e0ca1c0f69ebc42f4cd (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:21.597230  9055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ec08d78bd0d4e0ca1c0f69ebc42f4cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.597384  9129 tablet_service.cc:1511] Processing CreateTablet for tablet 8ec08d78bd0d4e0ca1c0f69ebc42f4cd (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:21.602759  9204 tablet_service.cc:1511] Processing CreateTablet for tablet 8ec08d78bd0d4e0ca1c0f69ebc42f4cd (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:09:21.604305  9204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ec08d78bd0d4e0ca1c0f69ebc42f4cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.611781  9051 tablet_service.cc:1511] Processing CreateTablet for tablet 7a8e9f167f21457eb5bd7ed9151445a1 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:21.621191  9129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ec08d78bd0d4e0ca1c0f69ebc42f4cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.622763  9356 tablet_bootstrap.cc:654] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:21.634995  9012 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:21.636780  9051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a8e9f167f21457eb5bd7ed9151445a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.655175  9125 tablet_service.cc:1511] Processing CreateTablet for tablet 7a8e9f167f21457eb5bd7ed9151445a1 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
W20260709 13:09:21.656105  9104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:37425, user_credentials={real_user=slave}} blocked reactor thread for 45676.5us
I20260709 13:09:21.656426  9199 tablet_service.cc:1511] Processing CreateTablet for tablet 7a8e9f167f21457eb5bd7ed9151445a1 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:09:21.656945  9125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a8e9f167f21457eb5bd7ed9151445a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.657913  9199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a8e9f167f21457eb5bd7ed9151445a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.665131  9050 tablet_service.cc:1511] Processing CreateTablet for tablet c6200e2af384461eb93770f68b27cfa6 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:21.669332  9126 tablet_service.cc:1511] Processing CreateTablet for tablet c6200e2af384461eb93770f68b27cfa6 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:21.682319  9126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6200e2af384461eb93770f68b27cfa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.670673  9202 tablet_service.cc:1511] Processing CreateTablet for tablet c6200e2af384461eb93770f68b27cfa6 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:09:21.684819  9202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6200e2af384461eb93770f68b27cfa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.676685  9359 tablet_bootstrap.cc:492] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:21.687117  9353 tablet_bootstrap.cc:492] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:21.687777  9353 ts_tablet_manager.cc:1403] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.131s	user 0.030s	sys 0.023s
I20260709 13:09:21.693328  9353 raft_consensus.cc:359] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.694160  9353 raft_consensus.cc:385] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:21.694492  9353 raft_consensus.cc:740] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:21.695336  9353 consensus_queue.cc:260] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.697906  9353 ts_tablet_manager.cc:1434] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:09:21.702334  9359 tablet_bootstrap.cc:654] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:21.708715  9050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6200e2af384461eb93770f68b27cfa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.712442  9353 tablet_bootstrap.cc:492] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:21.722343  9053 tablet_service.cc:1511] Processing CreateTablet for tablet 0f44bd882825450c93046d494e2f9264 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:21.729817  9053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f44bd882825450c93046d494e2f9264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.737830  9203 tablet_service.cc:1511] Processing CreateTablet for tablet 0f44bd882825450c93046d494e2f9264 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:21.739456  9203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f44bd882825450c93046d494e2f9264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.728673  9129 tablet_service.cc:1511] Processing CreateTablet for tablet 0f44bd882825450c93046d494e2f9264 (DEFAULT_TABLE table=soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:09:21.745807  9129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f44bd882825450c93046d494e2f9264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.762706  9353 tablet_bootstrap.cc:654] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:21.762399  9356 tablet_bootstrap.cc:492] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:21.764075  9359 tablet_bootstrap.cc:492] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:21.764492  9356 ts_tablet_manager.cc:1403] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.197s	user 0.013s	sys 0.005s
I20260709 13:09:21.765342  9359 ts_tablet_manager.cc:1403] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.089s	user 0.017s	sys 0.026s
I20260709 13:09:21.769459  9356 raft_consensus.cc:359] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.770417  9359 raft_consensus.cc:359] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.771525  9359 raft_consensus.cc:385] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:21.771970  9359 raft_consensus.cc:740] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:21.770583  9356 raft_consensus.cc:385] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:21.772722  9356 raft_consensus.cc:740] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:21.773711  9359 consensus_queue.cc:260] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.773726  9356 consensus_queue.cc:260] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.778582  9359 ts_tablet_manager.cc:1434] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 13:09:21.780395  9359 tablet_bootstrap.cc:492] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:21.787617  9326 raft_consensus.cc:493] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.788470  9326 raft_consensus.cc:515] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.794710  9359 tablet_bootstrap.cc:654] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:21.796856  9065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7389efc195ac42f6a444c254e095eab8" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:21.797794  9065 raft_consensus.cc:2468] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:21.793761  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7389efc195ac42f6a444c254e095eab8" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:21.799557  9215 raft_consensus.cc:2468] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:21.803169  9353 tablet_bootstrap.cc:492] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:21.804776  9353 ts_tablet_manager.cc:1403] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.093s	user 0.017s	sys 0.000s
I20260709 13:09:21.803547  9104 leader_election.cc:304] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:21.813597  9353 raft_consensus.cc:359] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.822161  9326 leader_election.cc:290] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:21.820380  9358 raft_consensus.cc:2804] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:21.825584  9358 raft_consensus.cc:493] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:21.814445  9353 raft_consensus.cc:385] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:21.827873  9353 raft_consensus.cc:740] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:21.827855  9358 raft_consensus.cc:3060] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:21.840420  9353 consensus_queue.cc:260] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.844659  9353 ts_tablet_manager.cc:1434] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.005s
I20260709 13:09:21.845953  9353 tablet_bootstrap.cc:492] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:21.848636  9358 raft_consensus.cc:515] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.851038  9356 ts_tablet_manager.cc:1434] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.086s	user 0.006s	sys 0.004s
I20260709 13:09:21.857455  9356 tablet_bootstrap.cc:492] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:21.863822  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7389efc195ac42f6a444c254e095eab8" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:21.864675  9215 raft_consensus.cc:3060] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:21.865125  9353 tablet_bootstrap.cc:654] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:21.866291  9356 tablet_bootstrap.cc:654] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:21.869365  9359 tablet_bootstrap.cc:492] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:21.870498  9359 ts_tablet_manager.cc:1403] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.091s	user 0.011s	sys 0.006s
W20260709 13:09:21.877007  9104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.71.129:37425, user_credentials={real_user=slave}} blocked reactor thread for 60193.2us
I20260709 13:09:21.876053  9358 leader_election.cc:290] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:21.886536  9065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7389efc195ac42f6a444c254e095eab8" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:21.887349  9065 raft_consensus.cc:3060] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:21.887936  9215 raft_consensus.cc:2468] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:21.888541  9353 tablet_bootstrap.cc:492] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:21.889207  9353 ts_tablet_manager.cc:1403] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.004s
I20260709 13:09:21.889575  9107 leader_election.cc:304] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:21.890918  9358 raft_consensus.cc:2804] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:21.891783  9358 raft_consensus.cc:697] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Becoming Leader. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:21.892973  9353 raft_consensus.cc:359] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.893076  9358 consensus_queue.cc:237] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.894016  9353 raft_consensus.cc:385] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:21.898084  9353 raft_consensus.cc:740] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:21.899149  9353 consensus_queue.cc:260] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.901819  9353 ts_tablet_manager.cc:1434] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:09:21.903415  9353 tablet_bootstrap.cc:492] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:21.913007  9353 tablet_bootstrap.cc:654] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:21.914827  9356 tablet_bootstrap.cc:492] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:21.915429  9356 ts_tablet_manager.cc:1403] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.015s
I20260709 13:09:21.918123  9065 raft_consensus.cc:2468] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:21.875542  9359 raft_consensus.cc:359] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.919116  9356 raft_consensus.cc:359] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.920876  9359 raft_consensus.cc:385] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:21.920905  9356 raft_consensus.cc:385] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:21.922551  9359 raft_consensus.cc:740] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:21.922571  9356 raft_consensus.cc:740] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:21.923825  9356 consensus_queue.cc:260] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.927150  9356 ts_tablet_manager.cc:1434] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.007s
I20260709 13:09:21.923821  9359 consensus_queue.cc:260] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.930410  9359 ts_tablet_manager.cc:1434] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.059s	user 0.011s	sys 0.000s
I20260709 13:09:21.931980  9358 raft_consensus.cc:493] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.930927  8961 catalog_manager.cc:5697] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "75f16c06c14045a5b104c416d3fd6242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:21.932554  9358 raft_consensus.cc:515] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.936444  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefbbc6c9ca4d0dbe09c57dec1b0ed2" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:21.937214  9215 raft_consensus.cc:2468] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:21.938714  9107 leader_election.cc:304] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:21.939581  9358 leader_election.cc:290] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:21.940146  9358 raft_consensus.cc:2804] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:21.940495  9358 raft_consensus.cc:493] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:21.941247  9358 raft_consensus.cc:3060] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:21.945701  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefbbc6c9ca4d0dbe09c57dec1b0ed2" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:21.940335  9353 tablet_bootstrap.cc:492] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:21.946787  9066 raft_consensus.cc:2468] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 0.
I20260709 13:09:21.947077  9353 ts_tablet_manager.cc:1403] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.021s
I20260709 13:09:21.949730  9356 tablet_bootstrap.cc:492] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:21.952507  9358 raft_consensus.cc:515] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.955757  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefbbc6c9ca4d0dbe09c57dec1b0ed2" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:21.956463  9066 raft_consensus.cc:3060] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:21.956965  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefbbc6c9ca4d0dbe09c57dec1b0ed2" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:21.957633  9215 raft_consensus.cc:3060] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:21.965618  9356 tablet_bootstrap.cc:654] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:21.965938  9066 raft_consensus.cc:2468] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:21.950037  9359 tablet_bootstrap.cc:492] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:21.969384  9104 leader_election.cc:304] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:21.969848  9215 raft_consensus.cc:2468] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 1.
I20260709 13:09:21.966845  9353 raft_consensus.cc:359] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.970350  9326 raft_consensus.cc:2804] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:21.971282  9353 raft_consensus.cc:385] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:21.971519  9326 raft_consensus.cc:697] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Becoming Leader. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:21.971616  9353 raft_consensus.cc:740] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:21.972693  9326 consensus_queue.cc:237] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.972749  9353 consensus_queue.cc:260] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:21.975749  9353 ts_tablet_manager.cc:1434] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.003s
I20260709 13:09:21.969412  9358 leader_election.cc:290] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:21.978544  9359 tablet_bootstrap.cc:654] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:21.979830  9353 tablet_bootstrap.cc:492] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:21.993774  9356 tablet_bootstrap.cc:492] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:21.994958  9356 ts_tablet_manager.cc:1403] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.004s
I20260709 13:09:22.000448  9356 raft_consensus.cc:359] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.001628  9356 raft_consensus.cc:385] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.001947  9356 raft_consensus.cc:740] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.003079  9356 consensus_queue.cc:260] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.006841  9356 ts_tablet_manager.cc:1434] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:09:22.008527  9356 tablet_bootstrap.cc:492] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:22.009459  9353 tablet_bootstrap.cc:654] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.011235  9359 tablet_bootstrap.cc:492] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:22.011976  9359 ts_tablet_manager.cc:1403] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.080s	user 0.014s	sys 0.023s
I20260709 13:09:22.016073  9359 raft_consensus.cc:359] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.017220  9356 tablet_bootstrap.cc:654] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.017215  9359 raft_consensus.cc:385] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.017767  9359 raft_consensus.cc:740] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.018973  9359 consensus_queue.cc:260] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.024022  9353 tablet_bootstrap.cc:492] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:22.024612  9353 ts_tablet_manager.cc:1403] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.045s	user 0.014s	sys 0.010s
I20260709 13:09:22.022426  9359 ts_tablet_manager.cc:1434] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 13:09:22.027477  9359 tablet_bootstrap.cc:492] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:22.028616  9353 raft_consensus.cc:359] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.029589  9353 raft_consensus.cc:385] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.029877  9353 raft_consensus.cc:740] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.028666  8964 catalog_manager.cc:5697] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130). New cstate: current_term: 1 leader_uuid: "75f16c06c14045a5b104c416d3fd6242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:22.030843  9356 tablet_bootstrap.cc:492] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:22.031416  9356 ts_tablet_manager.cc:1403] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.000s
I20260709 13:09:22.036793  9353 consensus_queue.cc:260] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.038275  9356 raft_consensus.cc:359] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.039202  9356 raft_consensus.cc:385] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.039556  9356 raft_consensus.cc:740] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.040027  9353 ts_tablet_manager.cc:1434] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.015s	user 0.012s	sys 0.003s
I20260709 13:09:22.040764  9356 consensus_queue.cc:260] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.041376  9353 tablet_bootstrap.cc:492] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:22.042891  9359 tablet_bootstrap.cc:654] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.043855  9356 ts_tablet_manager.cc:1434] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.004s
I20260709 13:09:22.045147  9356 tablet_bootstrap.cc:492] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:22.051616  9353 tablet_bootstrap.cc:654] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.052924  9356 tablet_bootstrap.cc:654] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.056156  9359 tablet_bootstrap.cc:492] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:22.057076  9359 ts_tablet_manager.cc:1403] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.011s
I20260709 13:09:22.062296  9359 raft_consensus.cc:359] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.063426  9359 raft_consensus.cc:385] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.063838  9359 raft_consensus.cc:740] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.065315  9359 consensus_queue.cc:260] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.069590  9359 ts_tablet_manager.cc:1434] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.001s
I20260709 13:09:22.071310  9359 tablet_bootstrap.cc:492] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:22.083020  9353 tablet_bootstrap.cc:492] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:22.084092  9353 ts_tablet_manager.cc:1403] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.043s	user 0.026s	sys 0.016s
I20260709 13:09:22.085376  9359 tablet_bootstrap.cc:654] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.087942  9353 raft_consensus.cc:359] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.095039  9353 raft_consensus.cc:385] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.095336  9353 raft_consensus.cc:740] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.096271  9353 consensus_queue.cc:260] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.100503  9353 ts_tablet_manager.cc:1434] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:09:22.102056  9353 tablet_bootstrap.cc:492] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:22.114571  9355 raft_consensus.cc:493] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.115473  9355 raft_consensus.cc:515] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.119884  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a90f2828074148a8dd9ab98ade3083" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:22.120223  9355 leader_election.cc:290] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:22.120666  9066 raft_consensus.cc:2468] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:22.121874  9356 tablet_bootstrap.cc:492] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:22.122192  9359 tablet_bootstrap.cc:492] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:22.122495  9356 ts_tablet_manager.cc:1403] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.078s	user 0.025s	sys 0.029s
I20260709 13:09:22.122692  9359 ts_tablet_manager.cc:1403] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.052s	user 0.019s	sys 0.021s
I20260709 13:09:22.123612  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a90f2828074148a8dd9ab98ade3083" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:22.124347  9140 raft_consensus.cc:2468] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:22.125345  9353 tablet_bootstrap.cc:654] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.126858  9359 raft_consensus.cc:359] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.127177  9178 leader_election.cc:304] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.127672  9359 raft_consensus.cc:385] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.128042  9359 raft_consensus.cc:740] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.128309  9355 raft_consensus.cc:2804] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:22.128844  9355 raft_consensus.cc:493] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.128778  9359 consensus_queue.cc:260] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.129558  9355 raft_consensus.cc:3060] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.131479  9359 ts_tablet_manager.cc:1434] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 13:09:22.132727  9359 tablet_bootstrap.cc:492] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:22.140077  9356 raft_consensus.cc:359] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.140980  9356 raft_consensus.cc:385] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.141345  9356 raft_consensus.cc:740] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.151032  9359 tablet_bootstrap.cc:654] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.153528  9355 raft_consensus.cc:515] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.157722  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a90f2828074148a8dd9ab98ade3083" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:22.158631  9066 raft_consensus.cc:3060] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.162214  9356 consensus_queue.cc:260] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.170550  9356 ts_tablet_manager.cc:1434] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.001s
I20260709 13:09:22.171959  9356 tablet_bootstrap.cc:492] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:22.190902  9356 tablet_bootstrap.cc:654] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.198084  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a90f2828074148a8dd9ab98ade3083" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:22.199173  9141 raft_consensus.cc:3060] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.208065  9355 leader_election.cc:290] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:22.210330  9356 tablet_bootstrap.cc:492] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:22.210945  9356 ts_tablet_manager.cc:1403] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.006s
I20260709 13:09:22.211567  9141 raft_consensus.cc:2468] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:22.212108  9353 tablet_bootstrap.cc:492] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:22.213030  9353 ts_tablet_manager.cc:1403] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.111s	user 0.035s	sys 0.057s
I20260709 13:09:22.213522  9180 leader_election.cc:304] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:22.214316  9356 raft_consensus.cc:359] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.214803  9355 raft_consensus.cc:2804] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:22.215133  9356 raft_consensus.cc:385] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.215518  9356 raft_consensus.cc:740] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.216353  9355 raft_consensus.cc:697] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Leader. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:22.216297  9356 consensus_queue.cc:260] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.216563  9066 raft_consensus.cc:2468] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:22.217576  9355 consensus_queue.cc:237] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.219089  9353 raft_consensus.cc:359] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.224565  9353 raft_consensus.cc:385] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.224888  9353 raft_consensus.cc:740] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.225687  9353 consensus_queue.cc:260] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.218886  9356 ts_tablet_manager.cc:1434] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:09:22.227664  9356 tablet_bootstrap.cc:492] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:22.227824  9353 ts_tablet_manager.cc:1434] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:09:22.228796  9353 tablet_bootstrap.cc:492] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:22.232000  8963 catalog_manager.cc:5697] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 reported cstate change: term changed from 0 to 1, leader changed from <none> to dad482c12e774a2a8df70dc54dde1822 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "dad482c12e774a2a8df70dc54dde1822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:22.238817  9356 tablet_bootstrap.cc:654] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.246572  9359 tablet_bootstrap.cc:492] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:22.247182  9359 ts_tablet_manager.cc:1403] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.115s	user 0.019s	sys 0.064s
I20260709 13:09:22.248742  9353 tablet_bootstrap.cc:654] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.250435  9359 raft_consensus.cc:359] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.251235  9359 raft_consensus.cc:385] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.251559  9359 raft_consensus.cc:740] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.252485  9359 consensus_queue.cc:260] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.255252  9359 ts_tablet_manager.cc:1434] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:09:22.257946  9359 tablet_bootstrap.cc:492] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:22.266530  9353 tablet_bootstrap.cc:492] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:22.266783  9359 tablet_bootstrap.cc:654] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.267110  9353 ts_tablet_manager.cc:1403] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.010s
I20260709 13:09:22.268908  9356 tablet_bootstrap.cc:492] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:22.269482  9356 ts_tablet_manager.cc:1403] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.016s
I20260709 13:09:22.270139  9353 raft_consensus.cc:359] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.270861  9353 raft_consensus.cc:385] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.271117  9353 raft_consensus.cc:740] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.271826  9353 consensus_queue.cc:260] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.272984  9355 raft_consensus.cc:493] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.273447  9355 raft_consensus.cc:515] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.273998  9353 ts_tablet_manager.cc:1434] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:09:22.275156  9353 tablet_bootstrap.cc:492] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822: Bootstrap starting.
I20260709 13:09:22.275601  9363 raft_consensus.cc:493] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.276158  9363 raft_consensus.cc:515] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.273813  9356 raft_consensus.cc:359] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.277756  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:22.277990  9356 raft_consensus.cc:385] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.278396  9363 leader_election.cc:290] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:22.278504  9356 raft_consensus.cc:740] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.278631  9141 raft_consensus.cc:2468] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:22.279387  9356 consensus_queue.cc:260] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.279975  9379 raft_consensus.cc:493] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.280297  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:22.280509  9379 raft_consensus.cc:515] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.281008  9066 raft_consensus.cc:2468] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:22.281415  9353 tablet_bootstrap.cc:654] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.281958  9356 ts_tablet_manager.cc:1434] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:09:22.282914  9379 leader_election.cc:290] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:22.283062  9356 tablet_bootstrap.cc:492] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:22.285430  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:22.286159  9215 raft_consensus.cc:2468] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 0.
I20260709 13:09:22.287992  9180 leader_election.cc:304] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:22.288265  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:22.288998  9140 raft_consensus.cc:2468] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 0.
I20260709 13:09:22.289973  9356 tablet_bootstrap.cc:654] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.290081  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ec08d78bd0d4e0ca1c0f69ebc42f4cd" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:22.290689  9032 leader_election.cc:304] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.291265  9141 raft_consensus.cc:2468] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:22.275612  9355 leader_election.cc:290] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:22.293411  9355 raft_consensus.cc:493] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.294067  9355 raft_consensus.cc:515] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.292219  9311 raft_consensus.cc:2804] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:22.296499  9311 raft_consensus.cc:493] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.296836  9311 raft_consensus.cc:3060] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.297952  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ec08d78bd0d4e0ca1c0f69ebc42f4cd" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:22.299015  9066 raft_consensus.cc:2468] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:22.305305  9178 leader_election.cc:304] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.307173  9379 raft_consensus.cc:2804] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:22.307902  9379 raft_consensus.cc:493] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.312589  9379 raft_consensus.cc:3060] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.329018  9355 leader_election.cc:290] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:22.333787  9355 raft_consensus.cc:2804] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:22.336371  9355 raft_consensus.cc:493] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.330250  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4451c98b52b407cb8ef9af4abcf8426" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:22.337436  9363 raft_consensus.cc:493] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.343484  9355 raft_consensus.cc:3060] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.343066  9363 raft_consensus.cc:515] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.343063  9066 raft_consensus.cc:2468] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:22.346093  9178 leader_election.cc:304] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.347991  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e9f167f21457eb5bd7ed9151445a1" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:22.348290  9363 leader_election.cc:290] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:22.351016  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e9f167f21457eb5bd7ed9151445a1" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:22.352008  9215 raft_consensus.cc:2468] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 0.
I20260709 13:09:22.353829  9032 leader_election.cc:304] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.357185  9391 raft_consensus.cc:2804] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:22.357981  9391 raft_consensus.cc:493] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.358875  9391 raft_consensus.cc:3060] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:22.357913  9180 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.130:34911, user_credentials={real_user=slave}} blocked reactor thread for 70080us
I20260709 13:09:22.361887  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4451c98b52b407cb8ef9af4abcf8426" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:22.362681  9141 raft_consensus.cc:2468] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:22.370268  9391 raft_consensus.cc:515] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.359642  9311 raft_consensus.cc:515] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.357100  9388 raft_consensus.cc:493] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.379984  9391 leader_election.cc:290] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Requested vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:22.379987  9311 leader_election.cc:290] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Requested vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:22.380295  9388 raft_consensus.cc:515] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.402115  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e9f167f21457eb5bd7ed9151445a1" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
W20260709 13:09:22.404269  9031 leader_election.cc:343] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:22.404788  9180 leader_election.cc:343] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:22.405431  9388 leader_election.cc:290] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:22.412865  9388 raft_consensus.cc:2804] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:22.412747  9326 consensus_queue.cc:1048] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:22.413419  9388 raft_consensus.cc:493] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.413664  9353 tablet_bootstrap.cc:492] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822: No bootstrap required, opened a new log
I20260709 13:09:22.413811  9388 raft_consensus.cc:3060] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.414547  9353 ts_tablet_manager.cc:1403] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822: Time spent bootstrapping tablet: real 0.140s	user 0.033s	sys 0.085s
I20260709 13:09:22.414302  9355 raft_consensus.cc:515] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.406735  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e9f167f21457eb5bd7ed9151445a1" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:22.409814  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:22.385667  9379 raft_consensus.cc:515] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.419528  9215 raft_consensus.cc:3060] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.420116  9214 raft_consensus.cc:2393] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fcb5f9589eb7482cab677f08b73602a1 in current term 1: Already voted for candidate dad482c12e774a2a8df70dc54dde1822 in this term.
I20260709 13:09:22.421054  9379 leader_election.cc:290] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:22.422045  9356 tablet_bootstrap.cc:492] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:22.417929  9353 raft_consensus.cc:359] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.422662  9356 ts_tablet_manager.cc:1403] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.140s	user 0.064s	sys 0.056s
I20260709 13:09:22.422864  9353 raft_consensus.cc:385] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.423225  9353 raft_consensus.cc:740] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.423295  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:22.423974  9141 raft_consensus.cc:3060] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.424044  9353 consensus_queue.cc:260] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.418696  9359 tablet_bootstrap.cc:492] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
W20260709 13:09:22.411672  9178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.71.129:37425, user_credentials={real_user=slave}} blocked reactor thread for 65723.7us
I20260709 13:09:22.425284  9359 ts_tablet_manager.cc:1403] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.168s	user 0.007s	sys 0.027s
I20260709 13:09:22.426468  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e9f167f21457eb5bd7ed9151445a1" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:22.427435  9066 raft_consensus.cc:2393] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dad482c12e774a2a8df70dc54dde1822 in current term 1: Already voted for candidate fcb5f9589eb7482cab677f08b73602a1 in this term.
I20260709 13:09:22.430202  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:22.431283  9066 raft_consensus.cc:2393] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dad482c12e774a2a8df70dc54dde1822 in current term 1: Already voted for candidate fcb5f9589eb7482cab677f08b73602a1 in this term.
I20260709 13:09:22.431625  9353 ts_tablet_manager.cc:1434] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:09:22.430204  9359 raft_consensus.cc:359] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.433163  9359 raft_consensus.cc:385] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.433502  9359 raft_consensus.cc:740] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.434252  9359 consensus_queue.cc:260] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.434394  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ec08d78bd0d4e0ca1c0f69ebc42f4cd" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:22.435243  9066 raft_consensus.cc:3060] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.436765  9359 ts_tablet_manager.cc:1434] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:09:22.437908  9359 tablet_bootstrap.cc:492] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:22.445044  9355 leader_election.cc:290] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:22.445731  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ec08d78bd0d4e0ca1c0f69ebc42f4cd" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:22.446316  9359 tablet_bootstrap.cc:654] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.446381  9140 raft_consensus.cc:3060] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.447048  9178 leader_election.cc:304] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822; no voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:22.451057  9356 raft_consensus.cc:359] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.454677  9066 raft_consensus.cc:2468] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:22.456717  9140 raft_consensus.cc:2468] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:22.457464  9215 raft_consensus.cc:2468] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 1.
I20260709 13:09:22.458935  9139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:22.459537  9355 raft_consensus.cc:2749] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:22.460460  9138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e9f167f21457eb5bd7ed9151445a1" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:22.461150  9138 raft_consensus.cc:3060] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.456508  9178 leader_election.cc:304] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.464730  9355 raft_consensus.cc:2804] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:22.451795  9356 raft_consensus.cc:385] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.465487  9356 raft_consensus.cc:740] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.465539  9355 raft_consensus.cc:697] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Leader. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:22.466347  9356 consensus_queue.cc:260] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.467674  9326 consensus_queue.cc:1048] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:22.467326  9355 consensus_queue.cc:237] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.472921  9141 raft_consensus.cc:2468] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:22.474774  9180 leader_election.cc:304] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:22.475752  9388 raft_consensus.cc:515] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.476622  9390 raft_consensus.cc:2804] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:22.477363  9356 ts_tablet_manager.cc:1434] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.001s
I20260709 13:09:22.479373  9032 leader_election.cc:304] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.480755  9393 raft_consensus.cc:2804] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:22.481262  9393 raft_consensus.cc:697] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Becoming Leader. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:22.477592  9390 raft_consensus.cc:697] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Leader. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:22.482148  9393 consensus_queue.cc:237] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.483521  9390 consensus_queue.cc:237] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.479502  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4451c98b52b407cb8ef9af4abcf8426" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:22.492067  9066 raft_consensus.cc:3060] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.495420  9031 leader_election.cc:304] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcb5f9589eb7482cab677f08b73602a1; no voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822
I20260709 13:09:22.496567  9392 raft_consensus.cc:2749] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:09:22.497107  9358 consensus_queue.cc:1048] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:22.497587  9138 raft_consensus.cc:2468] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 1.
I20260709 13:09:22.501008  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4451c98b52b407cb8ef9af4abcf8426" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:22.503895  9140 raft_consensus.cc:3060] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.509534  9359 tablet_bootstrap.cc:492] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:22.502743  9388 leader_election.cc:290] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:22.510139  9359 ts_tablet_manager.cc:1403] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.073s	user 0.028s	sys 0.032s
I20260709 13:09:22.502209  9356 tablet_bootstrap.cc:492] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1: Bootstrap starting.
I20260709 13:09:22.518656  9363 raft_consensus.cc:493] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.519306  9363 raft_consensus.cc:515] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.521591  9363 leader_election.cc:290] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:22.522624  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6200e2af384461eb93770f68b27cfa6" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822" is_pre_election: true
I20260709 13:09:22.523554  9214 raft_consensus.cc:2468] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 0.
I20260709 13:09:22.524931  9032 leader_election.cc:304] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.526033  9363 raft_consensus.cc:2804] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:22.526490  9363 raft_consensus.cc:493] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.526897  9363 raft_consensus.cc:3060] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.528800  9140 raft_consensus.cc:2468] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:22.529091  8961 catalog_manager.cc:5697] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "fcb5f9589eb7482cab677f08b73602a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:22.530156  9180 leader_election.cc:304] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:22.531062  9388 raft_consensus.cc:2804] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:22.531550  9388 raft_consensus.cc:697] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Leader. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:22.532460  9388 consensus_queue.cc:237] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.539687  9356 tablet_bootstrap.cc:654] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.540546  9066 raft_consensus.cc:2468] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:22.546761  9138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6200e2af384461eb93770f68b27cfa6" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:22.513207  9359 raft_consensus.cc:359] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.548941  9359 raft_consensus.cc:385] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.549268  9359 raft_consensus.cc:740] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.550030  9359 consensus_queue.cc:260] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.550994  9138 raft_consensus.cc:2468] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 0.
I20260709 13:09:22.558531  9363 raft_consensus.cc:515] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.560716  9363 leader_election.cc:290] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Requested vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:22.561473  9356 tablet_bootstrap.cc:492] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1: No bootstrap required, opened a new log
I20260709 13:09:22.561779  9213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6200e2af384461eb93770f68b27cfa6" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:22.562060  9356 ts_tablet_manager.cc:1403] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1: Time spent bootstrapping tablet: real 0.060s	user 0.011s	sys 0.002s
I20260709 13:09:22.562402  9213 raft_consensus.cc:3060] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.565491  9359 ts_tablet_manager.cc:1434] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.002s
I20260709 13:09:22.566635  9359 tablet_bootstrap.cc:492] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242: Bootstrap starting.
I20260709 13:09:22.568956  8960 catalog_manager.cc:5697] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 reported cstate change: term changed from 0 to 1, leader changed from <none> to dad482c12e774a2a8df70dc54dde1822 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "dad482c12e774a2a8df70dc54dde1822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:22.569859  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6200e2af384461eb93770f68b27cfa6" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:22.570371  9213 raft_consensus.cc:2468] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 1.
I20260709 13:09:22.570499  9140 raft_consensus.cc:3060] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.571794  9032 leader_election.cc:304] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.572731  9363 raft_consensus.cc:2804] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:22.573197  9363 raft_consensus.cc:697] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Becoming Leader. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:22.573344  9359 tablet_bootstrap.cc:654] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:22.574117  9363 consensus_queue.cc:237] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.571007  8960 catalog_manager.cc:5697] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 reported cstate change: term changed from 0 to 1, leader changed from <none> to dad482c12e774a2a8df70dc54dde1822 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "dad482c12e774a2a8df70dc54dde1822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:22.581144  9356 raft_consensus.cc:359] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.582000  9356 raft_consensus.cc:385] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.582273  9356 raft_consensus.cc:740] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.582931  9356 consensus_queue.cc:260] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.585371  9356 ts_tablet_manager.cc:1434] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 13:09:22.590437  9403 consensus_queue.cc:1048] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:22.591673  9140 raft_consensus.cc:2468] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 1.
I20260709 13:09:22.611905  8961 catalog_manager.cc:5697] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 reported cstate change: term changed from 0 to 1, leader changed from <none> to dad482c12e774a2a8df70dc54dde1822 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "dad482c12e774a2a8df70dc54dde1822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:22.619983  9359 tablet_bootstrap.cc:492] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242: No bootstrap required, opened a new log
I20260709 13:09:22.620671  9359 ts_tablet_manager.cc:1403] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242: Time spent bootstrapping tablet: real 0.054s	user 0.028s	sys 0.023s
I20260709 13:09:22.625875  8960 catalog_manager.cc:5697] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129). New cstate: current_term: 1 leader_uuid: "fcb5f9589eb7482cab677f08b73602a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:22.623441  9359 raft_consensus.cc:359] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.627542  9359 raft_consensus.cc:385] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:22.627864  9359 raft_consensus.cc:740] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Initialized, Role: FOLLOWER
I20260709 13:09:22.628664  9359 consensus_queue.cc:260] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.630784  9359 ts_tablet_manager.cc:1434] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
W20260709 13:09:22.645885  9012 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:22.699111  9388 raft_consensus.cc:493] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.700549  9388 raft_consensus.cc:515] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.704746  9388 leader_election.cc:290] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:22.706784  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f44bd882825450c93046d494e2f9264" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1" is_pre_election: true
I20260709 13:09:22.707582  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f44bd882825450c93046d494e2f9264" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242" is_pre_election: true
I20260709 13:09:22.708483  9066 raft_consensus.cc:2468] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:22.708806  9140 raft_consensus.cc:2468] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 0.
I20260709 13:09:22.710647  9178 leader_election.cc:304] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.711959  9388 raft_consensus.cc:2804] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:22.712494  9388 raft_consensus.cc:493] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.712967  9388 raft_consensus.cc:3060] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.724759  9388 raft_consensus.cc:515] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.726604  9388 leader_election.cc:290] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:22.727450  9066 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f44bd882825450c93046d494e2f9264" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:22.727636  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f44bd882825450c93046d494e2f9264" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:22.728171  9066 raft_consensus.cc:3060] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.728310  9140 raft_consensus.cc:3060] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:22.737596  9066 raft_consensus.cc:2468] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:22.738962  9178 leader_election.cc:304] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.739061  9140 raft_consensus.cc:2468] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 1.
I20260709 13:09:22.740077  9388 raft_consensus.cc:2804] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:22.740836  9388 raft_consensus.cc:697] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Leader. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:22.743114  9388 consensus_queue.cc:237] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.758249  8961 catalog_manager.cc:5697] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 reported cstate change: term changed from 0 to 1, leader changed from <none> to dad482c12e774a2a8df70dc54dde1822 (127.27.71.131). New cstate: current_term: 1 leader_uuid: "dad482c12e774a2a8df70dc54dde1822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:22.779464  9388 consensus_queue.cc:1048] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:22.794021  9388 consensus_queue.cc:1048] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:22.918071  9138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7389efc195ac42f6a444c254e095eab8"
dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
mode: GRACEFUL
new_leader_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:40254
I20260709 13:09:22.918776  9138 raft_consensus.cc:606] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Received request to transfer leadership to TS fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:22.926324  9138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcefbbc6c9ca4d0dbe09c57dec1b0ed2"
dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
mode: GRACEFUL
new_leader_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:40254
I20260709 13:09:22.927055  9138 raft_consensus.cc:606] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Received request to transfer leadership to TS fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:22.933374  9213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5a90f2828074148a8dd9ab98ade3083"
dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
mode: GRACEFUL
new_leader_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:42752
I20260709 13:09:22.934041  9213 raft_consensus.cc:606] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Received request to transfer leadership to TS fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:22.937395  9403 raft_consensus.cc:993] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242: : Instructing follower fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:22.937920  9358 raft_consensus.cc:1081] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Signalling peer fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:22.939924  9066 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7389efc195ac42f6a444c254e095eab8"
dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:37748
I20260709 13:09:22.940642  9066 raft_consensus.cc:493] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:22.941087  9066 raft_consensus.cc:3060] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:22.940644  9213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4451c98b52b407cb8ef9af4abcf8426"
dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
mode: GRACEFUL
new_leader_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:42752
I20260709 13:09:22.942874  9213 raft_consensus.cc:606] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Received request to transfer leadership to TS fcb5f9589eb7482cab677f08b73602a1
I20260709 13:09:22.943073  9388 consensus_queue.cc:1048] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:22.950973  9066 raft_consensus.cc:515] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.956137  9066 leader_election.cc:290] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Requested vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:22.961174  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7389efc195ac42f6a444c254e095eab8" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:22.961975  9215 raft_consensus.cc:3060] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:22.962512  9138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7389efc195ac42f6a444c254e095eab8" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:22.963215  9138 raft_consensus.cc:3055] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:22.963534  9138 raft_consensus.cc:740] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:22.964608  9138 consensus_queue.cc:260] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.966626  9213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a"
dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
mode: GRACEFUL
new_leader_uuid: "75f16c06c14045a5b104c416d3fd6242"
 from {username='slave'} at 127.0.0.1:42752
I20260709 13:09:22.967106  9388 consensus_queue.cc:1048] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:22.968745  9138 raft_consensus.cc:3060] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:22.970597  9393 consensus_queue.cc:1048] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:22.978492  9138 raft_consensus.cc:2468] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:22.967319  9213 raft_consensus.cc:606] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Received request to transfer leadership to TS 75f16c06c14045a5b104c416d3fd6242
I20260709 13:09:22.979875  9031 leader_election.cc:304] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:22.980798  9393 raft_consensus.cc:2804] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:22.981431  9393 raft_consensus.cc:697] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Becoming Leader. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:22.982740  9393 consensus_queue.cc:237] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:22.990561  9355 consensus_queue.cc:1048] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:22.995667  9393 consensus_queue.cc:1048] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:22.998325  9214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ec08d78bd0d4e0ca1c0f69ebc42f4cd"
dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
mode: GRACEFUL
new_leader_uuid: "75f16c06c14045a5b104c416d3fd6242"
 from {username='slave'} at 127.0.0.1:42752
I20260709 13:09:22.999050  9214 raft_consensus.cc:606] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Received request to transfer leadership to TS 75f16c06c14045a5b104c416d3fd6242
I20260709 13:09:23.006594  9064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a8e9f167f21457eb5bd7ed9151445a1"
dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
mode: GRACEFUL
new_leader_uuid: "75f16c06c14045a5b104c416d3fd6242"
 from {username='slave'} at 127.0.0.1:37766
I20260709 13:09:23.007319  9064 raft_consensus.cc:606] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Received request to transfer leadership to TS 75f16c06c14045a5b104c416d3fd6242
I20260709 13:09:23.009853  9215 raft_consensus.cc:2468] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:23.014667  9064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6200e2af384461eb93770f68b27cfa6"
dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
mode: GRACEFUL
new_leader_uuid: "75f16c06c14045a5b104c416d3fd6242"
 from {username='slave'} at 127.0.0.1:37766
I20260709 13:09:23.015373  9064 raft_consensus.cc:606] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Received request to transfer leadership to TS 75f16c06c14045a5b104c416d3fd6242
I20260709 13:09:23.017307  8961 catalog_manager.cc:5697] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 reported cstate change: term changed from 1 to 2, leader changed from 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130) to fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "fcb5f9589eb7482cab677f08b73602a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:23.026337  9379 consensus_queue.cc:1048] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:23.031204  9392 raft_consensus.cc:993] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1: : Instructing follower 75f16c06c14045a5b104c416d3fd6242 to start an election
I20260709 13:09:23.031769  9393 raft_consensus.cc:1081] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Signalling peer 75f16c06c14045a5b104c416d3fd6242 to start an election
I20260709 13:09:23.034298  9138 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a8e9f167f21457eb5bd7ed9151445a1"
dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
 from {username='slave'} at 127.0.0.1:40224
I20260709 13:09:23.035038  9138 raft_consensus.cc:493] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:23.035537  9138 raft_consensus.cc:3060] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.037561  9355 raft_consensus.cc:993] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822: : Instructing follower 75f16c06c14045a5b104c416d3fd6242 to start an election
I20260709 13:09:23.038141  9355 raft_consensus.cc:1081] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Signalling peer 75f16c06c14045a5b104c416d3fd6242 to start an election
I20260709 13:09:23.043249  9140 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ec08d78bd0d4e0ca1c0f69ebc42f4cd"
dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
 from {username='slave'} at 127.0.0.1:40238
I20260709 13:09:23.044013  9140 raft_consensus.cc:493] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:23.044497  9140 raft_consensus.cc:3060] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.046213  9138 raft_consensus.cc:515] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.052717  9065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e9f167f21457eb5bd7ed9151445a1" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:23.052819  9140 raft_consensus.cc:515] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.053488  9065 raft_consensus.cc:3055] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:23.053488  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e9f167f21457eb5bd7ed9151445a1" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:23.053972  9065 raft_consensus.cc:740] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:23.054230  9215 raft_consensus.cc:3060] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.055039  9065 consensus_queue.cc:260] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.056335  9065 raft_consensus.cc:3060] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.057981  9138 leader_election.cc:290] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 2 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:23.058081  9212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ec08d78bd0d4e0ca1c0f69ebc42f4cd" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:23.059034  9212 raft_consensus.cc:3055] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:23.059484  9212 raft_consensus.cc:740] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:23.055373  9140 leader_election.cc:290] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 2 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:23.060582  9212 consensus_queue.cc:260] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.058416  9064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ec08d78bd0d4e0ca1c0f69ebc42f4cd" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:23.062017  9064 raft_consensus.cc:3060] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.062258  9212 raft_consensus.cc:3060] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.064504  9355 consensus_queue.cc:1048] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:23.069356  9065 raft_consensus.cc:2468] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 2.
I20260709 13:09:23.071201  9104 leader_election.cc:304] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:23.071732  9215 raft_consensus.cc:2468] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 2.
I20260709 13:09:23.071843  9212 raft_consensus.cc:2468] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 2.
I20260709 13:09:23.072320  9358 raft_consensus.cc:2804] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:23.073266  9358 raft_consensus.cc:697] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Becoming Leader. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:23.074080  9064 raft_consensus.cc:2468] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 2.
I20260709 13:09:23.074184  9107 leader_election.cc:304] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:23.074471  9358 consensus_queue.cc:237] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.076725  9403 raft_consensus.cc:2804] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:23.077502  9403 raft_consensus.cc:697] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Becoming Leader. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:23.078801  9403 consensus_queue.cc:237] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.091133  8960 catalog_manager.cc:5697] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 reported cstate change: term changed from 1 to 2, leader changed from fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129) to 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "75f16c06c14045a5b104c416d3fd6242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:23.092832  9379 consensus_queue.cc:1048] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:23.093288  8960 catalog_manager.cc:5697] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 reported cstate change: term changed from 1 to 2, leader changed from dad482c12e774a2a8df70dc54dde1822 (127.27.71.131) to 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "75f16c06c14045a5b104c416d3fd6242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:23.102440  9379 raft_consensus.cc:993] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822: : Instructing follower fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:23.103011  9379 raft_consensus.cc:1081] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Signalling peer fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:23.104979  9064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4451c98b52b407cb8ef9af4abcf8426"
dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:37752
I20260709 13:09:23.105727  9064 raft_consensus.cc:493] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:23.106184  9064 raft_consensus.cc:3060] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.109285  9393 consensus_queue.cc:1048] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:23.116797  9064 raft_consensus.cc:515] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.119704  9064 leader_election.cc:290] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Requested vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:23.121634  9138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4451c98b52b407cb8ef9af4abcf8426" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:23.122398  9138 raft_consensus.cc:3060] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.122539  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4451c98b52b407cb8ef9af4abcf8426" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:23.123359  9215 raft_consensus.cc:3055] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:23.123715  9215 raft_consensus.cc:740] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:23.124866  9215 consensus_queue.cc:260] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.126413  9215 raft_consensus.cc:3060] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.128611  9363 consensus_queue.cc:1048] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:23.131286  9138 raft_consensus.cc:2468] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:23.133622  9388 consensus_queue.cc:1048] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:23.135661  9031 leader_election.cc:304] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:23.136612  9392 raft_consensus.cc:2804] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:23.137070  9392 raft_consensus.cc:697] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Becoming Leader. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:23.138295  9392 consensus_queue.cc:237] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.140364  9215 raft_consensus.cc:2468] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:23.148494  9388 consensus_queue.cc:1048] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:23.180410  9355 raft_consensus.cc:993] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822: : Instructing follower fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:23.181053  9355 raft_consensus.cc:1081] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Signalling peer fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:23.183818  9065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5a90f2828074148a8dd9ab98ade3083"
dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:37752
I20260709 13:09:23.184623  9065 raft_consensus.cc:493] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:23.185137  9065 raft_consensus.cc:3060] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.194573  9425 raft_consensus.cc:993] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242: : Instructing follower fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:23.195350  9426 raft_consensus.cc:1081] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Signalling peer fcb5f9589eb7482cab677f08b73602a1 to start an election
I20260709 13:09:23.197747  9066 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcefbbc6c9ca4d0dbe09c57dec1b0ed2"
dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
 from {username='slave'} at 127.0.0.1:37748
I20260709 13:09:23.198375  9066 raft_consensus.cc:493] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:23.198748  9066 raft_consensus.cc:3060] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.199618  9065 raft_consensus.cc:515] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.202739  9138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a90f2828074148a8dd9ab98ade3083" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:23.202876  9215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a90f2828074148a8dd9ab98ade3083" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:23.203392  9138 raft_consensus.cc:3060] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.203567  9215 raft_consensus.cc:3055] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:23.204135  9215 raft_consensus.cc:740] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:23.205446  9215 consensus_queue.cc:260] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.206575  9215 raft_consensus.cc:3060] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.206874  8963 catalog_manager.cc:5697] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 reported cstate change: term changed from 1 to 2, leader changed from dad482c12e774a2a8df70dc54dde1822 (127.27.71.131) to fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "fcb5f9589eb7482cab677f08b73602a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:23.208102  9065 leader_election.cc:290] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Requested vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:23.212256  9392 raft_consensus.cc:993] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1: : Instructing follower 75f16c06c14045a5b104c416d3fd6242 to start an election
I20260709 13:09:23.212882  9392 raft_consensus.cc:1081] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Signalling peer 75f16c06c14045a5b104c416d3fd6242 to start an election
I20260709 13:09:23.212234  9138 raft_consensus.cc:2468] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:23.214931  9140 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6200e2af384461eb93770f68b27cfa6"
dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
 from {username='slave'} at 127.0.0.1:40224
I20260709 13:09:23.215363  9031 leader_election.cc:304] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:23.215651  9140 raft_consensus.cc:493] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:23.216203  9140 raft_consensus.cc:3060] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.216533  9431 raft_consensus.cc:2804] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:23.217082  9431 raft_consensus.cc:697] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Becoming Leader. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:23.218103  9431 consensus_queue.cc:237] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.225936  9140 raft_consensus.cc:515] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.227172  9215 raft_consensus.cc:2468] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:23.229403  9064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6200e2af384461eb93770f68b27cfa6" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:23.229511  9212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6200e2af384461eb93770f68b27cfa6" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:23.229959  9140 leader_election.cc:290] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 2 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:23.230180  9212 raft_consensus.cc:3060] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.231266  9066 raft_consensus.cc:515] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.232246  9064 raft_consensus.cc:3055] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:23.232651  9064 raft_consensus.cc:740] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:23.233585  9064 consensus_queue.cc:260] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.235270  9064 raft_consensus.cc:3060] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.234586  8960 catalog_manager.cc:5697] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 reported cstate change: term changed from 1 to 2, leader changed from dad482c12e774a2a8df70dc54dde1822 (127.27.71.131) to fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "fcb5f9589eb7482cab677f08b73602a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:23.236164  9140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefbbc6c9ca4d0dbe09c57dec1b0ed2" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:23.237617  9140 raft_consensus.cc:3055] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:23.238360  9214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcefbbc6c9ca4d0dbe09c57dec1b0ed2" candidate_uuid: "fcb5f9589eb7482cab677f08b73602a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:23.239015  9214 raft_consensus.cc:3060] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.237113  9066 leader_election.cc:290] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Requested vote from peers 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:23.241379  9140 raft_consensus.cc:740] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:23.242431  9140 consensus_queue.cc:260] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.244225  9140 raft_consensus.cc:3060] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.244254  9064 raft_consensus.cc:2468] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 2.
I20260709 13:09:23.245690  9104 leader_election.cc:304] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:23.246640  9426 raft_consensus.cc:2804] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:23.247107  9426 raft_consensus.cc:697] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Becoming Leader. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:23.248378  9214 raft_consensus.cc:2468] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:23.248137  9426 consensus_queue.cc:237] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.249902  9032 leader_election.cc:304] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:23.251398  9431 raft_consensus.cc:2804] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:23.251547  9212 raft_consensus.cc:2468] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 2.
I20260709 13:09:23.251986  9431 raft_consensus.cc:697] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Becoming Leader. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:23.252183  9140 raft_consensus.cc:2468] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcb5f9589eb7482cab677f08b73602a1 in term 2.
I20260709 13:09:23.253082  9431 consensus_queue.cc:237] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.258991  8960 catalog_manager.cc:5697] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 reported cstate change: term changed from 1 to 2, leader changed from fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129) to 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "75f16c06c14045a5b104c416d3fd6242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:23.261085  8961 catalog_manager.cc:5697] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 reported cstate change: term changed from 1 to 2, leader changed from 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130) to fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129). New cstate: current_term: 2 leader_uuid: "fcb5f9589eb7482cab677f08b73602a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:23.447053  9140 raft_consensus.cc:1275] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.448450  9392 consensus_queue.cc:1048] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.461846  9212 raft_consensus.cc:1275] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.463217  9431 consensus_queue.cc:1048] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.528661  9355 raft_consensus.cc:993] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822: : Instructing follower 75f16c06c14045a5b104c416d3fd6242 to start an election
I20260709 13:09:23.529452  9390 raft_consensus.cc:1081] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Signalling peer 75f16c06c14045a5b104c416d3fd6242 to start an election
I20260709 13:09:23.531129  9138 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a"
dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
 from {username='slave'} at 127.0.0.1:40238
I20260709 13:09:23.531638  9138 raft_consensus.cc:493] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:23.531901  9138 raft_consensus.cc:3060] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.537420  9138 raft_consensus.cc:515] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.539222  9138 leader_election.cc:290] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 2 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), dad482c12e774a2a8df70dc54dde1822 (127.27.71.131:36547)
I20260709 13:09:23.540148  9064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:23.540341  9212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe5b7bf832245ecb1471c18d72caa7a" candidate_uuid: "75f16c06c14045a5b104c416d3fd6242" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
I20260709 13:09:23.540786  9064 raft_consensus.cc:3060] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.540947  9212 raft_consensus.cc:3055] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:23.541342  9212 raft_consensus.cc:740] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:23.542130  9212 consensus_queue.cc:260] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.543220  9212 raft_consensus.cc:3060] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:23.547143  9064 raft_consensus.cc:2468] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 2.
I20260709 13:09:23.548101  9214 raft_consensus.cc:1275] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.548688  9104 leader_election.cc:304] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:23.549597  9426 raft_consensus.cc:2804] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:23.549695  9431 consensus_queue.cc:1048] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:23.550141  9426 raft_consensus.cc:697] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Becoming Leader. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:23.552731  9212 raft_consensus.cc:2468] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f16c06c14045a5b104c416d3fd6242 in term 2.
I20260709 13:09:23.552856  9064 raft_consensus.cc:1275] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Refusing update from remote peer 75f16c06c14045a5b104c416d3fd6242: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.551396  9426 consensus_queue.cc:237] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:23.555691  9215 raft_consensus.cc:1275] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Refusing update from remote peer 75f16c06c14045a5b104c416d3fd6242: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.557919  9403 consensus_queue.cc:1048] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:23.555711  9425 consensus_queue.cc:1048] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:09:23.568871  9138 raft_consensus.cc:1275] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.574577  9393 consensus_queue.cc:1048] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.574295  8961 catalog_manager.cc:5697] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 reported cstate change: term changed from 1 to 2, leader changed from dad482c12e774a2a8df70dc54dde1822 (127.27.71.131) to 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130). New cstate: current_term: 2 leader_uuid: "75f16c06c14045a5b104c416d3fd6242" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:23.577824  9066 raft_consensus.cc:1275] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Refusing update from remote peer 75f16c06c14045a5b104c416d3fd6242: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.583259  9425 consensus_queue.cc:1048] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.598266  9215 raft_consensus.cc:1275] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Refusing update from remote peer 75f16c06c14045a5b104c416d3fd6242: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.599778  9358 consensus_queue.cc:1048] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.604169  9138 raft_consensus.cc:1275] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.605651  9431 consensus_queue.cc:1048] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.617066  9215 raft_consensus.cc:1275] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.618379  9363 consensus_queue.cc:1048] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.638347  9215 raft_consensus.cc:1275] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Refusing update from remote peer 75f16c06c14045a5b104c416d3fd6242: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.639675  9358 consensus_queue.cc:1048] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.647655  9064 raft_consensus.cc:1275] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Refusing update from remote peer 75f16c06c14045a5b104c416d3fd6242: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.651417  9358 consensus_queue.cc:1048] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.821897  9215 raft_consensus.cc:1275] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.823155  9363 consensus_queue.cc:1048] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.833005  9138 raft_consensus.cc:1275] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Refusing update from remote peer fcb5f9589eb7482cab677f08b73602a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:23.835044  9392 consensus_queue.cc:1048] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:24.051743  9064 raft_consensus.cc:1275] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Refusing update from remote peer 75f16c06c14045a5b104c416d3fd6242: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:24.053076  9358 consensus_queue.cc:1048] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:24.061599  9215 raft_consensus.cc:1275] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Refusing update from remote peer 75f16c06c14045a5b104c416d3fd6242: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:24.063252  9426 consensus_queue.cc:1048] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:26.023685  8963 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45784:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:09:26.047036  8963 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45784:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:09:26.110745 27934 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:26.297931  9488 tablet.cc:1724] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:26.303172  9461 log.cc:927] Time spent T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242: Append to log took a long time: real 0.156s	user 0.003s	sys 0.000s
W20260709 13:09:26.305830  9466 log.cc:927] Time spent T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
W20260709 13:09:26.334311  9462 log.cc:927] Time spent T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:09:26.338307  9482 tablet.cc:1724] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.378551  9487 tablet.cc:1724] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.414731  9500 tablet.cc:1724] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.424213  9140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b"
dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
mode: GRACEFUL
new_leader_uuid: "dad482c12e774a2a8df70dc54dde1822"
 from {username='slave'} at 127.0.0.1:46542
I20260709 13:09:26.425209  9140 raft_consensus.cc:606] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Received request to transfer leadership to TS dad482c12e774a2a8df70dc54dde1822
I20260709 13:09:26.437490  9499 tablet.cc:1724] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.418254  9493 tablet.cc:1724] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.441349  9495 tablet.cc:1724] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:09:26.454648  9496 log.cc:927] Time spent T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
I20260709 13:09:26.448386  9501 tablet.cc:1724] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.497188  9482 tablet.cc:1724] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.557608  9500 tablet.cc:1724] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.575052  9506 tablet.cc:1724] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.594157  9487 tablet.cc:1724] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.624553  9493 tablet.cc:1724] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.626057  9495 tablet.cc:1724] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.628278  9488 tablet.cc:1724] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.631903  9468 raft_consensus.cc:993] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242: : Instructing follower dad482c12e774a2a8df70dc54dde1822 to start an election
I20260709 13:09:26.632490  9468 raft_consensus.cc:1081] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Signalling peer dad482c12e774a2a8df70dc54dde1822 to start an election
I20260709 13:09:26.638232  9215 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b"
dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
 from {username='slave'} at 127.0.0.1:42744
I20260709 13:09:26.639488  9215 raft_consensus.cc:493] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:26.639835  9215 raft_consensus.cc:3060] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:26.662559  9508 tablet.cc:1724] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.665246  9482 tablet.cc:1724] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.670624  9501 tablet.cc:1724] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.676101  9510 tablet.cc:1724] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.648133  9500 tablet.cc:1724] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.681056  9064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62"
dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
mode: GRACEFUL
new_leader_uuid: "dad482c12e774a2a8df70dc54dde1822"
 from {username='slave'} at 127.0.0.1:33088
I20260709 13:09:26.688822  9064 raft_consensus.cc:606] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Received request to transfer leadership to TS dad482c12e774a2a8df70dc54dde1822
I20260709 13:09:26.690704 27934 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:09:26.680593  9513 tablet.cc:1724] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.691380 27934 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:26.703878  9493 tablet.cc:1724] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.705739  9506 tablet.cc:1724] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.702188  9499 tablet.cc:1724] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.708696  9488 tablet.cc:1724] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.709165  9487 tablet.cc:1724] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.740882  9215 raft_consensus.cc:515] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:26.743798  9064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:26.744469  9064 raft_consensus.cc:3060] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:26.750877  9506 tablet.cc:1724] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:09:26.752816  9064 raft_consensus.cc:2468] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 3.
I20260709 13:09:26.754256  9178 leader_election.cc:304] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dad482c12e774a2a8df70dc54dde1822, fcb5f9589eb7482cab677f08b73602a1; no voters: 
I20260709 13:09:26.755318  9507 raft_consensus.cc:2804] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:26.755792  9507 raft_consensus.cc:697] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 3 LEADER]: Becoming Leader. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:26.756722  9507 consensus_queue.cc:237] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:26.767098  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c1edfd2067f45a789de9f93b37d683b" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:26.767794  9141 raft_consensus.cc:3055] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:26.768162  9141 raft_consensus.cc:740] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 75f16c06c14045a5b104c416d3fd6242, State: Running, Role: LEADER
I20260709 13:09:26.769136  9141 consensus_queue.cc:260] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:26.770350  9141 raft_consensus.cc:3060] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:26.772042  9215 leader_election.cc:290] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 3 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:26.783305  9141 raft_consensus.cc:2468] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 3.
I20260709 13:09:26.796615  8963 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3: soft_deleted_table [id=03a7e953587b4d96be789232ee0c1fa9] alter complete (version 1)
I20260709 13:09:26.804368  8961 catalog_manager.cc:5697] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 reported cstate change: term changed from 2 to 3, leader changed from 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130) to dad482c12e774a2a8df70dc54dde1822 (127.27.71.131). New cstate: current_term: 3 leader_uuid: "dad482c12e774a2a8df70dc54dde1822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:26.915279  9485 raft_consensus.cc:993] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1: : Instructing follower dad482c12e774a2a8df70dc54dde1822 to start an election
I20260709 13:09:26.915971  9491 raft_consensus.cc:1081] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Signalling peer dad482c12e774a2a8df70dc54dde1822 to start an election
I20260709 13:09:26.917920  9215 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62"
dest_uuid: "dad482c12e774a2a8df70dc54dde1822"
 from {username='slave'} at 127.0.0.1:42732
I20260709 13:09:26.918532  9215 raft_consensus.cc:493] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:26.918855  9215 raft_consensus.cc:3060] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:26.924269  9215 raft_consensus.cc:515] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:26.926086  9215 leader_election.cc:290] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 3 election: Requested vote from peers fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425), 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911)
I20260709 13:09:26.927289  9064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fcb5f9589eb7482cab677f08b73602a1"
I20260709 13:09:26.927206  9141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b7213d7986410cb9d070e4e36e1b62" candidate_uuid: "dad482c12e774a2a8df70dc54dde1822" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75f16c06c14045a5b104c416d3fd6242"
I20260709 13:09:26.928086  9064 raft_consensus.cc:3055] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:26.928122  9141 raft_consensus.cc:3060] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:26.928570  9064 raft_consensus.cc:740] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fcb5f9589eb7482cab677f08b73602a1, State: Running, Role: LEADER
I20260709 13:09:26.929491  9064 consensus_queue.cc:260] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:26.931093  9064 raft_consensus.cc:3060] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:26.936906  9141 raft_consensus.cc:2468] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 3.
I20260709 13:09:26.938469  9180 leader_election.cc:304] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f16c06c14045a5b104c416d3fd6242, dad482c12e774a2a8df70dc54dde1822; no voters: 
I20260709 13:09:26.939047  9064 raft_consensus.cc:2468] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad482c12e774a2a8df70dc54dde1822 in term 3.
I20260709 13:09:26.939606  9507 raft_consensus.cc:2804] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:26.940864  9507 raft_consensus.cc:697] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 3 LEADER]: Becoming Leader. State: Replica: dad482c12e774a2a8df70dc54dde1822, State: Running, Role: LEADER
I20260709 13:09:26.941840  9507 consensus_queue.cc:237] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } }
I20260709 13:09:26.951171  8963 catalog_manager.cc:5697] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 reported cstate change: term changed from 2 to 3, leader changed from fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129) to dad482c12e774a2a8df70dc54dde1822 (127.27.71.131). New cstate: current_term: 3 leader_uuid: "dad482c12e774a2a8df70dc54dde1822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad482c12e774a2a8df70dc54dde1822" member_type: VOTER last_known_addr { host: "127.27.71.131" port: 36547 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:27.229583  9064 raft_consensus.cc:1275] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 3 FOLLOWER]: Refusing update from remote peer dad482c12e774a2a8df70dc54dde1822: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.230757  9507 consensus_queue.cc:1048] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.242990  9141 raft_consensus.cc:1275] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 3 FOLLOWER]: Refusing update from remote peer dad482c12e774a2a8df70dc54dde1822: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.244508  9507 consensus_queue.cc:1048] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.355316  9141 raft_consensus.cc:1275] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 3 FOLLOWER]: Refusing update from remote peer dad482c12e774a2a8df70dc54dde1822: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.356616  9507 consensus_queue.cc:1048] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f16c06c14045a5b104c416d3fd6242" member_type: VOTER last_known_addr { host: "127.27.71.130" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.365434  9064 raft_consensus.cc:1275] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 3 FOLLOWER]: Refusing update from remote peer dad482c12e774a2a8df70dc54dde1822: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.367758  9523 consensus_queue.cc:1048] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb5f9589eb7482cab677f08b73602a1" member_type: VOTER last_known_addr { host: "127.27.71.129" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.758751 27934 tablet_server.cc:179] TabletServer@127.27.71.129:0 shutting down...
W20260709 13:09:27.786854  9104 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 16 similar messages]
W20260709 13:09:27.793605  9104 consensus_peers.cc:597] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 -> Peer fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425): Couldn't send request to peer fcb5f9589eb7482cab677f08b73602a1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:27.802126 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:27.803260 27934 tablet_replica.cc:333] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.804169 27934 raft_consensus.cc:2243] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:27.804740 27934 raft_consensus.cc:2272] T c6200e2af384461eb93770f68b27cfa6 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.807032 27934 tablet_replica.cc:333] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.807693 27934 raft_consensus.cc:2243] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:27.808197 27934 raft_consensus.cc:2272] T 0f44bd882825450c93046d494e2f9264 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.810282 27934 tablet_replica.cc:333] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.810916 27934 raft_consensus.cc:2243] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:27.811739 27934 raft_consensus.cc:2272] T d4451c98b52b407cb8ef9af4abcf8426 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.813867 27934 tablet_replica.cc:333] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.814718 27934 raft_consensus.cc:2243] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:27.815481 27934 raft_consensus.cc:2272] T f5a90f2828074148a8dd9ab98ade3083 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.818063 27934 tablet_replica.cc:333] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.819026 27934 raft_consensus.cc:2243] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:27.819593 27934 raft_consensus.cc:2272] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.822613 27934 tablet_replica.cc:333] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.823693 27934 raft_consensus.cc:2243] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:27.824287 27934 raft_consensus.cc:2272] T 7a8e9f167f21457eb5bd7ed9151445a1 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.826380 27934 tablet_replica.cc:333] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.826928 27934 raft_consensus.cc:2243] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:27.827332 27934 raft_consensus.cc:2272] T 4fe5b7bf832245ecb1471c18d72caa7a P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.829494 27934 tablet_replica.cc:333] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.830003 27934 raft_consensus.cc:2243] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:27.830394 27934 raft_consensus.cc:2272] T 0a1f95384455460a840a18f5746aaae8 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.832726 27934 tablet_replica.cc:333] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.833316 27934 raft_consensus.cc:2243] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:27.834137 27934 raft_consensus.cc:2272] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.836452 27934 tablet_replica.cc:333] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.837095 27934 raft_consensus.cc:2243] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:27.838068 27934 raft_consensus.cc:2272] T ac243bc7f6234c60951698df8322d1c0 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.840587 27934 tablet_replica.cc:333] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.841259 27934 raft_consensus.cc:2243] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:27.841750 27934 raft_consensus.cc:2272] T 7c1edfd2067f45a789de9f93b37d683b P fcb5f9589eb7482cab677f08b73602a1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.844741 27934 tablet_replica.cc:333] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.845813 27934 raft_consensus.cc:2243] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:27.846882 27934 raft_consensus.cc:2272] T 3e01e0329e284d0b9f423618165717b5 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.849519 27934 tablet_replica.cc:333] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.850179 27934 raft_consensus.cc:2243] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:27.850708 27934 raft_consensus.cc:2272] T d78d91e5ea574eeb99069dda72c76bac P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.853241 27934 tablet_replica.cc:333] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.853895 27934 raft_consensus.cc:2243] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:27.854354 27934 raft_consensus.cc:2272] T b1b7213d7986410cb9d070e4e36e1b62 P fcb5f9589eb7482cab677f08b73602a1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.857342 27934 tablet_replica.cc:333] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.858006 27934 raft_consensus.cc:2243] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:27.858526 27934 raft_consensus.cc:2272] T 7eb3a7f451bb463d87fb2f927470dd7f P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.860863 27934 tablet_replica.cc:333] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.861414 27934 raft_consensus.cc:2243] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:27.861876 27934 raft_consensus.cc:2272] T ff8fc36481524d908afb6d9debb715ef P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:27.864992 27934 tablet_replica.cc:333] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.865772 27934 raft_consensus.cc:2243] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:27.866871 27934 raft_consensus.cc:2272] T 7389efc195ac42f6a444c254e095eab8 P fcb5f9589eb7482cab677f08b73602a1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:27.868214  9178 consensus_peers.cc:597] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 -> Peer fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425): Couldn't send request to peer fcb5f9589eb7482cab677f08b73602a1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:27.869884 27934 tablet_replica.cc:333] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1: stopping tablet replica
I20260709 13:09:27.870537 27934 raft_consensus.cc:2243] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:27.871610 27934 raft_consensus.cc:2272] T 916527a7aa6340e092512d7175e79017 P fcb5f9589eb7482cab677f08b73602a1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:27.879082  9104 consensus_peers.cc:597] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 -> Peer fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425): Couldn't send request to peer fcb5f9589eb7482cab677f08b73602a1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:27.915521 27934 tablet_server.cc:196] TabletServer@127.27.71.129:0 shutdown complete.
W20260709 13:09:27.940781  9178 consensus_peers.cc:597] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 -> Peer fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425): Couldn't send request to peer fcb5f9589eb7482cab677f08b73602a1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:37425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:27.964465 27934 tablet_server.cc:179] TabletServer@127.27.71.130:0 shutting down...
W20260709 13:09:27.989356  9180 consensus_peers.cc:597] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 -> Peer 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911): Couldn't send request to peer 75f16c06c14045a5b104c416d3fd6242. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:28.002808 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:28.003669 27934 tablet_replica.cc:333] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.004508 27934 raft_consensus.cc:2243] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.005578 27934 raft_consensus.cc:2272] T c6200e2af384461eb93770f68b27cfa6 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.008404 27934 tablet_replica.cc:333] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.009045 27934 raft_consensus.cc:2243] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.009795 27934 raft_consensus.cc:2272] T 7a8e9f167f21457eb5bd7ed9151445a1 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.011868 27934 tablet_replica.cc:333] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.012529 27934 raft_consensus.cc:2243] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.013247 27934 raft_consensus.cc:2272] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.015264 27934 tablet_replica.cc:333] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.015828 27934 raft_consensus.cc:2243] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.016335 27934 raft_consensus.cc:2272] T 0f44bd882825450c93046d494e2f9264 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.018342 27934 tablet_replica.cc:333] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.018857 27934 raft_consensus.cc:2243] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.019333 27934 raft_consensus.cc:2272] T d4451c98b52b407cb8ef9af4abcf8426 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.021389 27934 tablet_replica.cc:333] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.021915 27934 raft_consensus.cc:2243] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.022301 27934 raft_consensus.cc:2272] T f5a90f2828074148a8dd9ab98ade3083 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.024363 27934 tablet_replica.cc:333] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.024891 27934 raft_consensus.cc:2243] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.025619 27934 raft_consensus.cc:2272] T 4fe5b7bf832245ecb1471c18d72caa7a P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.027575 27934 tablet_replica.cc:333] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.028082 27934 raft_consensus.cc:2243] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.028858 27934 raft_consensus.cc:2272] T 0a1f95384455460a840a18f5746aaae8 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.031162 27934 tablet_replica.cc:333] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.031699 27934 raft_consensus.cc:2243] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.032174 27934 raft_consensus.cc:2272] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.034314 27934 tablet_replica.cc:333] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.034837 27934 raft_consensus.cc:2243] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.035221 27934 raft_consensus.cc:2272] T b1b7213d7986410cb9d070e4e36e1b62 P 75f16c06c14045a5b104c416d3fd6242 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.038007 27934 tablet_replica.cc:333] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.038570 27934 raft_consensus.cc:2243] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.039383 27934 raft_consensus.cc:2272] T 7eb3a7f451bb463d87fb2f927470dd7f P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.041489 27934 tablet_replica.cc:333] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.041991 27934 raft_consensus.cc:2243] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.042471 27934 raft_consensus.cc:2272] T 3e01e0329e284d0b9f423618165717b5 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.044836 27934 tablet_replica.cc:333] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.045354 27934 raft_consensus.cc:2243] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.045939 27934 raft_consensus.cc:2272] T d78d91e5ea574eeb99069dda72c76bac P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.048148 27934 tablet_replica.cc:333] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.048660 27934 raft_consensus.cc:2243] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.049043 27934 raft_consensus.cc:2272] T ac243bc7f6234c60951698df8322d1c0 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.051227 27934 tablet_replica.cc:333] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.051748 27934 raft_consensus.cc:2243] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.052238 27934 raft_consensus.cc:2272] T 7c1edfd2067f45a789de9f93b37d683b P 75f16c06c14045a5b104c416d3fd6242 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.055080 27934 tablet_replica.cc:333] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.055828 27934 raft_consensus.cc:2243] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.056864 27934 raft_consensus.cc:2272] T ff8fc36481524d908afb6d9debb715ef P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.059204 27934 tablet_replica.cc:333] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.059734 27934 raft_consensus.cc:2243] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.060227 27934 raft_consensus.cc:2272] T 7389efc195ac42f6a444c254e095eab8 P 75f16c06c14045a5b104c416d3fd6242 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.064860 27934 tablet_replica.cc:333] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242: stopping tablet replica
I20260709 13:09:28.065434 27934 raft_consensus.cc:2243] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.065925 27934 raft_consensus.cc:2272] T 916527a7aa6340e092512d7175e79017 P 75f16c06c14045a5b104c416d3fd6242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.112337 27934 tablet_server.cc:196] TabletServer@127.27.71.130:0 shutdown complete.
W20260709 13:09:28.147073  9178 consensus_peers.cc:597] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 -> Peer fcb5f9589eb7482cab677f08b73602a1 (127.27.71.129:37425): Couldn't send request to peer fcb5f9589eb7482cab677f08b73602a1. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.129:37425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:28.154318 27934 tablet_server.cc:179] TabletServer@127.27.71.131:0 shutting down...
I20260709 13:09:28.185623 27934 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:09:28.186443 27934 tablet_replica.cc:333] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.187326 27934 raft_consensus.cc:2243] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.187878 27934 raft_consensus.cc:2272] T c6200e2af384461eb93770f68b27cfa6 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.190258 27934 tablet_replica.cc:333] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.190842 27934 raft_consensus.cc:2243] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.191295 27934 raft_consensus.cc:2272] T 8ec08d78bd0d4e0ca1c0f69ebc42f4cd P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.193945 27934 tablet_replica.cc:333] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.194739 27934 raft_consensus.cc:2243] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.195354 27934 raft_consensus.cc:2272] T 7a8e9f167f21457eb5bd7ed9151445a1 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.197708 27934 tablet_replica.cc:333] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.198252 27934 raft_consensus.cc:2243] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.198988 27934 raft_consensus.cc:2272] T 0f44bd882825450c93046d494e2f9264 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.201017 27934 tablet_replica.cc:333] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.201594 27934 raft_consensus.cc:2243] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.201995 27934 raft_consensus.cc:2272] T d4451c98b52b407cb8ef9af4abcf8426 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.204005 27934 tablet_replica.cc:333] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.204557 27934 raft_consensus.cc:2243] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.204955 27934 raft_consensus.cc:2272] T f5a90f2828074148a8dd9ab98ade3083 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.207557 27934 tablet_replica.cc:333] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.208340 27934 raft_consensus.cc:2243] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.208940 27934 raft_consensus.cc:2272] T 4fe5b7bf832245ecb1471c18d72caa7a P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:09:28.210880  9180 consensus_peers.cc:597] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 -> Peer 75f16c06c14045a5b104c416d3fd6242 (127.27.71.130:34911): Couldn't send request to peer 75f16c06c14045a5b104c416d3fd6242. Status: Network error: Client connection negotiation failed: client connection to 127.27.71.130:34911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:28.211695 27934 tablet_replica.cc:333] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.212409 27934 raft_consensus.cc:2243] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.212870 27934 raft_consensus.cc:2272] T 0a1f95384455460a840a18f5746aaae8 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.214974 27934 tablet_replica.cc:333] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.215528 27934 raft_consensus.cc:2243] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.216004 27934 raft_consensus.cc:2272] T bcefbbc6c9ca4d0dbe09c57dec1b0ed2 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.218144 27934 tablet_replica.cc:333] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.218796 27934 raft_consensus.cc:2243] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.219241 27934 raft_consensus.cc:2272] T ac243bc7f6234c60951698df8322d1c0 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.221607 27934 tablet_replica.cc:333] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.222191 27934 raft_consensus.cc:2243] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.222973 27934 raft_consensus.cc:2272] T 7c1edfd2067f45a789de9f93b37d683b P dad482c12e774a2a8df70dc54dde1822 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.225651 27934 tablet_replica.cc:333] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.226217 27934 raft_consensus.cc:2243] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.226679 27934 raft_consensus.cc:2272] T 916527a7aa6340e092512d7175e79017 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.228832 27934 tablet_replica.cc:333] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.229388 27934 raft_consensus.cc:2243] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.230192 27934 raft_consensus.cc:2272] T b1b7213d7986410cb9d070e4e36e1b62 P dad482c12e774a2a8df70dc54dde1822 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.232753 27934 tablet_replica.cc:333] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.233332 27934 raft_consensus.cc:2243] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.233775 27934 raft_consensus.cc:2272] T 7eb3a7f451bb463d87fb2f927470dd7f P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.236013 27934 tablet_replica.cc:333] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.236579 27934 raft_consensus.cc:2243] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.237039 27934 raft_consensus.cc:2272] T 3e01e0329e284d0b9f423618165717b5 P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.239157 27934 tablet_replica.cc:333] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.239655 27934 raft_consensus.cc:2243] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.240541 27934 raft_consensus.cc:2272] T d78d91e5ea574eeb99069dda72c76bac P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.242770 27934 tablet_replica.cc:333] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.243304 27934 raft_consensus.cc:2243] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.243733 27934 raft_consensus.cc:2272] T 7389efc195ac42f6a444c254e095eab8 P dad482c12e774a2a8df70dc54dde1822 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.245780 27934 tablet_replica.cc:333] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822: stopping tablet replica
I20260709 13:09:28.246255 27934 raft_consensus.cc:2243] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:28.246691 27934 raft_consensus.cc:2272] T ff8fc36481524d908afb6d9debb715ef P dad482c12e774a2a8df70dc54dde1822 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.289211 27934 tablet_server.cc:196] TabletServer@127.27.71.131:0 shutdown complete.
I20260709 13:09:28.324497 27934 master.cc:562] Master@127.27.71.190:44109 shutting down...
I20260709 13:09:28.348824 27934 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:09:28.349372 27934 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:28.349687 27934 tablet_replica.cc:333] T 00000000000000000000000000000000 P 23cd56ebed5841a8ab5d236a3267abd3: stopping tablet replica
I20260709 13:09:28.371515 27934 master.cc:584] Master@127.27.71.190:44109 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (12019 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (25583 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (388766 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:09:28.432212 27934 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 432 messages since previous log ~15 seconds ago
I20260709 13:09:28.432436 27934 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 431 messages since previous log ~15 seconds ago
I20260709 13:09:28.432667 27934 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 428 messages since previous log ~15 seconds ago
I20260709 13:09:28.432914 27934 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~34 seconds ago
I20260709 13:09:28.433092 27934 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~34 seconds ago
I20260709 13:09:28.433350 27934 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 7 messages since previous log ~153 seconds ago
I20260709 13:09:28.433527 27934 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 18 messages since previous log ~282 seconds ago
I20260709 13:09:28.433746 27934 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 6 messages since previous log ~281 seconds ago
I20260709 13:09:28.434062 27934 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 6 messages since previous log ~0 seconds ago
I20260709 13:09:28.434377 27934 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 44 messages since previous log ~10 seconds ago
